@lenne.tech/nest-server 12.1.1 → 12.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (317) hide show
  1. package/.claude/rules/architecture.md +1 -0
  2. package/.claude/rules/better-auth.md +31 -4
  3. package/.claude/rules/configurable-features.md +8 -6
  4. package/.claude/rules/package-management.md +29 -2
  5. package/.claude/rules/persistence.md +27 -1
  6. package/.claude/rules/testing.md +80 -15
  7. package/.claude/rules/versioning.md +13 -2
  8. package/CLAUDE.md +7 -1
  9. package/FRAMEWORK-API.md +20 -4
  10. package/README.md +20 -2
  11. package/dist/config.env.js +2 -2
  12. package/dist/config.env.js.map +1 -1
  13. package/dist/core/common/helpers/graphql-ws-connection.helper.d.ts +4 -0
  14. package/dist/core/common/helpers/graphql-ws-connection.helper.js +17 -3
  15. package/dist/core/common/helpers/graphql-ws-connection.helper.js.map +1 -1
  16. package/dist/core/common/interfaces/server-options.interface.d.ts +49 -0
  17. package/dist/core/common/interfaces/service-options.interface.d.ts +3 -0
  18. package/dist/core/common/persistence/core-persistence.module.js +12 -2
  19. package/dist/core/common/persistence/core-persistence.module.js.map +1 -1
  20. package/dist/core/common/persistence/data-source.runtime.js +4 -2
  21. package/dist/core/common/persistence/data-source.runtime.js.map +1 -1
  22. package/dist/core/common/persistence/data-store.interface.d.ts +24 -9
  23. package/dist/core/common/persistence/id-format.d.ts +2 -0
  24. package/dist/core/common/persistence/id-format.js +14 -0
  25. package/dist/core/common/persistence/id-format.js.map +1 -1
  26. package/dist/core/common/persistence/model-descriptor.interface.d.ts +6 -0
  27. package/dist/core/common/persistence/model-metadata.registry.d.ts +2 -1
  28. package/dist/core/common/persistence/model-metadata.registry.js +20 -2
  29. package/dist/core/common/persistence/model-metadata.registry.js.map +1 -1
  30. package/dist/core/common/persistence/mongo/sync-schema.plugin.d.ts +18 -0
  31. package/dist/core/common/persistence/mongo/sync-schema.plugin.js +368 -0
  32. package/dist/core/common/persistence/mongo/sync-schema.plugin.js.map +1 -0
  33. package/dist/core/common/persistence/mongoose-data-store.d.ts +19 -9
  34. package/dist/core/common/persistence/mongoose-data-store.js +107 -31
  35. package/dist/core/common/persistence/mongoose-data-store.js.map +1 -1
  36. package/dist/core/common/persistence/persisted-model.decorator.d.ts +3 -0
  37. package/dist/core/common/persistence/persisted-model.decorator.js +9 -0
  38. package/dist/core/common/persistence/persisted-model.decorator.js.map +1 -1
  39. package/dist/core/common/persistence/policies/sync.policy.d.ts +17 -0
  40. package/dist/core/common/persistence/policies/sync.policy.js +77 -0
  41. package/dist/core/common/persistence/policies/sync.policy.js.map +1 -0
  42. package/dist/core/common/persistence/sql/mysql/mysql-data-store.d.ts +12 -8
  43. package/dist/core/common/persistence/sql/mysql/mysql-data-store.js +82 -29
  44. package/dist/core/common/persistence/sql/mysql/mysql-data-store.js.map +1 -1
  45. package/dist/core/common/persistence/sql/mysql/mysql-schema.compiler.d.ts +10 -0
  46. package/dist/core/common/persistence/sql/mysql/mysql-schema.compiler.js +61 -4
  47. package/dist/core/common/persistence/sql/mysql/mysql-schema.compiler.js.map +1 -1
  48. package/dist/core/common/persistence/sql/mysql/mysql-schema.migrator.js +101 -4
  49. package/dist/core/common/persistence/sql/mysql/mysql-schema.migrator.js.map +1 -1
  50. package/dist/core/common/persistence/sql/postgres/postgres-schema.compiler.d.ts +10 -0
  51. package/dist/core/common/persistence/sql/postgres/postgres-schema.compiler.js +41 -2
  52. package/dist/core/common/persistence/sql/postgres/postgres-schema.compiler.js.map +1 -1
  53. package/dist/core/common/persistence/sql/postgres/postgres-schema.migrator.js +98 -10
  54. package/dist/core/common/persistence/sql/postgres/postgres-schema.migrator.js.map +1 -1
  55. package/dist/core/common/persistence/sql/sql-data-store.d.ts +38 -14
  56. package/dist/core/common/persistence/sql/sql-data-store.js +212 -38
  57. package/dist/core/common/persistence/sql/sql-data-store.js.map +1 -1
  58. package/dist/core/common/persistence/sql/sql-errors.d.ts +2 -1
  59. package/dist/core/common/persistence/sql/sql-errors.js +3 -1
  60. package/dist/core/common/persistence/sql/sql-errors.js.map +1 -1
  61. package/dist/core/common/persistence/sync-access.registry.d.ts +17 -0
  62. package/dist/core/common/persistence/sync-access.registry.js +8 -0
  63. package/dist/core/common/persistence/sync-access.registry.js.map +1 -0
  64. package/dist/core/common/persistence/sync-model.d.ts +29 -0
  65. package/dist/core/common/persistence/sync-model.js +138 -0
  66. package/dist/core/common/persistence/sync-model.js.map +1 -0
  67. package/dist/core/common/services/crud.service.d.ts +13 -1
  68. package/dist/core/common/services/crud.service.js +174 -16
  69. package/dist/core/common/services/crud.service.js.map +1 -1
  70. package/dist/core/modules/ai/core-ai-mcp.controller.d.ts +3 -0
  71. package/dist/core/modules/ai/core-ai-mcp.controller.js +29 -0
  72. package/dist/core/modules/ai/core-ai-mcp.controller.js.map +1 -1
  73. package/dist/core/modules/ai/core-ai.constants.d.ts +2 -0
  74. package/dist/core/modules/ai/core-ai.constants.js +2 -0
  75. package/dist/core/modules/ai/core-ai.constants.js.map +1 -1
  76. package/dist/core/modules/ai/core-ai.controller.d.ts +6 -0
  77. package/dist/core/modules/ai/core-ai.controller.js +31 -1
  78. package/dist/core/modules/ai/core-ai.controller.js.map +1 -1
  79. package/dist/core/modules/ai/core-ai.module.d.ts +2 -0
  80. package/dist/core/modules/ai/core-ai.module.js +8 -1
  81. package/dist/core/modules/ai/core-ai.module.js.map +1 -1
  82. package/dist/core/modules/ai/core-ai.resolver.d.ts +6 -0
  83. package/dist/core/modules/ai/core-ai.resolver.js +32 -0
  84. package/dist/core/modules/ai/core-ai.resolver.js.map +1 -1
  85. package/dist/core/modules/ai/helpers/ai-mcp-oauth-consent.helper.d.ts +65 -1
  86. package/dist/core/modules/ai/helpers/ai-mcp-oauth-consent.helper.js +6 -3
  87. package/dist/core/modules/ai/helpers/ai-mcp-oauth-consent.helper.js.map +1 -1
  88. package/dist/core/modules/ai/helpers/ai-mcp-oauth.helper.js +48 -1
  89. package/dist/core/modules/ai/helpers/ai-mcp-oauth.helper.js.map +1 -1
  90. package/dist/core/modules/ai/helpers/ai-tool-authorization.helper.d.ts +9 -0
  91. package/dist/core/modules/ai/helpers/ai-tool-authorization.helper.js +54 -0
  92. package/dist/core/modules/ai/helpers/ai-tool-authorization.helper.js.map +1 -0
  93. package/dist/core/modules/ai/index.d.ts +7 -0
  94. package/dist/core/modules/ai/index.js +6 -0
  95. package/dist/core/modules/ai/index.js.map +1 -1
  96. package/dist/core/modules/ai/inputs/core-ai-settings.input.d.ts +3 -0
  97. package/dist/core/modules/ai/inputs/core-ai-settings.input.js +31 -0
  98. package/dist/core/modules/ai/inputs/core-ai-settings.input.js.map +1 -0
  99. package/dist/core/modules/ai/interfaces/ai-tool.interface.d.ts +2 -0
  100. package/dist/core/modules/ai/models/core-ai-settings.model.d.ts +4 -0
  101. package/dist/core/modules/ai/models/core-ai-settings.model.js +30 -0
  102. package/dist/core/modules/ai/models/core-ai-settings.model.js.map +1 -0
  103. package/dist/core/modules/ai/models/core-ai-user-setting.model.d.ts +8 -0
  104. package/dist/core/modules/ai/models/core-ai-user-setting.model.js +45 -0
  105. package/dist/core/modules/ai/models/core-ai-user-setting.model.js.map +1 -0
  106. package/dist/core/modules/ai/services/core-ai-conversation.service.d.ts +4 -1
  107. package/dist/core/modules/ai/services/core-ai-conversation.service.js +16 -4
  108. package/dist/core/modules/ai/services/core-ai-conversation.service.js.map +1 -1
  109. package/dist/core/modules/ai/services/core-ai-mcp-oauth.service.js +6 -6
  110. package/dist/core/modules/ai/services/core-ai-mcp-oauth.service.js.map +1 -1
  111. package/dist/core/modules/ai/services/core-ai-mcp.service.d.ts +27 -2
  112. package/dist/core/modules/ai/services/core-ai-mcp.service.js +118 -11
  113. package/dist/core/modules/ai/services/core-ai-mcp.service.js.map +1 -1
  114. package/dist/core/modules/ai/services/core-ai-prompt-builder.service.js +9 -6
  115. package/dist/core/modules/ai/services/core-ai-prompt-builder.service.js.map +1 -1
  116. package/dist/core/modules/ai/services/core-ai-user-setting.service.d.ts +17 -0
  117. package/dist/core/modules/ai/services/core-ai-user-setting.service.js +72 -0
  118. package/dist/core/modules/ai/services/core-ai-user-setting.service.js.map +1 -0
  119. package/dist/core/modules/ai/services/core-ai.service.d.ts +8 -3
  120. package/dist/core/modules/ai/services/core-ai.service.js +68 -29
  121. package/dist/core/modules/ai/services/core-ai.service.js.map +1 -1
  122. package/dist/core/modules/better-auth/better-auth.config.js +3 -1
  123. package/dist/core/modules/better-auth/better-auth.config.js.map +1 -1
  124. package/dist/core/modules/better-auth/core-better-auth-email-verification.service.js +2 -2
  125. package/dist/core/modules/better-auth/core-better-auth-email-verification.service.js.map +1 -1
  126. package/dist/core/modules/better-auth/core-better-auth-path-guard.middleware.d.ts +8 -0
  127. package/dist/core/modules/better-auth/core-better-auth-path-guard.middleware.js +64 -0
  128. package/dist/core/modules/better-auth/core-better-auth-path-guard.middleware.js.map +1 -0
  129. package/dist/core/modules/better-auth/core-better-auth-rate-limit.middleware.js +2 -1
  130. package/dist/core/modules/better-auth/core-better-auth-rate-limit.middleware.js.map +1 -1
  131. package/dist/core/modules/better-auth/core-better-auth-rate-limiter.service.d.ts +1 -1
  132. package/dist/core/modules/better-auth/core-better-auth-rate-limiter.service.js +5 -3
  133. package/dist/core/modules/better-auth/core-better-auth-rate-limiter.service.js.map +1 -1
  134. package/dist/core/modules/better-auth/core-better-auth.module.js +2 -0
  135. package/dist/core/modules/better-auth/core-better-auth.module.js.map +1 -1
  136. package/dist/core/modules/better-auth/index.d.ts +1 -0
  137. package/dist/core/modules/better-auth/index.js +1 -0
  138. package/dist/core/modules/better-auth/index.js.map +1 -1
  139. package/dist/core/modules/error-code/error-codes.d.ts +72 -0
  140. package/dist/core/modules/error-code/error-codes.js +64 -0
  141. package/dist/core/modules/error-code/error-codes.js.map +1 -1
  142. package/dist/core/modules/migrate/sql-migration.helper.d.ts +15 -1
  143. package/dist/core/modules/migrate/sql-migration.helper.js +90 -5
  144. package/dist/core/modules/migrate/sql-migration.helper.js.map +1 -1
  145. package/dist/core/modules/sync/core-sync-base.controller.d.ts +7 -0
  146. package/dist/core/modules/sync/core-sync-base.controller.js +44 -0
  147. package/dist/core/modules/sync/core-sync-base.controller.js.map +1 -0
  148. package/dist/core/modules/sync/core-sync-cursor.d.ts +19 -0
  149. package/dist/core/modules/sync/core-sync-cursor.js +77 -0
  150. package/dist/core/modules/sync/core-sync-cursor.js.map +1 -0
  151. package/dist/core/modules/sync/core-sync-hints.controller.d.ts +4 -0
  152. package/dist/core/modules/sync/core-sync-hints.controller.js +55 -0
  153. package/dist/core/modules/sync/core-sync-hints.controller.js.map +1 -0
  154. package/dist/core/modules/sync/core-sync-hints.service.d.ts +28 -0
  155. package/dist/core/modules/sync/core-sync-hints.service.js +155 -0
  156. package/dist/core/modules/sync/core-sync-hints.service.js.map +1 -0
  157. package/dist/core/modules/sync/core-sync-infrastructure.models.d.ts +2 -0
  158. package/dist/core/modules/sync/core-sync-infrastructure.models.js +17 -0
  159. package/dist/core/modules/sync/core-sync-infrastructure.models.js.map +1 -0
  160. package/dist/core/modules/sync/core-sync-mutation.store.d.ts +43 -0
  161. package/dist/core/modules/sync/core-sync-mutation.store.js +88 -0
  162. package/dist/core/modules/sync/core-sync-mutation.store.js.map +1 -0
  163. package/dist/core/modules/sync/core-sync-state.store.d.ts +18 -0
  164. package/dist/core/modules/sync/core-sync-state.store.js +64 -0
  165. package/dist/core/modules/sync/core-sync-state.store.js.map +1 -0
  166. package/dist/core/modules/sync/core-sync.constants.d.ts +28 -0
  167. package/dist/core/modules/sync/core-sync.constants.js +23 -0
  168. package/dist/core/modules/sync/core-sync.constants.js.map +1 -0
  169. package/dist/core/modules/sync/core-sync.controller.d.ts +8 -0
  170. package/dist/core/modules/sync/core-sync.controller.js +79 -0
  171. package/dist/core/modules/sync/core-sync.controller.js.map +1 -0
  172. package/dist/core/modules/sync/core-sync.decorators.d.ts +12 -0
  173. package/dist/core/modules/sync/core-sync.decorators.js +28 -0
  174. package/dist/core/modules/sync/core-sync.decorators.js.map +1 -0
  175. package/dist/core/modules/sync/core-sync.module.d.ts +16 -0
  176. package/dist/core/modules/sync/core-sync.module.js +50 -0
  177. package/dist/core/modules/sync/core-sync.module.js.map +1 -0
  178. package/dist/core/modules/sync/core-sync.resolver.d.ts +9 -0
  179. package/dist/core/modules/sync/core-sync.resolver.js +46 -0
  180. package/dist/core/modules/sync/core-sync.resolver.js.map +1 -0
  181. package/dist/core/modules/sync/core-sync.service.d.ts +143 -0
  182. package/dist/core/modules/sync/core-sync.service.js +1237 -0
  183. package/dist/core/modules/sync/core-sync.service.js.map +1 -0
  184. package/dist/core/modules/sync/interfaces/core-sync.interfaces.d.ts +56 -0
  185. package/dist/core/modules/sync/interfaces/core-sync.interfaces.js +2 -0
  186. package/dist/core/modules/sync/interfaces/core-sync.interfaces.js.map +1 -0
  187. package/dist/core/modules/sync/models/core-sync-hint.model.d.ts +4 -0
  188. package/dist/core/modules/sync/models/core-sync-hint.model.js +30 -0
  189. package/dist/core/modules/sync/models/core-sync-hint.model.js.map +1 -0
  190. package/dist/core/modules/tenant/core-tenant.guard.js +14 -0
  191. package/dist/core/modules/tenant/core-tenant.guard.js.map +1 -1
  192. package/dist/core/modules/tenant/core-tenant.service.d.ts +21 -1
  193. package/dist/core/modules/tenant/core-tenant.service.js +153 -1
  194. package/dist/core/modules/tenant/core-tenant.service.js.map +1 -1
  195. package/dist/core.module.js +14 -0
  196. package/dist/core.module.js.map +1 -1
  197. package/dist/index.d.ts +16 -0
  198. package/dist/index.js +16 -0
  199. package/dist/index.js.map +1 -1
  200. package/dist/server/modules/error-code/error-codes.d.ts +8 -0
  201. package/dist/server/modules/sync-note/inputs/sync-note-create.input.d.ts +4 -0
  202. package/dist/server/modules/sync-note/inputs/sync-note-create.input.js +27 -0
  203. package/dist/server/modules/sync-note/inputs/sync-note-create.input.js.map +1 -0
  204. package/dist/server/modules/sync-note/inputs/sync-note.input.d.ts +6 -0
  205. package/dist/server/modules/sync-note/inputs/sync-note.input.js +37 -0
  206. package/dist/server/modules/sync-note/inputs/sync-note.input.js.map +1 -0
  207. package/dist/server/modules/sync-note/sync-note.model.d.ts +10 -0
  208. package/dist/server/modules/sync-note/sync-note.model.js +73 -0
  209. package/dist/server/modules/sync-note/sync-note.model.js.map +1 -0
  210. package/dist/server/modules/sync-note/sync-note.module.d.ts +2 -0
  211. package/dist/server/modules/sync-note/sync-note.module.js +21 -0
  212. package/dist/server/modules/sync-note/sync-note.module.js.map +1 -0
  213. package/dist/server/modules/sync-note/sync-note.service.d.ts +10 -0
  214. package/dist/server/modules/sync-note/sync-note.service.js +35 -0
  215. package/dist/server/modules/sync-note/sync-note.service.js.map +1 -0
  216. package/dist/test/test.helper.d.ts +1 -0
  217. package/dist/test/test.helper.js +9 -2
  218. package/dist/test/test.helper.js.map +1 -1
  219. package/docs/REQUEST-LIFECYCLE.md +4 -2
  220. package/docs/error-codes.md +14 -0
  221. package/docs/offline-sync-protocol.md +111 -68
  222. package/docs/security-overrides.md +14 -0
  223. package/migration-guides/12.0.x-to-12.1.x.md +0 -1
  224. package/migration-guides/12.1.x-to-12.2.x.md +154 -0
  225. package/migration-guides/12.2.x-to-12.3.x.md +440 -0
  226. package/package.json +23 -21
  227. package/src/config.env.ts +6 -2
  228. package/src/core/common/helpers/graphql-ws-connection.helper.ts +32 -4
  229. package/src/core/common/interfaces/server-options.interface.ts +229 -8
  230. package/src/core/common/interfaces/service-options.interface.ts +15 -0
  231. package/src/core/common/persistence/core-persistence.module.ts +21 -2
  232. package/src/core/common/persistence/data-source.runtime.ts +5 -2
  233. package/src/core/common/persistence/data-store.interface.ts +65 -12
  234. package/src/core/common/persistence/id-format.ts +26 -0
  235. package/src/core/common/persistence/model-descriptor.interface.ts +12 -0
  236. package/src/core/common/persistence/model-metadata.registry.ts +31 -2
  237. package/src/core/common/persistence/mongo/sync-schema.plugin.ts +508 -0
  238. package/src/core/common/persistence/mongoose-data-store.ts +171 -35
  239. package/src/core/common/persistence/persisted-model.decorator.ts +26 -0
  240. package/src/core/common/persistence/policies/sync.policy.ts +121 -0
  241. package/src/core/common/persistence/sql/mysql/mysql-data-store.ts +108 -30
  242. package/src/core/common/persistence/sql/mysql/mysql-schema.compiler.ts +107 -4
  243. package/src/core/common/persistence/sql/mysql/mysql-schema.migrator.ts +129 -3
  244. package/src/core/common/persistence/sql/postgres/postgres-schema.compiler.ts +74 -2
  245. package/src/core/common/persistence/sql/postgres/postgres-schema.migrator.ts +139 -7
  246. package/src/core/common/persistence/sql/sql-data-store.ts +325 -44
  247. package/src/core/common/persistence/sql/sql-errors.ts +5 -1
  248. package/src/core/common/persistence/sync-access.registry.ts +50 -0
  249. package/src/core/common/persistence/sync-model.ts +270 -0
  250. package/src/core/common/services/crud.service.ts +257 -19
  251. package/src/core/modules/ai/INTEGRATION-CHECKLIST.md +36 -15
  252. package/src/core/modules/ai/README.md +163 -25
  253. package/src/core/modules/ai/core-ai-mcp.controller.ts +48 -1
  254. package/src/core/modules/ai/core-ai.constants.ts +6 -0
  255. package/src/core/modules/ai/core-ai.controller.ts +36 -1
  256. package/src/core/modules/ai/core-ai.module.ts +12 -0
  257. package/src/core/modules/ai/core-ai.resolver.ts +36 -0
  258. package/src/core/modules/ai/helpers/ai-mcp-oauth-consent.helper.ts +35 -4
  259. package/src/core/modules/ai/helpers/ai-mcp-oauth.helper.ts +68 -1
  260. package/src/core/modules/ai/helpers/ai-tool-authorization.helper.ts +99 -0
  261. package/src/core/modules/ai/index.ts +18 -0
  262. package/src/core/modules/ai/inputs/core-ai-settings.input.ts +24 -0
  263. package/src/core/modules/ai/interfaces/ai-hook.interface.ts +1 -1
  264. package/src/core/modules/ai/interfaces/ai-tool.interface.ts +38 -26
  265. package/src/core/modules/ai/models/core-ai-mode.model.ts +1 -1
  266. package/src/core/modules/ai/models/core-ai-settings.model.ts +23 -0
  267. package/src/core/modules/ai/models/core-ai-tool-policy.model.ts +1 -1
  268. package/src/core/modules/ai/models/core-ai-user-setting.model.ts +46 -0
  269. package/src/core/modules/ai/services/core-ai-conversation.service.ts +31 -6
  270. package/src/core/modules/ai/services/core-ai-mcp-oauth.service.ts +12 -9
  271. package/src/core/modules/ai/services/core-ai-mcp.service.ts +206 -13
  272. package/src/core/modules/ai/services/core-ai-prompt-builder.service.ts +13 -6
  273. package/src/core/modules/ai/services/core-ai-user-setting.service.ts +85 -0
  274. package/src/core/modules/ai/services/core-ai.service.ts +134 -35
  275. package/src/core/modules/api-token/README.md +4 -2
  276. package/src/core/modules/better-auth/CUSTOMIZATION.md +1 -1
  277. package/src/core/modules/better-auth/INTEGRATION-CHECKLIST.md +7 -0
  278. package/src/core/modules/better-auth/README.md +32 -1
  279. package/src/core/modules/better-auth/better-auth.config.ts +12 -1
  280. package/src/core/modules/better-auth/core-better-auth-email-verification.service.ts +5 -2
  281. package/src/core/modules/better-auth/core-better-auth-path-guard.middleware.ts +85 -0
  282. package/src/core/modules/better-auth/core-better-auth-rate-limit.middleware.ts +5 -2
  283. package/src/core/modules/better-auth/core-better-auth-rate-limiter.service.ts +14 -7
  284. package/src/core/modules/better-auth/core-better-auth.module.ts +12 -2
  285. package/src/core/modules/better-auth/index.ts +1 -0
  286. package/src/core/modules/error-code/error-codes.ts +77 -0
  287. package/src/core/modules/migrate/sql-migration.helper.ts +141 -7
  288. package/src/core/modules/sync/INTEGRATION-CHECKLIST.md +95 -0
  289. package/src/core/modules/sync/README.md +198 -0
  290. package/src/core/modules/sync/SECURITY.md +65 -0
  291. package/src/core/modules/sync/core-sync-base.controller.ts +43 -0
  292. package/src/core/modules/sync/core-sync-cursor.ts +117 -0
  293. package/src/core/modules/sync/core-sync-hints.controller.ts +52 -0
  294. package/src/core/modules/sync/core-sync-hints.service.ts +178 -0
  295. package/src/core/modules/sync/core-sync-infrastructure.models.ts +40 -0
  296. package/src/core/modules/sync/core-sync-mutation.store.ts +146 -0
  297. package/src/core/modules/sync/core-sync-state.store.ts +89 -0
  298. package/src/core/modules/sync/core-sync.constants.ts +51 -0
  299. package/src/core/modules/sync/core-sync.controller.ts +68 -0
  300. package/src/core/modules/sync/core-sync.decorators.ts +84 -0
  301. package/src/core/modules/sync/core-sync.module.ts +75 -0
  302. package/src/core/modules/sync/core-sync.resolver.ts +35 -0
  303. package/src/core/modules/sync/core-sync.service.ts +1773 -0
  304. package/src/core/modules/sync/interfaces/core-sync.interfaces.ts +85 -0
  305. package/src/core/modules/sync/models/core-sync-hint.model.ts +18 -0
  306. package/src/core/modules/tenant/INTEGRATION-CHECKLIST.md +12 -1
  307. package/src/core/modules/tenant/README.md +49 -3
  308. package/src/core/modules/tenant/core-tenant.guard.ts +26 -2
  309. package/src/core/modules/tenant/core-tenant.service.ts +287 -2
  310. package/src/core.module.ts +25 -0
  311. package/src/index.ts +21 -0
  312. package/src/server/modules/sync-note/inputs/sync-note-create.input.ts +14 -0
  313. package/src/server/modules/sync-note/inputs/sync-note.input.ts +23 -0
  314. package/src/server/modules/sync-note/sync-note.model.ts +77 -0
  315. package/src/server/modules/sync-note/sync-note.module.ts +16 -0
  316. package/src/server/modules/sync-note/sync-note.service.ts +25 -0
  317. package/src/test/test.helper.ts +17 -3
@@ -39,6 +39,7 @@ import { DefaultBetterAuthResolver } from './better-auth.resolver.js';
39
39
  import { CoreBetterAuthApiMiddleware } from './core-better-auth-api.middleware.js';
40
40
  import { CoreBetterAuthChallengeService } from './core-better-auth-challenge.service.js';
41
41
  import { CoreBetterAuthEmailVerificationService } from './core-better-auth-email-verification.service.js';
42
+ import { CoreBetterAuthPathGuardMiddleware } from './core-better-auth-path-guard.middleware.js';
42
43
  import { CoreBetterAuthRateLimitMiddleware } from './core-better-auth-rate-limit.middleware.js';
43
44
  import { CoreBetterAuthRateLimiter } from './core-better-auth-rate-limiter.service.js';
44
45
  import { revokeApiTokensOfUser } from '../api-token/core-api-token.registry.js';
@@ -431,8 +432,10 @@ export class CoreBetterAuthModule implements NestModule, OnModuleInit {
431
432
  * 1. CoreBetterAuthMiddleware - Session validation and user mapping for all routes
432
433
  * Must run FIRST so that req.betterAuthSession is available for downstream middleware.
433
434
  * In JWT mode, this resolves the JWT to a real DB session (via getActiveSessionForUser).
434
- * 2. CoreBetterAuthRateLimitMiddleware - Rate limiting for auth endpoints
435
- * 3. CoreBetterAuthApiMiddleware - Forwards plugin endpoints (passkey, 2FA, etc.) to Better Auth's native handler
435
+ * 2. CoreBetterAuthPathGuardMiddleware - Refuses a non-canonical path under the base path (400), before
436
+ * anything below reads it
437
+ * 3. CoreBetterAuthRateLimitMiddleware - Rate limiting for auth endpoints
438
+ * 4. CoreBetterAuthApiMiddleware - Forwards plugin endpoints (passkey, 2FA, etc.) to Better Auth's native handler
436
439
  * Runs AFTER session middleware so it can use req.betterAuthSession.session.token
437
440
  * to authenticate requests in JWT mode.
438
441
  */
@@ -453,6 +456,13 @@ export class CoreBetterAuthModule implements NestModule, OnModuleInit {
453
456
  consumer.apply(CoreBetterAuthMiddleware).forRoutes('{*path}');
454
457
  CoreBetterAuthModule.logger.debug('CoreBetterAuthMiddleware registered for all routes');
455
458
 
459
+ // Refuse a non-canonical path (`.` / `..` segments, also encoded, backslashes, encoded slashes) before anything
460
+ // under the base path reads it: every check below reads the path as SENT, while Better-Auth routes the one
461
+ // `new URL()` resolved — `/callback/../sign-in/email` would pass the rate limiter as a callback and run as a
462
+ // sign-in. Registered unconditionally and FIRST, so it guards the limiter, the controller routes and the
463
+ // forwarding alike.
464
+ consumer.apply(CoreBetterAuthPathGuardMiddleware).forRoutes(`${basePath}/*path`);
465
+
456
466
  // Apply rate limiting to Better-Auth endpoints only.
457
467
  // Mirrors the "presence implies enabled" contract `CoreBetterAuthRateLimiter.configure()`
458
468
  // implements. Gating the MOUNT on `.enabled` alone made that fix unobservable: the limiter
@@ -34,6 +34,7 @@ export * from './core-better-auth-email-verification.service.js';
34
34
  export * from './core-better-auth-error-codes.helper.js';
35
35
  export * from './core-better-auth-migration-status.model.js';
36
36
  export * from './core-better-auth-models.js';
37
+ export * from './core-better-auth-path-guard.middleware.js';
37
38
  export * from './core-better-auth-rate-limit.middleware.js';
38
39
  export * from './core-better-auth-rate-limiter.service.js';
39
40
  export * from './core-better-auth-signup-validator.service.js';
@@ -22,6 +22,7 @@
22
22
  * - LTNS_0400-LTNS_0499: Resource errors
23
23
  * - LTNS_0500-LTNS_0599: File errors
24
24
  * - LTNS_0600-LTNS_0699: AI errors
25
+ * - LTNS_0700-LTNS_0799: Offline sync errors
25
26
  * - LTNS_0900-LTNS_0999: Internal errors
26
27
  *
27
28
  * @example
@@ -493,6 +494,15 @@ export const LtnsErrors = {
493
494
  },
494
495
  },
495
496
 
497
+ INVALID_REQUEST_PATH: {
498
+ code: 'LTNS_0305',
499
+ message: 'Invalid request path',
500
+ translations: {
501
+ de: 'Der Anfragepfad ist ungültig.',
502
+ en: 'The request path is invalid.',
503
+ },
504
+ },
505
+
496
506
  // =====================================================
497
507
  // Resource Errors (LTNS_0400-LTNS_0499)
498
508
  // =====================================================
@@ -515,6 +525,15 @@ export const LtnsErrors = {
515
525
  },
516
526
  },
517
527
 
528
+ VERSION_CONFLICT: {
529
+ code: 'LTNS_0402',
530
+ message: 'Version conflict',
531
+ translations: {
532
+ de: 'Der Datensatz wurde inzwischen geändert.',
533
+ en: 'The record was changed in the meantime.',
534
+ },
535
+ },
536
+
518
537
  // =====================================================
519
538
  // File Errors (LTNS_0500-LTNS_0599)
520
539
  // =====================================================
@@ -702,6 +721,64 @@ export const LtnsErrors = {
702
721
  en: 'The AI connection has no valid base URL.',
703
722
  },
704
723
  },
724
+
725
+ AI_SETTING_ENFORCED: {
726
+ code: 'LTNS_0609',
727
+ message: 'This AI setting is fixed by the server',
728
+ translations: {
729
+ de: 'Diese KI-Einstellung ist vom Server festgelegt und kann nicht geändert werden.',
730
+ en: 'This AI setting is fixed by the server and cannot be changed.',
731
+ },
732
+ },
733
+
734
+ // =====================================================
735
+ // Offline Sync Errors (LTNS_0700-LTNS_0799)
736
+ // =====================================================
737
+
738
+ SYNC_RESYNC_REQUIRED: {
739
+ code: 'LTNS_0700',
740
+ message: 'Resync required',
741
+ translations: {
742
+ de: 'Die lokalen Daten müssen vollständig neu geladen werden.',
743
+ en: 'The local data must be loaded again in full.',
744
+ },
745
+ },
746
+
747
+ SYNC_MODEL_NOT_FOUND: {
748
+ code: 'LTNS_0701',
749
+ message: 'Model is not available for sync',
750
+ translations: {
751
+ de: 'Diese Daten können nicht synchronisiert werden.',
752
+ en: 'This data cannot be synchronized.',
753
+ },
754
+ },
755
+
756
+ SYNC_INVALID_REQUEST: {
757
+ code: 'LTNS_0702',
758
+ message: 'Invalid sync request',
759
+ translations: {
760
+ de: 'Die Synchronisierungsanfrage ist ungültig.',
761
+ en: 'The synchronization request is invalid.',
762
+ },
763
+ },
764
+
765
+ SYNC_ORIGIN_REFUSED: {
766
+ code: 'LTNS_0703',
767
+ message: 'Origin not allowed for this request',
768
+ translations: {
769
+ de: 'Diese Anfrage ist von dieser Seite aus nicht erlaubt.',
770
+ en: 'This request is not allowed from this site.',
771
+ },
772
+ },
773
+
774
+ SYNC_STORE_UNAVAILABLE: {
775
+ code: 'LTNS_0704',
776
+ message: 'Sync store unavailable',
777
+ translations: {
778
+ de: 'Die Synchronisierung ist gerade nicht verfügbar. Bitte später erneut versuchen.',
779
+ en: 'Synchronization is unavailable right now. Please try again later.',
780
+ },
781
+ },
705
782
  } as const satisfies IErrorRegistry;
706
783
 
707
784
  // =====================================================
@@ -1,13 +1,44 @@
1
1
  import type { DatabaseSqlEngine } from '../../common/interfaces/server-options.interface.js';
2
2
  import { withMysqlSchemaLock } from '../../common/persistence/sql/mysql/mysql-schema.migrator.js';
3
+ import { MYSQL_SYNC_COUNTERS } from '../../common/persistence/sql/mysql/mysql-schema.compiler.js';
4
+ import { syncSequenceName } from '../../common/persistence/sql/postgres/postgres-schema.compiler.js';
3
5
  import { SCHEMA_LOCK_SQL } from '../../common/persistence/sql/postgres/postgres-schema.migrator.js';
4
6
  import { createSqlExecutor } from '../../common/persistence/sql/sql-executor.factory.js';
7
+ import { quoteIdentifier, quoteMysqlIdentifier } from '../../common/persistence/sql/sql-naming.js';
8
+
9
+ /**
10
+ * A data migration's multi-row write on the table of a synchronized model, numbered the way the store numbers it (plan
11
+ * §3.4, C1d): the rows are locked first (`SELECT … FOR UPDATE`), then the change numbers taken, then the rows written
12
+ * with their numbers set and their `version` incremented. "Rows first, counter last" is what keeps the migration from
13
+ * deadlocking with a running application's writes on MySQL and MariaDB, whose row locks also lock index gaps; the
14
+ * incremented version turns the change into a conflict for a client that edited the old one, never a silent overwrite.
15
+ * Clients receive the rows at their next pull, like any change.
16
+ *
17
+ * `set` and `where` are SQL, reviewed like every statement of the migration — never built from request data.
18
+ */
19
+ export interface SqlSyncUpdate {
20
+ syncUpdate: {
21
+ /** Assignments without `SET`, e.g. `"status" = 'archived'` (never `version` or `sync_seq`, both set here) */
22
+ set: string;
23
+ /** The table of the synchronized model, e.g. `sync_notes` */
24
+ table: string;
25
+ /** The rows to write, a condition without `WHERE` */
26
+ where: string;
27
+ };
28
+ }
29
+
30
+ /** How a migration statement runs: on the migration's connection, inside its transaction where there is one */
31
+ export type SqlMigrationRun = (statement: { params: unknown[]; text: string }) => Promise<{ rows: any[] }>;
32
+
33
+ /** Rows written per statement */
34
+ const SYNC_UPDATE_CHUNK = 500;
5
35
 
6
36
  /**
7
37
  * Run the reviewed statements of a SQL migration (`migrate create <name> --sql`) as the table OWNER, under the lock the
8
38
  * boot's schema migration takes, so a replica booting meanwhile waits and then sees the result. On PostgreSQL in one
9
39
  * transaction: either every statement applies or none does. MySQL and MariaDB commit every DDL statement on its own, so
10
- * there a failure leaves the statements before it applied — write such a migration so it can run again.
40
+ * there a failure leaves the statements before it applied — write such a migration so it can run again. A `syncUpdate`
41
+ * runs in a transaction of its own there, because a DDL statement before it has ended the migration's.
11
42
  *
12
43
  * The statements are the ones a refused boot suggested (`SchemaMigrationRequiredError`) — after review: a narrowed
13
44
  * type, a new unique index or a rename collision can reject or rewrite stored rows, which is why the framework never
@@ -15,7 +46,7 @@ import { createSqlExecutor } from '../../common/persistence/sql/sql-executor.fac
15
46
  */
16
47
  export async function runSqlMigration(
17
48
  ownerUrl: string,
18
- statements: readonly string[],
49
+ statements: readonly (SqlSyncUpdate | string)[],
19
50
  options: { engine?: DatabaseSqlEngine } = {},
20
51
  ): Promise<void> {
21
52
  if (!ownerUrl) {
@@ -30,19 +61,122 @@ export async function runSqlMigration(
30
61
  try {
31
62
  if (/^(mysql|mariadb):/i.test(ownerUrl)) {
32
63
  await withMysqlSchemaLock(executor, async (run) => {
33
- for (const text of statements) {
34
- await run({ params: [], text });
64
+ for (const statement of statements) {
65
+ await (typeof statement === 'string'
66
+ ? run({ params: [], text: statement })
67
+ : applySqlSyncUpdate(run, 'mysql', statement.syncUpdate));
35
68
  }
36
69
  });
37
70
  return;
38
71
  }
39
72
  await executor.transaction(async (tx) => {
40
- await tx.run({ params: [], text: SCHEMA_LOCK_SQL }, { kind: 'system' });
41
- for (const text of statements) {
42
- await tx.run({ params: [], text }, { kind: 'system' });
73
+ const run: SqlMigrationRun = (statement) => tx.run(statement, { kind: 'system' });
74
+ await run({ params: [], text: SCHEMA_LOCK_SQL });
75
+ for (const statement of statements) {
76
+ await (typeof statement === 'string'
77
+ ? run({ params: [], text: statement })
78
+ : applySqlSyncUpdate(run, 'postgres', statement.syncUpdate));
43
79
  }
44
80
  });
45
81
  } finally {
46
82
  await executor.close();
47
83
  }
48
84
  }
85
+
86
+ /**
87
+ * One `syncUpdate` of a migration: rows locked, numbers taken, rows written, in one transaction — the migration's on
88
+ * PostgreSQL. On MySQL / MariaDB in a transaction of its own: a DDL statement earlier in the migration committed the
89
+ * migration's transaction implicitly, and in autocommit every statement would release its locks at once — a running
90
+ * application could then commit a higher number before these rows (a client's cursor passes them for good) or read the
91
+ * counter between the increment and the read here (one number twice). Answers the number of rows written.
92
+ */
93
+ export async function applySqlSyncUpdate(
94
+ run: SqlMigrationRun,
95
+ dialect: 'mysql' | 'postgres',
96
+ update: SqlSyncUpdate['syncUpdate'],
97
+ ): Promise<number> {
98
+ if (dialect === 'postgres') {
99
+ return writeSyncUpdate(run, dialect, update);
100
+ }
101
+ await run({ params: [], text: 'START TRANSACTION' });
102
+ try {
103
+ const written = await writeSyncUpdate(run, dialect, update);
104
+ await run({ params: [], text: 'COMMIT' });
105
+ return written;
106
+ } catch (error) {
107
+ await run({ params: [], text: 'ROLLBACK' }).catch(() => undefined);
108
+ throw error;
109
+ }
110
+ }
111
+
112
+ async function writeSyncUpdate(
113
+ run: SqlMigrationRun,
114
+ dialect: 'mysql' | 'postgres',
115
+ update: SqlSyncUpdate['syncUpdate'],
116
+ ): Promise<number> {
117
+ const quote = dialect === 'mysql' ? quoteMysqlIdentifier : quoteIdentifier;
118
+ const table = quote(update.table);
119
+ const id = quote('id');
120
+ const { rows } = await run({
121
+ params: [],
122
+ text: `SELECT ${id} AS id FROM ${table} WHERE ${update.where} ORDER BY ${id} FOR UPDATE`,
123
+ });
124
+ const ids = rows.map((row) => String(row.id));
125
+ if (!ids.length) {
126
+ return 0;
127
+ }
128
+ const numbers =
129
+ dialect === 'mysql'
130
+ ? await mysqlNumbers(run, update.table, ids.length)
131
+ : await postgresNumbers(run, update.table, ids.length);
132
+ const bigint = dialect === 'mysql' ? '' : '::bigint';
133
+ for (let start = 0; start < ids.length; start += SYNC_UPDATE_CHUNK) {
134
+ const chunk = ids.slice(start, start + SYNC_UPDATE_CHUNK);
135
+ const params: unknown[] = [];
136
+ const param = (value: unknown) => {
137
+ params.push(value);
138
+ return `$${params.length}`;
139
+ };
140
+ const cases = chunk
141
+ .map((row, index) => `WHEN ${param(row)} THEN ${param(numbers[start + index])}${bigint}`)
142
+ .join(' ');
143
+ const list = chunk.map((row) => param(row)).join(', ');
144
+ await run({
145
+ params,
146
+ text:
147
+ `UPDATE ${table} SET ${update.set}, ${quote('version')} = ${quote('version')} + 1, ` +
148
+ `${quote('sync_seq')} = CASE ${id} ${cases} END WHERE ${id} IN (${list})`,
149
+ });
150
+ }
151
+ return ids.length;
152
+ }
153
+
154
+ /** The next change numbers of a table on PostgreSQL, as the store takes them (advisory lock, then the sequence) */
155
+ async function postgresNumbers(run: SqlMigrationRun, table: string, count: number): Promise<string[]> {
156
+ await run({
157
+ params: [table],
158
+ text: `SELECT pg_advisory_xact_lock(hashtext('lenne.tech:sync:' || current_schema() || '.' || $1))`,
159
+ });
160
+ const { rows } = await run({
161
+ params: [syncSequenceName(table), count],
162
+ text: `SELECT nextval(format('%I.%I', current_schema(), $1::text)::regclass)::text AS n FROM generate_series(1, $2::int)`,
163
+ });
164
+ return rows.map((row) => String(row.n)).sort((a, b) => (BigInt(a) < BigInt(b) ? -1 : 1));
165
+ }
166
+
167
+ /** The next change numbers of a table on MySQL / MariaDB, as the store takes them (the counter row, last) */
168
+ async function mysqlNumbers(run: SqlMigrationRun, table: string, count: number): Promise<string[]> {
169
+ await run({
170
+ params: [count, table],
171
+ text: `UPDATE ${quoteMysqlIdentifier(MYSQL_SYNC_COUNTERS)} SET \`seq\` = \`seq\` + $1 WHERE \`model\` = $2`,
172
+ });
173
+ const { rows } = await run({
174
+ params: [table],
175
+ text: `SELECT \`seq\` FROM ${quoteMysqlIdentifier(MYSQL_SYNC_COUNTERS)} WHERE \`model\` = $1`,
176
+ });
177
+ if (!rows.length) {
178
+ throw new Error(`runSqlMigration(): ${table} is not the table of a synchronized model (no change counter)`);
179
+ }
180
+ const last = BigInt(rows[0].seq);
181
+ return Array.from({ length: count }, (_, index) => (last - BigInt(count - 1 - index)).toString());
182
+ }
@@ -0,0 +1,95 @@
1
+ # Sync Integration Checklist
2
+
3
+ Offer a model to offline clients (`docs/offline-sync-protocol.md`). Background: [`README.md`](./README.md).
4
+
5
+ ## Reference Implementation
6
+
7
+ - Local: `node_modules/@lenne.tech/nest-server/src/server/modules/sync-note/`
8
+ - GitHub: https://github.com/lenneTech/nest-server/tree/develop/src/server/modules/sync-note
9
+
10
+ The reference is not part of the reference server (`ServerModule` does not import it); the sync contract suite runs on
11
+ exactly these files.
12
+
13
+ ## Required Files (Create in Order)
14
+
15
+ ### 1. Enable the module
16
+
17
+ **Edit:** `src/config.env.ts`
18
+
19
+ ```typescript
20
+ sync: { cursorSecret: process.env.SYNC_CURSOR_SECRET },
21
+ ```
22
+
23
+ WHY: the cursors are signed with it. Without `sync.cursorSecret` (or `SECRETS_ENCRYPTION_KEY`) the boot fails in
24
+ `production` / `staging`.
25
+
26
+ ### 2. The model
27
+
28
+ **Create:** `src/server/modules/<module>/<module>.model.ts`
29
+ **Copy from:** `sync-note.model.ts`
30
+
31
+ - `@PersistedModel({ sync: { roles: { pull, create, update, delete }, strategy } })` — declare all four roles.
32
+ WHY: the roles on your controllers do not reach `/sync`; an operation without roles here is open to admins only.
33
+ - Do **not** declare `version`, `deletedAt`, `syncAt`, `syncSeq` — the store adds them (a declared one is refused).
34
+ - Declare `createdBy` when an operation uses `S_CREATOR` or `securityCheck()` decides by the creator.
35
+ - `securityCheck()` decides who sees a record — a pull and every push lookup run through it.
36
+ - **MySQL / MariaDB:** the owner account creates the change-order triggers at boot. WHY: with binary logging on (the
37
+ MySQL 8.4 default) that needs `SUPER` or `log_bin_trust_function_creators=1` on the server — without it the boot stops
38
+ with error 1419. The application account needs neither.
39
+
40
+ ### 3. The inputs
41
+
42
+ **Create:** `inputs/<module>-create.input.ts`, `inputs/<module>.input.ts`
43
+ **Copy from:** `inputs/sync-note-create.input.ts`, `inputs/sync-note.input.ts`
44
+
45
+ WHY: a push's `data` is validated and whitelisted against them exactly like a request body — only what they declare
46
+ can be set.
47
+
48
+ ### 4. The service
49
+
50
+ **Create / edit:** `<module>.service.ts`
51
+ **Copy from:** `sync-note.service.ts`
52
+
53
+ ```typescript
54
+ @SyncService({ createInput: NoteCreateInput, model: Note, updateInput: NoteInput })
55
+ ```
56
+
57
+ For `strategy: 'custom'` implement `resolveSyncConflict(context)` (`SyncConflictResolver`); the boot fails without it.
58
+
59
+ ### 5. The module
60
+
61
+ **Edit:** `<module>.module.ts` — register the model with `CorePersistenceModule.forFeature([{ model, name, schema }])`
62
+ **Copy from:** `sync-note.module.ts`
63
+
64
+ WHY: only a model registered this way can be synchronized (its store maintains the system fields).
65
+
66
+ ### 6. An EXISTING model with data
67
+
68
+ Making an existing model synchronized is a migration on every database: the boot refuses with the statements to apply
69
+ (SQL: the `deleted_at` column, the live-only unique index, `DROP` of the full one; MongoDB:
70
+ `SyncMigrationRequiredError`). See `.claude/rules/persistence.md` → "Synchronized models".
71
+
72
+ ## Verification Checklist
73
+
74
+ - [ ] The boot logs `Offline sync for <Model>` and no "declares no sync roles" warning
75
+ - [ ] `GET /sync/<Model>` answers `{ items, cursor, hasMore }` for a signed-in user with the pull roles, 403 without
76
+ - [ ] A push of `{ mutationId, op: 'upsert', id, data }` answers `applied` with `version: 1`; the same push again too
77
+ - [ ] A user of another tenant pulls none of the records and gets `not_found` / `ID_UNAVAILABLE` for their ids
78
+ - [ ] `GET /sync/hints` (EventSource) sends `event: change` after a write in the user's scope, and nothing for another
79
+ tenant's; the client pulls on every hint and once after every reconnect
80
+ - [ ] Writes past the CrudService (data store, native API, migration) call `CoreSyncService.notifyChange()`, and
81
+ multi-row data migrations on a synchronized SQL table use `runSqlMigration()` with `syncUpdate`
82
+ - [ ] The client deletes every local record, cursor and unsent change on logout and on an account switch (the device
83
+ holds personal data — see [SECURITY.md](./SECURITY.md))
84
+
85
+ ## Common Mistakes
86
+
87
+ | Mistake | Symptom | Fix |
88
+ | ----------------------------------------------------- | ------------------------------------------------------------------------ | ----------------------------------------------------------- |
89
+ | No `sync.roles` | Every push of a user is `rejected` / `FORBIDDEN`, pull 403; boot warning | Declare all four operations |
90
+ | `@SyncService()` missing | `404` for the model | Decorate the CrudService |
91
+ | Model registered with `MongooseModule.forFeature()` | Boot error "not registered with CorePersistenceModule.forFeature()" | Use `CorePersistenceModule.forFeature()` |
92
+ | A system field declared on the model | Boot error "reserved on a synchronized model" | Remove it — the store maintains it |
93
+ | Writes to a tenant-scoped model without `X-Tenant-Id` | Rejected by the tenant policy | Send the header on writes; pulls may span all memberships |
94
+ | Records moved by a direct store / native write | Previous owners keep their copies | Call `CoreSyncService.bumpAccessEpoch()` after such a write |
95
+ | A driver's default `new ObjectId()` on the client | Ids predictable from one observed id | Random tail per protocol §3, or uuidv7 |
@@ -0,0 +1,198 @@
1
+ # Sync (offline sync)
2
+
3
+ Clients read and write while offline and reconcile with the server afterwards: they **pull** what changed since their
4
+ cursor, **push** what they changed locally, and receive **tombstones** for deletions. The client contract is the
5
+ protocol in [`docs/offline-sync-protocol.md`](../../../../docs/offline-sync-protocol.md) — a client needs `fetch` and
6
+ nothing from this package. This module is the server side, on every database the framework supports (MongoDB
7
+ standalone and replica set, PostgreSQL with TypeORM, Kysely or Drizzle, MySQL, MariaDB), with the same observable
8
+ behaviour.
9
+
10
+ Integration steps: [`INTEGRATION-CHECKLIST.md`](./INTEGRATION-CHECKLIST.md). Reference implementation:
11
+ [`src/server/modules/sync-note/`](../../../server/modules/sync-note/).
12
+
13
+ ## What it adds
14
+
15
+ | Route | Protocol | |
16
+ | ------------------------------------ | -------- | ---------------------------------------------------------- |
17
+ | `GET /sync/{model}?cursor=…&limit=…` | §5 | One page of changes after the cursor, tombstones included |
18
+ | `POST /sync/{model}` | §6 | Apply `{ items: [...] }` item by item, one result per item |
19
+ | `GET /sync/hints?models=…` | §9 | SSE: `event: change` with `{ model, at }` — pull now |
20
+ | `subscription { syncHint(models) }` | §9 | The same hints over GraphQL (where GraphQL is enabled) |
21
+
22
+ Nothing without the `sync` configuration: no route, no infrastructure model, no listener on the write paths.
23
+
24
+ ## Configuration (`sync`, presence implies enabled)
25
+
26
+ ```typescript
27
+ sync: {
28
+ cursorSecret: process.env.SYNC_CURSOR_SECRET, // else SECRETS_ENCRYPTION_KEY; REQUIRED in production / staging
29
+ path: 'sync',
30
+ pull: { defaultLimit: 200, maxLimit: 1000, maxBytes: 1_048_576, windowSeconds: undefined /* MongoDB, derived */ },
31
+ push: { maxItems: 100 },
32
+ hints: { debounceMs: 250, sse: true, graphql: true }, // false: both channels off
33
+ retention: { mutationsDays: 30, tombstonesDays: 90 }, // tombstonesDays: false keeps them for ever
34
+ rateLimit: { pull: { max: 120, windowSeconds: 60 }, push: { max: 60, windowSeconds: 60 } }, // false: off
35
+ }
36
+ ```
37
+
38
+ `true` or `{}` enables it with these defaults. The full reference: `ISync` in
39
+ `common/interfaces/server-options.interface.ts`.
40
+
41
+ ## Offering a model
42
+
43
+ Two declarations, both explicit — a model is never offered by accident:
44
+
45
+ 1. **The model** — `@PersistedModel({ sync: { roles, strategy, scope, idFormat } })`. This makes it a synchronized
46
+ model in the data store (version, tombstones, live-only unique indexes, change order — see
47
+ `.claude/rules/persistence.md` → "Synchronized models") and carries what the sync routes need:
48
+ - `roles` per operation (`pull`, `create`, `update`, `delete`). The roles a project puts on its own routes do **not**
49
+ reach `/sync`; an operation without roles is open to `ADMIN` only, and the boot warns until they are declared. They
50
+ are passed as `serviceOptions.roles`, so `S_CREATOR` / `S_SELF` are decided against the stored record.
51
+ - `strategy`: `reject` (default), `lww`, `merge` (field by field; `@SyncField('reject')` marks a field that still
52
+ conflicts), `custom` (the service's `resolveSyncConflict()`).
53
+ - `scope`: the user-id fields a tombstone keeps (e.g. `['ownerId']`), so a deletion reaches exactly its users.
54
+ 2. **The service** — `@SyncService({ model, createInput, updateInput })` on the model's `CrudService`. Every pull and
55
+ push runs through that service; `data` is validated and whitelisted against the inputs like a request body.
56
+
57
+ The model must be registered with `CorePersistenceModule.forFeature()`.
58
+
59
+ ## Security model
60
+
61
+ | Rule | How |
62
+ | ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
63
+ | The full pipeline, no lean path | Pull reads through the service's `find()` (`process()`, output `@Restricted`, tenant policy, RLS) and runs every record through `securityCheck()`; push writes through `create()` / `update()` / `delete()` |
64
+ | Operation rights | The model's `sync.roles`, deny by default (ADMIN only) |
65
+ | Hidden fields | No answer depends on a value the caller may not read: a repeated create is always `conflict` with the record as the caller sees it, never compared with `data` |
66
+ | Foreign ids | Change / delete: `not_found`, exactly like a missing id. Create: `ID_UNAVAILABLE` — the same answer for every reason an id cannot be used |
67
+ | No hard delete | Push `delete` goes through `CrudService.delete()`, which tombstones a synchronized model on every path |
68
+ | Tombstones | Carry id, version, `deletedAt`, `updatedAt`, `tenantId` and the scope fields the caller may see — no content |
69
+ | Cursor | Opaque, HMAC-signed, bound to model and user; another user's, another model's or a tampered cursor is `410 invalid` |
70
+ | Access loss | A fingerprint of the caller's access (user, tenant scope, roles, membership role, verification) and access epochs travel in the cursor; a change answers `410 access`, the client resyncs in full; a caller who may not pull at all, or names a left tenant in `X-Tenant-Id`, gets `403` and drops its copies the same way |
71
+ | Cookie push | `Content-Type: application/json`; while the server uses cookies, `Sec-Fetch-Site: cross-site` and an untrusted `Origin` answer 403 |
72
+ | API tokens | Refused like on every route without `@ApiTokenScopes()` (see "Extending") |
73
+ | Session expired | 401 for the whole request before any item; the client resends the same items, mutation ids recognise them |
74
+
75
+ A pull or push without `X-Tenant-Id` spans every tenant the caller belongs to (the tenant guard's membership logic);
76
+ writes to a tenant-scoped model need the header, as everywhere — a create without it is `rejected` / `FORBIDDEN` on
77
+ every database, because nothing could tell which of the caller's tenants the record belongs to. A pull without it reads
78
+ each of the caller's tenants on its own and merges them in change order, so the tenant-led change index serves every
79
+ page.
80
+
81
+ What a client must do on its side — delete its local data on logout and on an account switch, keep credentials out of
82
+ reach of injected scripts, cache no API responses — is in [SECURITY.md](./SECURITY.md).
83
+
84
+ ## Clients
85
+
86
+ The protocol needs `fetch` and nothing from the server's code (`docs/offline-sync-protocol.md`). Two test clients in
87
+ the framework repository prove it end to end against the reference server, with one shared protocol implementation
88
+ (`tests/clients/sync-client.mjs`, not part of the package):
89
+
90
+ | Client | Storage | Suite |
91
+ | -------------------------------------------------------------------------------------------- | ---------------------- | ------------------------------------ |
92
+ | PWA in Chrome, served by the test server, service worker for offline loading, session cookie | IndexedDB | `tests/sync-pwa-client.e2e-spec.ts` |
93
+ | Node, as a desktop or React Native app would be, bearer session token | SQLite (`node:sqlite`) | `tests/sync-node-client.e2e-spec.ts` |
94
+
95
+ Both pass the same scenarios (`tests/helpers/sync-client-scenarios.ts`): create, change and delete offline, then online;
96
+ a conflict; two tabs pushing at once; a session that expired while offline; cleared storage. Two server processes with
97
+ clocks 10 s apart lose no change under parallel writers (`tests/sync-clock-skew.e2e-spec.ts`). A Nuxt composable for
98
+ projects follows in `@lenne.tech/nuxt-extensions`.
99
+
100
+ ## Idempotency and repeats
101
+
102
+ Every item carries a `mutationId`. The module **claims** it before the write (model, user, record, mutation id —
103
+ hashed, unique), records the outcome afterwards and answers a repeat with that outcome for `retention.mutationsDays`.
104
+ A repeat that finds a claim still being written answers `retry` — that is how two tabs sending the same push end in one
105
+ write and no false conflict. A claim older than 30 s belongs to a request that died and is taken over.
106
+
107
+ Once an item of a record is not stored (`conflict`, `rejected`, `not_found`, `retry`), the later items of that record
108
+ in the same push are **held back** (12.3.0+): each builds on the version the one before it produces (protocol §6.2), and
109
+ applied on its own its base version could match what another writer left. A held-back item is never applied or
110
+ claimed; it is answered `retry` behind a `retry`, otherwise `conflict` with the record as the caller may see it
111
+ (`not_found` without one). Items of other records are unaffected. This holds within one push; across pushes the
112
+ client keeps the order (protocol §11).
113
+
114
+ ## Change order and windows
115
+
116
+ The store decides how changes are ordered (`capabilities.changeOrder`): on SQL a commit-ordered sequence (`syncSeq`) —
117
+ no repeats; on MongoDB the server clock (`syncAt`) with a window W below each run's start — records changed within W
118
+ are delivered again, clients keep the higher `version`. W is derived: replica set or sharded cluster — the server's
119
+ `transactionLifetimeLimitSeconds` (60 when unreadable) + 30 s; standalone — 30 s. `sync.pull.windowSeconds` overrides
120
+ it; it must cover the longest time a write can take from its stamp until it is visible.
121
+
122
+ On MySQL / MariaDB the change numbers come from triggers the owner account (`database.ownerUrl`) creates at boot. With
123
+ binary logging on — the MySQL 8.4 default — that needs `SUPER` or `log_bin_trust_function_creators=1` on the server
124
+ (the parameter group on managed MySQL), otherwise the boot stops with error 1419. The application account needs
125
+ neither.
126
+
127
+ ## Hints
128
+
129
+ A hint tells a connected client that pulling is worthwhile; sync works without them (protocol §9).
130
+
131
+ - **Source:** every successful `create()` / `update()` / `delete()` of the model's `CrudService` — on every database,
132
+ no change feed. A write past it (the data store, the native API, a migration) calls
133
+ `CoreSyncService.notifyChange(model, tenantIds)`, or clients learn of it at their next pull.
134
+ - **Content:** `{ model, at }` only. **Recipients:** callers with the model's pull roles and, on a tenant-scoped model,
135
+ only for changes in their own tenants (the validated `X-Tenant-Id`, without it every membership). `models` narrows a
136
+ channel; a name that is not offered never matches and reveals nothing.
137
+ - **Bursts:** one hint per model and tenant per `hints.debounceMs`, already at the sender. **Replicas:** with `redis`
138
+ every replica receives every hint through one channel and delivers it to its own subscribers; without Redis a hint
139
+ stays on its replica.
140
+ - **Long-lived channels are re-checked:** the GraphQL subscription runs on the WebSocket `CoreModule` re-checks; an SSE
141
+ stream is registered in the same registry and ends at once on an invalidation naming the caller or their tenant, and
142
+ otherwise before the first hint after `graphQl.subscriptionRevalidationMs` — the EventSource reconnects (`retry`) and
143
+ the new request passes the guards. A client pulls once after every (re)connect. In the log the planned end after the
144
+ interval is a debug line (`Closing SSE hint stream of user …: re-check interval reached, client reconnects`); an end
145
+ an invalidation brings stays at `log` (`… authorization invalidated, client reconnects`), so a real revocation stands
146
+ out (12.3.0; before, every planned end read `Closing GraphQL WebSocket …: Session no longer valid`).
147
+
148
+ ## Tombstone purge
149
+
150
+ Every hour, on one replica (a lease: Redis, else the SQL lease table, else `cron-locks` in MongoDB), each model's
151
+ horizon moves to `retention.tombstonesDays` ago **first**, then the older tombstones are deleted for good. A client
152
+ whose cursor is older than the horizon answers its next pull with `410 horizon` and resyncs; nothing is skipped.
153
+ The tombstones go in batches of 1000, read by id and deleted by id: one statement over the whole table would lock
154
+ every row it scans on MySQL / MariaDB and hold up every write of the model.
155
+ `CoreSyncService.purgeTombstones({ before?, model? })` runs it on demand.
156
+
157
+ ## Data migrations on a synchronized SQL table
158
+
159
+ A multi-row write in a migration goes through `runSqlMigration()` as a `syncUpdate`, never as a plain `UPDATE`:
160
+
161
+ ```typescript
162
+ await runSqlMigration(ownerUrl, [
163
+ { syncUpdate: { table: 'sync_notes', set: "status = 'archived'", where: "created_at < '2026-01-01'" } },
164
+ ]);
165
+ ```
166
+
167
+ It locks the rows first, then takes the change numbers like the store, then writes them with `version + 1` — the order
168
+ that cannot deadlock with a running application on MySQL / MariaDB, and the version a client's stale edit then
169
+ conflicts with. On MySQL / MariaDB each `syncUpdate` runs in a transaction of its own, because a DDL statement before
170
+ it has already committed the migration's. `set` and `where` are reviewed SQL, like every statement of a migration.
171
+
172
+ ## Infrastructure models
173
+
174
+ | Model | Collection / table | Holds |
175
+ | -------------- | ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
176
+ | `SyncMutation` | `syncMutations` / `sync_mutations` | Claims and outcomes of pushed mutations, `expiresAt` (TTL on MongoDB, expiry sweep on SQL; every reader checks it) |
177
+ | `SyncState` | `syncStates` / `sync_states` | Access epochs (`epoch:t:<model>:<tenant>`, `epoch:u:<model>:<tenant>:<user>`) and tombstone horizons (`horizon:<model>`) |
178
+
179
+ Both follow `database` like every other model.
180
+
181
+ ## Extending
182
+
183
+ Override through `CoreModule.forRoot(env, { sync: { controller, hintsController, resolver, service } })`:
184
+
185
+ - **`CoreSyncService`** — every step is `protected`: `readChanges()`, `toSyncRecord()`, `accessState()`,
186
+ `resolveUpdateConflict()`, `writeFailure()`, …
187
+ - **`CoreSyncController`** — e.g. to release a scope to API tokens: extend it, re-declare `pull()` / `push()` with their
188
+ route decorators plus `@ApiTokenScopes('sync')`, and call `super`.
189
+ - **`CoreSyncHintsController`** — `GET /sync/hints`, a controller of its own that the module registers before
190
+ `CoreSyncController`: Nest registers a class's own routes before the inherited ones, so a re-declared `pull()`
191
+ (`@Get(':model')`) would otherwise read `hints` as a model name. Extend it the same way to release hints to API
192
+ tokens. `CoreSyncBaseController` holds what both share (`scopeTenants()`, `limit()`).
193
+ - **`CoreSyncResolver`** — the `syncHint` subscription.
194
+ - **`CoreSyncService` is global** while `sync` is configured: inject it in any project service (with `@Optional()` where
195
+ the project also runs without `sync`) to call `notifyChange()` / `bumpAccessEpoch()`.
196
+ - **Access that moves past `CrudService.update()`** (a direct store write, a migration, the native API) bumps no epoch:
197
+ call `CoreSyncService.bumpAccessEpoch(model, tenantId, userId?)` after such a write, or the clients concerned keep
198
+ their copies until their next full resync.