jxp 3.0.0 → 4.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (219) hide show
  1. package/.env.sample +31 -6
  2. package/README.md +27 -13
  3. package/dist/bin/init_db.d.ts +3 -0
  4. package/dist/bin/init_db.d.ts.map +1 -0
  5. package/dist/bin/init_db.js +48 -0
  6. package/dist/bin/init_db.js.map +1 -0
  7. package/dist/bin/server.d.ts +4 -0
  8. package/dist/bin/server.d.ts.map +1 -0
  9. package/dist/bin/server.js +132 -0
  10. package/dist/bin/server.js.map +1 -0
  11. package/dist/bin/setup.d.ts +3 -0
  12. package/dist/bin/setup.d.ts.map +1 -0
  13. package/dist/bin/setup.js +185 -0
  14. package/dist/bin/setup.js.map +1 -0
  15. package/dist/globals.d.ts +14 -0
  16. package/dist/globals.d.ts.map +1 -0
  17. package/dist/globals.js +3 -0
  18. package/dist/globals.js.map +1 -0
  19. package/dist/libs/aggregate_guard.d.ts +5 -0
  20. package/dist/libs/aggregate_guard.d.ts.map +1 -0
  21. package/dist/libs/aggregate_guard.js +58 -0
  22. package/dist/libs/aggregate_guard.js.map +1 -0
  23. package/dist/libs/bulkwrite_guard.d.ts +5 -0
  24. package/dist/libs/bulkwrite_guard.d.ts.map +1 -0
  25. package/dist/libs/bulkwrite_guard.js +44 -0
  26. package/dist/libs/bulkwrite_guard.js.map +1 -0
  27. package/dist/libs/cache.d.ts +2 -0
  28. package/dist/libs/cache.d.ts.map +1 -0
  29. package/dist/libs/cache.js +94 -0
  30. package/dist/libs/cache.js.map +1 -0
  31. package/dist/libs/call_guard.d.ts +3 -0
  32. package/dist/libs/call_guard.d.ts.map +1 -0
  33. package/dist/libs/call_guard.js +17 -0
  34. package/dist/libs/call_guard.js.map +1 -0
  35. package/dist/libs/connection_string.d.ts +3 -0
  36. package/dist/libs/connection_string.d.ts.map +1 -0
  37. package/dist/libs/connection_string.js +4 -0
  38. package/dist/libs/connection_string.js.map +1 -0
  39. package/dist/libs/datamunging.d.ts +2 -0
  40. package/dist/libs/datamunging.d.ts.map +1 -0
  41. package/dist/libs/datamunging.js +320 -0
  42. package/dist/libs/datamunging.js.map +1 -0
  43. package/dist/libs/docs.d.ts +2 -0
  44. package/dist/libs/docs.d.ts.map +1 -0
  45. package/dist/libs/docs.js +307 -0
  46. package/dist/libs/docs.js.map +1 -0
  47. package/dist/libs/groups.d.ts +2 -0
  48. package/dist/libs/groups.d.ts.map +1 -0
  49. package/dist/libs/groups.js +136 -0
  50. package/dist/libs/groups.js.map +1 -0
  51. package/dist/libs/jxp.d.ts +4 -0
  52. package/dist/libs/jxp.d.ts.map +1 -0
  53. package/dist/libs/jxp.js +1157 -0
  54. package/dist/libs/jxp.js.map +1 -0
  55. package/dist/libs/link_index.d.ts +14 -0
  56. package/dist/libs/link_index.d.ts.map +1 -0
  57. package/dist/libs/link_index.js +37 -0
  58. package/dist/libs/link_index.js.map +1 -0
  59. package/dist/libs/load-config.d.ts +16 -0
  60. package/dist/libs/load-config.d.ts.map +1 -0
  61. package/dist/libs/load-config.js +155 -0
  62. package/dist/libs/load-config.js.map +1 -0
  63. package/dist/libs/login.d.ts +2 -0
  64. package/dist/libs/login.d.ts.map +1 -0
  65. package/dist/libs/login.js +244 -0
  66. package/dist/libs/login.js.map +1 -0
  67. package/dist/libs/modeldir.d.ts +5 -0
  68. package/dist/libs/modeldir.d.ts.map +1 -0
  69. package/dist/libs/modeldir.js +15 -0
  70. package/dist/libs/modeldir.js.map +1 -0
  71. package/dist/libs/query_limits.d.ts +2 -0
  72. package/dist/libs/query_limits.d.ts.map +1 -0
  73. package/dist/libs/query_limits.js +84 -0
  74. package/dist/libs/query_limits.js.map +1 -0
  75. package/dist/libs/query_manipulation.d.ts +3 -0
  76. package/dist/libs/query_manipulation.d.ts.map +1 -0
  77. package/{libs → dist/libs}/query_manipulation.js +22 -21
  78. package/dist/libs/query_manipulation.js.map +1 -0
  79. package/dist/libs/query_sanitize.d.ts +5 -0
  80. package/dist/libs/query_sanitize.d.ts.map +1 -0
  81. package/dist/libs/query_sanitize.js +109 -0
  82. package/dist/libs/query_sanitize.js.map +1 -0
  83. package/dist/libs/response_sanitize.d.ts +7 -0
  84. package/dist/libs/response_sanitize.d.ts.map +1 -0
  85. package/dist/libs/response_sanitize.js +57 -0
  86. package/dist/libs/response_sanitize.js.map +1 -0
  87. package/dist/libs/safe_error.d.ts +3 -0
  88. package/dist/libs/safe_error.d.ts.map +1 -0
  89. package/dist/libs/safe_error.js +14 -0
  90. package/dist/libs/safe_error.js.map +1 -0
  91. package/dist/libs/schema.d.ts +14 -0
  92. package/dist/libs/schema.d.ts.map +1 -0
  93. package/dist/libs/schema.js +98 -0
  94. package/dist/libs/schema.js.map +1 -0
  95. package/dist/libs/schema_description.d.ts +3 -0
  96. package/dist/libs/schema_description.d.ts.map +1 -0
  97. package/{libs → dist/libs}/schema_description.js +12 -21
  98. package/dist/libs/schema_description.js.map +1 -0
  99. package/dist/libs/security.d.ts +2 -0
  100. package/dist/libs/security.d.ts.map +1 -0
  101. package/dist/libs/security.js +428 -0
  102. package/dist/libs/security.js.map +1 -0
  103. package/dist/libs/setup.d.ts +2 -0
  104. package/dist/libs/setup.d.ts.map +1 -0
  105. package/dist/libs/setup.js +87 -0
  106. package/dist/libs/setup.js.map +1 -0
  107. package/dist/libs/startup-deprecations.d.ts +2 -0
  108. package/dist/libs/startup-deprecations.d.ts.map +1 -0
  109. package/dist/libs/startup-deprecations.js +21 -0
  110. package/dist/libs/startup-deprecations.js.map +1 -0
  111. package/dist/libs/startup.d.ts +16 -0
  112. package/dist/libs/startup.d.ts.map +1 -0
  113. package/dist/libs/startup.js +63 -0
  114. package/dist/libs/startup.js.map +1 -0
  115. package/dist/libs/ws.d.ts +2 -0
  116. package/dist/libs/ws.d.ts.map +1 -0
  117. package/{libs → dist/libs}/ws.js +91 -73
  118. package/dist/libs/ws.js.map +1 -0
  119. package/dist/models/apikey_model.d.ts +10 -0
  120. package/dist/models/apikey_model.d.ts.map +1 -0
  121. package/dist/models/apikey_model.js +14 -0
  122. package/dist/models/apikey_model.js.map +1 -0
  123. package/dist/models/link_model.d.ts +9 -0
  124. package/dist/models/link_model.d.ts.map +1 -0
  125. package/dist/models/link_model.js +20 -0
  126. package/dist/models/link_model.js.map +1 -0
  127. package/dist/models/refreshtoken_model.d.ts +11 -0
  128. package/dist/models/refreshtoken_model.d.ts.map +1 -0
  129. package/dist/models/refreshtoken_model.js +22 -0
  130. package/dist/models/refreshtoken_model.js.map +1 -0
  131. package/dist/models/test_model.d.ts +24 -0
  132. package/dist/models/test_model.d.ts.map +1 -0
  133. package/dist/models/test_model.js +44 -0
  134. package/dist/models/test_model.js.map +1 -0
  135. package/dist/models/token_model.d.ts +14 -0
  136. package/dist/models/token_model.d.ts.map +1 -0
  137. package/dist/models/token_model.js +25 -0
  138. package/dist/models/token_model.js.map +1 -0
  139. package/dist/models/user_model.d.ts +15 -0
  140. package/dist/models/user_model.d.ts.map +1 -0
  141. package/dist/models/user_model.js +38 -0
  142. package/dist/models/user_model.js.map +1 -0
  143. package/dist/models/usergroups_model.d.ts +9 -0
  144. package/dist/models/usergroups_model.d.ts.map +1 -0
  145. package/dist/models/usergroups_model.js +18 -0
  146. package/dist/models/usergroups_model.js.map +1 -0
  147. package/dist/types/jxp-config.d.ts +134 -0
  148. package/dist/types/jxp-config.d.ts.map +1 -0
  149. package/dist/types/jxp-config.js +3 -0
  150. package/dist/types/jxp-config.js.map +1 -0
  151. package/dist/types/schema-fields.d.ts +43 -0
  152. package/dist/types/schema-fields.d.ts.map +1 -0
  153. package/dist/types/schema-fields.js +3 -0
  154. package/dist/types/schema-fields.js.map +1 -0
  155. package/docs/aggregations.md +40 -21
  156. package/docs/api.md +55 -15
  157. package/docs/authentication.md +60 -37
  158. package/docs/bulk_writes.md +11 -8
  159. package/docs/caching.md +28 -11
  160. package/docs/changelog.md +237 -31
  161. package/docs/configuration.md +99 -118
  162. package/docs/hooks.md +12 -8
  163. package/docs/index.md +17 -17
  164. package/docs/installation.md +23 -18
  165. package/docs/permissions.md +33 -19
  166. package/docs/queries.md +9 -5
  167. package/docs/schemas.md +21 -10
  168. package/docs/special.md +4 -4
  169. package/docs/typescript.md +86 -0
  170. package/docs/websocket.md +99 -1
  171. package/mkdocs.yml +1 -0
  172. package/package.json +121 -69
  173. package/templates/api.pug +32 -0
  174. package/templates/app-server.js +43 -0
  175. package/templates/app.env.sample +8 -0
  176. package/templates/assets/api-console.js +133 -0
  177. package/templates/assets/docs.css +227 -0
  178. package/templates/index.pug +62 -2
  179. package/templates/layout.pug +17 -11
  180. package/templates/md.pug +1 -1
  181. package/templates/model.pug +117 -68
  182. package/templates/navbar.pug +14 -21
  183. package/templates/sidebar.pug +30 -0
  184. package/.env +0 -9
  185. package/.eslintrc.json +0 -20
  186. package/.jshintrc +0 -4
  187. package/.vscode/launch.json +0 -34
  188. package/.vscode/settings.json +0 -3
  189. package/Dockerfile +0 -7
  190. package/bin/init_db.js +0 -54
  191. package/bin/server.js +0 -131
  192. package/bin/setup.js +0 -190
  193. package/config_sample.json +0 -10
  194. package/docker-compose.yml +0 -32
  195. package/libs/cache.js +0 -92
  196. package/libs/connection_string.js +0 -24
  197. package/libs/datamunging.js +0 -335
  198. package/libs/docs.js +0 -162
  199. package/libs/groups.js +0 -133
  200. package/libs/jxp.js +0 -1208
  201. package/libs/login.js +0 -222
  202. package/libs/modeldir.js +0 -14
  203. package/libs/schema.js +0 -99
  204. package/libs/security.js +0 -410
  205. package/libs/setup.js +0 -90
  206. package/models/apikey_model.js +0 -10
  207. package/models/link_model.js +0 -17
  208. package/models/refreshtoken_model.js +0 -20
  209. package/models/test_model.js +0 -50
  210. package/models/token_model.js +0 -23
  211. package/models/user_model.js +0 -38
  212. package/models/usergroups_model.js +0 -15
  213. package/setup.sh +0 -3
  214. package/test/.eslintrc.json +0 -13
  215. package/test/auth.js +0 -190
  216. package/test/init.js +0 -92
  217. package/test/setup.test.js +0 -62
  218. package/test/test.js +0 -1527
  219. package/test/wstest.js.old +0 -124
package/docs/changelog.md CHANGED
@@ -1,56 +1,262 @@
1
1
  # Changelog
2
2
 
3
- ## v1.0.0
3
+ Notable changes to [JXP](https://github.com/WorkSpaceMan/jxp).
4
4
 
5
- - Mongoose v5
5
+ ## v4.0.0 — 2026-05-19
6
6
 
7
- ## v1.0.1
7
+ Major release: complete TypeScript rewrite and configuration overhaul.
8
8
 
9
- - Closed bad security hole in groups - now only admins can set groups. (Will probably change this in future so that only those defined in the usergroup model can change groups.)
9
+ ### Breaking changes
10
10
 
11
- ## v1.0.12
11
+ - **Security hardening (4.0)** — see [migration](#security-hardening-migration) below.
12
+ - **No `config` package** — configuration via `.env` / environment variables and `jxp/libs/load-config` (compiled to `dist/libs/load-config.js`). Removed `/config/*.json` and [node-config](https://www.npmjs.com/package/config).
13
+ - **TypeScript** — framework source under `src/`; npm package ships compiled `dist/` with `.d.ts` types.
14
+ - **Build from source** — `npm run build` when developing from a git clone; npm registry installs ship prebuilt `dist/`.
15
+ - **Entry point** — `main` / `types` → `dist/libs/jxp.js` and `dist/libs/jxp.d.ts`.
16
+ - **Node 22+** — `engines.node` is `>=22.0.0`.
17
+ - **Relative paths** — `model_dir` and `log` resolve from `process.cwd()`, not the server script directory.
18
+ - **Built-in models** — compile from `src/models/*.ts` to `dist/models/`; consumer apps still use `*_model.js` in `MODEL_DIR`.
12
19
 
13
- - Select individual fields in populate joins
20
+ ### Added
14
21
 
15
- ## v1.0.16
22
+ - **`loadJxpConfig()`** — builds `JXP()` options from environment variables (`.env`, `.env.test`).
23
+ - **`package.json` `exports` map** — `jxp`, `jxp/libs/query_manipulation`, `jxp/libs/load-config`, `jxp/globals`.
24
+ - **Globals types** — `/// <reference types="jxp/globals" />` for JavaScript model authoring.
25
+ - **[docs/typescript.md](typescript.md)** — v4 migration guide.
26
+ - **Built-in documentation UI** — landing page, MkDocs nav sidebar, per-model API reference.
27
+ - **Interactive API console** — try REST endpoints from model pages; optional API key in docs navbar.
28
+ - **Query limits** (from v3.1) — env/config `query_limits`; default limit on all list/query requests; required explicit `?limit=` on large collections.
29
+ - **Security modules** — `query_sanitize`, `aggregate_guard`, `bulkwrite_guard`, `call_guard`, `response_sanitize`, `link_index`, `safe_error`.
30
+ - **Schema options** — `callable_statics` (string array); `advanced_queries: { query?, aggregate?, bulkwrite? }` (bulkwrite off by default).
31
+ - **Config / env** — `security.strip_fields`, `cors.origins`; `QUERY_LIMITS_DEFAULT`, `QUERY_LIMITS_SKIP_COUNT_UNLESS_PAGINATED`, `CORS_ORIGINS` (see [configuration.md](configuration.md)).
32
+ - **Startup helpers** — Node 22+ deprecation warnings; `quiet_startup` option.
16
33
 
17
- - Add a /query endpoint, that allows you to POST { query } for advanced queries
34
+ ### Changed
18
35
 
19
- ## v1.2.3
36
+ - **List responses** — `count` and `page_count` are omitted unless the client passes `?count=true` or `?page=` (avoids `countDocuments` on every list request).
37
+ - **CORS** — origins read from `config.cors.origins` (default `["*"]` if unset).
38
+ - **`/update/:model/:id`** — uses document load + `save()` (validators/hooks) instead of deprecated `Model.update()`.
39
+ - **Delete** — referrer checks use a pre-built link index and run in parallel.
40
+ - **`admin_only` middleware** — passes errors via `next(err)` for correct Restify handling.
41
+ - **Mongoose 6.13.9** (pinned).
42
+ - **Cache** — in-process [node-cache](https://www.npmjs.com/package/node-cache) *(replaces apicache from v2.6)*.
43
+ - **CSV** — `@json2csv/plainjs` replaces `json2csv`.
44
+ - **Email** — `nodemailer` 8.x; removed `nodemailer-smtp-transport`.
45
+ - **Docs rendering** — `markdown-it`; MkDocs pages from `docs/*.md`.
46
+ - **Tests** — Mocha 11; `test/env.js` loads `.env.test`; `test/security_hardening.test.js` for guards and allowlists.
47
+ - **Documentation** — full MkDocs audit for v4 behavior; security notes in [api.md](api.md), [queries.md](queries.md), [aggregations.md](aggregations.md), [bulk_writes.md](bulk_writes.md), [caching.md](caching.md), [schemas.md](schemas.md), [special.md](special.md).
20
48
 
21
- - Throttling
49
+ ### Fixed
22
50
 
23
- ## v1.2.4
51
+ - **`actionCallItem`** — `findById` is awaited; deleted documents are rejected; request body is passed to the static.
52
+ - **`apiKeyAuth`** — missing `await` on `User.findOne` when resolving API key users.
53
+ - **`middlewareModel`** — returns 404 when model name is unknown (no silent `undefined` model).
54
+ - **Filter depth** — exceeding max depth returns 400 instead of passing raw input through.
24
55
 
25
- - Time log includes operation number
56
+ ### Security hardening migration
26
57
 
27
- ## v2.0.1-0
58
+ Consumer apps (e.g. RevEngine) should adjust clients and models as follows:
28
59
 
29
- - Start v2!
30
- - Move to "connection_string" for MongoDB connection - allows connectivity to Atlas, for example
31
- - Use our own (much smarter) Schema class
60
+ | Area | Before | After |
61
+ |------|--------|--------|
62
+ | List GET / POST `/query` | Unbounded or large default on small collections | Default `?limit=100` when omitted; collections ≥10k docs still require explicit `?limit=` |
63
+ | Totals in list JSON | `count` always present | Pass `?count=true` or `?page=` when you need `count` / `page_count` |
64
+ | `/call/:model/:method` | Any schema static callable | Only names listed in `callable_statics` |
65
+ | `/cache/stats`, `/cache/clear` | No auth | Admin login required |
66
+ | `password_override=1` | Any authenticated user | Admin only |
67
+ | User PUT | Could set `admin` / `password` | Non-admins have privilege fields stripped |
68
+ | `?filter[$where]=…` | Accepted | **400** — operator denied |
69
+ | POST `/aggregate` | Any pipeline stage | Allowlisted stages; `$out` / `$merge` / `$function` need admin |
70
+ | POST `/bulkwrite` | Open | **Disabled per model** unless `advanced_queries.bulkwrite: true`; op allowlist |
71
+ | List passwords | Could leak on `GET /api/user` | Stripped from list/query/aggregate responses |
32
72
 
33
- ## v2.0.1-1
73
+ Example model opts:
34
74
 
35
- - Tokens as the preferred login methodology
75
+ ```js
76
+ new JXPSchema({ ... }, {
77
+ perms: { admin: "crud", user: "r" },
78
+ callable_statics: ["preview_segment", "apply_segment"],
79
+ advanced_queries: { bulkwrite: true }, // only if HTTP bulkwrite is required
80
+ });
81
+ ```
36
82
 
37
- ## v2.4.x
83
+ ---
38
84
 
39
- - Websockets
40
- - Bulk uploads
41
- - Advanced queries
85
+ ## v3.1.0 — 2026-05-19
42
86
 
43
- ## v2.5.x
87
+ - **Query limits for large collections** — `query_limits` config; `GET /api/<model>`, `GET /csv/<model>`, and `POST /query/<model>` require `?limit=` when collection size exceeds threshold (default 10,000 documents).
88
+ - New `query_limits.js` module, tests, and API documentation.
44
89
 
45
- - Documentation system
90
+ ---
46
91
 
47
- ## 2.6.x
92
+ ## v3.0.0 — 2025-04-17
48
93
 
49
- - Replace home-grown cache with apicache
50
- - Major performance boost when getting large collections (50secs to < 1 sec)
51
- - /setup/data endpoint for scaffolding a complete system
94
+ Repository refresh and dependency modernization:
52
95
 
53
- ## 2.7.x
96
+ - Better MongoDB connection handling.
97
+ - Proper HTTP errors on login failure.
98
+ - Additional request logging.
99
+ - Package upgrades; Docker and ESLint scaffolding.
54
100
 
55
- - Permanent Delete
56
- - Cascade Delete
101
+ ### Patch-level (same era)
102
+
103
+ - **2025-04-17** — Correct documentation for `x-api-key` header.
104
+ - **2025-04-05** — Fix `deepExtend` bug.
105
+
106
+ ---
107
+
108
+ ## v2.15.0 — 2025-03-25
109
+
110
+ - Fix potential stack size exceeded errors.
111
+ - Fix documentation circular reference errors.
112
+ - Dependency updates.
113
+ - Date range handling improvements.
114
+
115
+ ---
116
+
117
+ ## v2.14.6 — 2024-01-24
118
+
119
+ - Package upgrades.
120
+ - Log Mongoose version on startup.
121
+
122
+ ## v2.14.5 — 2023-12-21
123
+
124
+ - Maintenance release.
125
+
126
+ ## v2.14.4 — 2023-12-21
127
+
128
+ - Dependency bumps.
129
+
130
+ ## v2.14.3 — 2023-12-21
131
+
132
+ - `jxp-helper` version update.
133
+
134
+ ## v2.14.0 — 2023-02-27
135
+
136
+ - **`.env` support** — environment-based configuration instructions and loading.
137
+ - Link to external docs instead of inline documentation.
138
+ - Repository moved to [WorkSpaceMan/jxp](https://github.com/WorkSpaceMan/jxp) (2023-07-10).
139
+ - Dependency upgrades.
140
+
141
+ ### Caching (v2.14 era, 2023-01)
142
+
143
+ - In-memory response caching with smarter invalidation.
144
+ - Invalidate whole cache when links complicate partial invalidation.
145
+ - Don't crash when cache object isn't configured.
146
+
147
+ ---
148
+
149
+ ## v2.12.3 — 2023-01-18
150
+
151
+ - Dependency upgrades.
152
+ - Require Mongoose as peer dependency.
153
+
154
+ ## v2.12.0 — 2022-12-05
155
+
156
+ - Better error handling; server stays up on non-fatal errors.
157
+ - Tests passing again.
158
+ - Mongoose and Restify upgrades.
159
+ - Default to `127.0.0.1` instead of `localhost`.
160
+ - Show JXP version on load; reduce noisy logging.
161
+ - `--legacy-peer-deps` note for source installs.
162
+
163
+ ## v2.11.0 — 2022-11-28
164
+
165
+ - Pass `__user` on `get` and `getOne` actions.
166
+ - Async/`next` fixes.
167
+ - **Note:** Node versions above 17 were unsupported at this release.
168
+
169
+ ---
170
+
171
+ ## v2.7.0 — 2021-02-09
172
+
173
+ - **Permanent delete** — `?_permaDelete=1`.
174
+ - **Cascade delete** — `?_cascade=1`.
175
+ - **`_updated_by_id`** set on PUT; populated as `_updated_by`.
176
+ - Default `count` returns `-1` on very large collections (performance).
177
+ - `/setup/data` casts `_id` fields to ObjectIds and sets `_deleted: false`.
178
+
179
+ ## v2.6.x — 2021-01-06
180
+
181
+ - **apicache** replaces home-grown cache; configurable cache timeout.
182
+ - Major **GET performance** improvement on large collections.
183
+ - **Built-in documentation system** (MkDocs + in-server docs routes).
184
+ - Default port changed to **4001** (conflict on M1 Macs).
185
+ - Model docs refactored to separate library.
186
+ - Dependency upgrades (including Axios security fix).
187
+ - Return 404 for missing model.
188
+
189
+ ## v2.5.x — 2021-01-06
190
+
191
+ - Documentation system introduced (see v2.6.x).
192
+
193
+ ## v2.4.x — 2020-07 – 2020-08
194
+
195
+ - **WebSockets** — subscribe to model/item changes; auth via basic, apikey, or bearer token; filtering on subscriptions.
196
+ - **Bulk writes** — `/bulkwrite/<model>` endpoint.
197
+ - **`/query/<model>`** — POST advanced MongoDB queries.
198
+ - **`/aggregate/<model>`** — aggregation pipelines; `allowDiskUse` query param; inline ObjectId and Date strings.
199
+ - **`relative_date()`** in aggregation pipelines.
200
+ - **`/count/<model>`** endpoint.
201
+ - Populate `_owner_id` as `_owner`.
202
+ - OAuth login uses renewable API keys (2022-01).
203
+
204
+ ## v2.0.x — 2020-05-06 – 2020-07
205
+
206
+ Major v2 rewrite ([PR #11](https://github.com/WorkSpaceMan/jxp/pull/11)):
207
+
208
+ - Renamed project to **JXP**; MongoDB **connection string** (Atlas-compatible).
209
+ - **JXPSchema** — links, autopopulate, `ObjectId`/`Mixed` globals, automagic `_owner_id` / `_deleted`.
210
+ - **Bearer tokens** and refresh tokens as preferred auth.
211
+ - **Stored procedures** via `/call/<model>/<static>`.
212
+ - **Hooks** — `pre_hooks` / `post_hooks` on server config.
213
+ - Response shape `{ data }` for single records.
214
+ - Setup scaffolding (`jxp-setup`), link model, soft delete.
215
+ - Pre/post Mongoose middleware documented.
216
+ - WebSocket support started (completed in v2.4).
217
+
218
+ ---
219
+
220
+ ## v1.x — 2016 – 2019
221
+
222
+ ### v1.2.4 — 2019-11-06
223
+
224
+ - Time log includes operation number.
225
+
226
+ ### v1.2.3 — 2019-11-06
227
+
228
+ - Request **throttling**.
229
+
230
+ ### v1.0.16 — 2019-05-02
231
+
232
+ - **`POST /query/<model>`** for advanced queries.
233
+
234
+ ### v1.0.12 — 2019-04-03
235
+
236
+ - Select individual fields in populate joins.
237
+
238
+ ### v1.0.1 — 2019-02-13
239
+
240
+ - **Security:** only admins can assign user groups.
241
+
242
+ ### Earlier v1 highlights
243
+
244
+ - **2019-03** — CSV export; async/await refactor; non-blocking bcrypt password checks; caching (early).
245
+ - **2019-02** — Renamed to JXP; CORS plugin; Mongoose 5; full-text search.
246
+ - **2018-12** — Soft delete (`_deleted`); `showDeleted`; refactor from Q to async/await; `bcryptjs`.
247
+ - **2018-02** — Password field auto-encryption; config module.
248
+ - **2017** — Setup endpoint; pre-hooks; `model_dir` (relative/absolute); field selection; groups.
249
+ - **2016-05** — Initial release: REST CRUD from Mongoose models, auth, populate, filter, search, pagination, permissions, `jexpress-setup` CLI.
250
+
251
+ ---
252
+
253
+ ## Version notes
254
+
255
+ | Era | Mongoose | Node (documented) |
256
+ |-----|----------|-------------------|
257
+ | v4 | 6.13.9 | 22+ |
258
+ | v3.x | 6.x | — |
259
+ | v2.11 | — | ≤17 (warning in release) |
260
+ | v1 / early v2 | 5.x | — |
261
+
262
+ See the [v4.0.0 release](https://github.com/WorkSpaceMan/jxp/releases/tag/v4.0.0) and [compare view since v3.1.0](https://github.com/WorkSpaceMan/jxp/compare/7224093...v4.0.0).
@@ -1,162 +1,143 @@
1
- # Configuring JXP
1
+ # Configuration
2
2
 
3
- ## Connecting to MongoDB
3
+ JXP 4 uses **environment variables** and [dotenv](https://github.com/motdotla/dotenv). Copy [`.env.sample`](../.env.sample) to `.env` and adjust values.
4
4
 
5
- JXP uses MongoDB for storage. You can use a local MongoDB instance, or a hosted one such as Mongo Atlas.
5
+ _As of version 4, there is no `config` npm package and no `/config/*.json` hierarchy._
6
6
 
7
- The configuration for the MongoDB connection is set as environmental variables (for instance, if running in Docker), or in the `.env` file. Alternatively, you can set the connection string in the config file as described below, although this is not recommended.
7
+ ## MongoDB
8
+
9
+ Set a full connection string:
10
+
11
+ ```
12
+ MONGO_CONNECTION_STRING=mongodb://127.0.0.1:27017/myapp
13
+ ```
14
+
15
+ Or build from parts (common in Docker):
8
16
 
9
17
  ```
10
18
  MONGODB_HOST=127.0.0.1
11
19
  MONGODB_PORT=27017
12
- MONGODB_NAME=jxp
20
+ MONGODB_NAME=myapp
21
+ MONGODB_USER=
22
+ MONGODB_PASSWORD=
13
23
  MONGODB_AUTH_DB=admin
14
- MONGODB_USER=user
15
- MONGODB_PASS=pass
16
24
  ```
17
25
 
18
- ## Config files
26
+ Optional driver options as JSON:
19
27
 
20
- The config files are stored in `/config` in strict JSON format. (Remember to enclose your keys in inverted commas!) Typically, you'd use `/config/default.json` for a simple configuration, or `/config/development.json` and `/config/production.json` for separating development and production.
28
+ ```
29
+ MONGO_OPTIONS={"maxPoolSize":50}
30
+ ```
21
31
 
22
- If you used the `jxp-setup` command for setting everything up, you should have a basic `/config/default.json` file ready to go.
32
+ ## Server
23
33
 
24
- JXP uses [config](https://www.npmjs.com/package/config) for configuration. This allows you to have different configuration files for development, staging and production, set config variables through [environmental variables](https://github.com/lorenwest/node-config/wiki/Environment-Variables) or [command line overrides](https://github.com/lorenwest/node-config/wiki/Command-Line-Overrides), and do lots of other cool config magic.
34
+ | Variable | Description | Default |
35
+ |----------|-------------|---------|
36
+ | `PORT` | HTTP port | `4001` |
37
+ | `API_URL` / `API_SERVER` | Public base URL | `http://localhost:{PORT}` |
38
+ | `MODEL_DIR` | Path to `*_model.js` files | `./dist/models` (sample server) |
39
+ | `LOG_FILE` | Access log path | `access.log` |
40
+ | `SHARED_SECRET` | JWT / recovery signing | — |
41
+ | `TOKEN_EXPIRY` | Access token TTL (seconds) | `86400` |
42
+ | `REFRESH_TOKEN_EXPIRY` | Refresh token TTL (seconds) | `2678400` |
43
+ | `APIKEY` | Internal API key for schema helpers | — |
25
44
 
26
- ## Port
45
+ ## Cache
27
46
 
28
- The `port` key defines the port you want the server to run on. The default is `4001`.
29
- ```json
30
- "port": 4001,
47
+ ```
48
+ CACHE_ENABLED=true
49
+ CACHE_DEBUG=false
50
+ CACHE_TTL=300
31
51
  ```
32
52
 
33
- ## Model Directory
34
-
35
- JXP will search for a model directory with "user_model.js" in it, else you can override this with the config parameter `model_dir`. (This should be an absolute directory.)
36
-
37
- ## MongoDB
53
+ When enabled, JXP uses an in-process cache ([node-cache](https://www.npmjs.com/package/node-cache)). GET responses may include `jxp-cache` (`hit` / `miss`) and `jxp-cache-key` headers. Admin endpoints (**require admin login**):
38
54
 
39
- The `connection_string` can include port, username and password, options, and other fancy stuff. If you have advanced needs, you can find out more (here)[https://docs.mongodb.com/manual/reference/connection-string/].
55
+ - `GET /cache/stats` cache statistics, or `{ cache_enabled: false }` when disabled
56
+ - `GET /cache/clear` — flush all cached entries
40
57
 
41
- If you use Atlas, it will give you a connection string that you can use as-is.
58
+ See [Caching](caching.md) for details.
42
59
 
43
- The `options` allow you to pass extra options such as `poolSize`, `useNewUrlParser`, `useCreateIndex`, and `authSource`, amongst others. We use Mongoose's connection, so you can find out about the connection options (here)[https://mongoosejs.com/docs/connections.html]. Typically we only use `poolSize` and `authSource` in production, and no options in development.
60
+ ## Query limits
44
61
 
45
- A simple, local connection:
46
- ```json
47
- "mongo": {
48
- "connection_string": "mongodb://localhost/test",
49
- "options": {}
50
- },
51
62
  ```
52
-
53
- A Mongo Atlas connection:
54
- ```json
55
- "mongo": {
56
- "connection_string": "mongodb+srv://admin:<your password>@<your server>.gcp.mongodb.net/<your database>?retryWrites=true&w=majority",
57
- "options": {
58
- "readPreference": "nearest",
59
- "poolSize": 20,
60
- "authSource": "admin"
61
- }
62
- },
63
+ QUERY_LIMITS_ENABLED=true
64
+ QUERY_LIMITS_LARGE_COLLECTION_THRESHOLD=10000
65
+ QUERY_LIMITS_MAX=1000
66
+ QUERY_LIMITS_DEFAULT=100
67
+ QUERY_LIMITS_SKIP_COUNT_UNLESS_PAGINATED=true
68
+ CORS_ORIGINS=http://localhost:3000,http://localhost:4001
63
69
  ```
64
70
 
65
- ## Token lifespans
71
+ ## Throttling
66
72
 
67
- Set how long you want your users' tokens to live for. Defaults to 24 hours for a token and 30 days for a refresh token.
73
+ Optional JSON blob:
68
74
 
69
- ```json
70
- "token_expiry": 86400,
71
- "refresh_token_expiry": 2678400
72
75
  ```
73
-
74
- ## Log
75
-
76
- You can define an access log as follows:
77
- ```json
78
- "log": "/var/log/openmembers/access.log",
76
+ THROTTLE_JSON={"burst":100,"rate":50,"ip":true}
79
77
  ```
80
78
 
81
- ## Memcached
79
+ ## Model directory
82
80
 
83
- Memcached is entirely optional. Leave it out completely to not use Memcached.
81
+ JXP discovers models by scanning `MODEL_DIR` for `*_model.js`. You can also pass `model_dir` in the object given to `JXP(apiconfig)`.
84
82
 
85
- ```json
86
- "memcached": {
87
- "server": "localhost:11211",
88
- "lifetime": 604800
89
- },
90
- ```
83
+ Relative `model_dir` paths resolve from `process.cwd()` (typical for npm scripts), not from the server script path.
91
84
 
92
- ## Throttling
85
+ ## Programmatic config
86
+
87
+ Apps typically build an options object and pass it to `JXP()`:
93
88
 
94
- You can throttle the API performance. In the following example, we offer 5 requests per second, 10 concurrent requests, throttle by IP, and allow clients from 192.168.1.1 to go crazy.
95
- ```json
96
- "throttle": {
97
- "burst": 10,
98
- "rate": 5,
99
- "ip": true,
100
- "overrides": {
101
- "192.168.1.1": {
102
- "rate": 0,
103
- "burst": 0
104
- }
105
- }
106
- }
89
+ ```javascript
90
+ const JXP = require("jxp");
91
+ const server = JXP({
92
+ port: 4001,
93
+ mongo: { connection_string: process.env.MONGO_CONNECTION_STRING },
94
+ model_dir: "./models",
95
+ pre_hooks: { get: (req, res, next) => next() },
96
+ });
107
97
  ```
108
98
 
109
- ## Password recovery setup
99
+ The sample JXP server uses `loadJxpConfig()` from `jxp/libs/load-config` (resolved via `package.json` `exports` to compiled `dist/libs/load-config.js`) to build the same shape from `.env`.
110
100
 
111
- We have a secret that we share with the front end in order to create Javascript Web Tokens as a login option. This would allow you to send a link to a client that would auto-log them in. We also use it for password recovery links. If you used `jxp-setup` it would have generated a random one for you.
101
+ ### SMTP and password recovery
112
102
 
113
- We also configure a recovery url, which is a front-end page you'll want to hit, such as a password reset page.
103
+ SMTP and password-recovery URLs are **not** read from environment variables by `loadJxpConfig()`. Pass them on the `JXP()` options object:
114
104
 
115
- We configure an SMTP connection to send out forgotten password links.
105
+ ```javascript
106
+ JXP({
107
+ smtp_server: "mail.example.com",
108
+ smtp_username: "user",
109
+ smtp_password: "secret",
110
+ smtp_from: "noreply@example.com",
111
+ password_recovery_url: "https://myapp.example.com/reset",
112
+ });
113
+ ```
116
114
 
117
- ```json
118
- "shared_secret": "SomeRandomString",
119
- "password_recovery_url": "https://blah.com/login/reset",
120
- "smtp": {
121
- "host": "smtp.gmail.com",
122
- "port": 587,
123
- "auth": {
124
- "user": "blah@blah.com",
125
- "pass": "blah"
115
+ Used by `POST /login/recover` (see [Authentication](authentication.md)).
116
+
117
+ ### OAuth
118
+
119
+ OAuth2 providers are configured programmatically on the `oauth` key:
120
+
121
+ ```javascript
122
+ JXP({
123
+ url: "http://localhost:4001",
124
+ oauth: {
125
+ success_uri: "https://myapp.example.com/oauth/success",
126
+ fail_uri: "https://myapp.example.com/oauth/fail",
127
+ google: {
128
+ auth_uri: "https://accounts.google.com/o/oauth2/v2/auth",
129
+ token_uri: "https://oauth2.googleapis.com/token",
130
+ api_uri: "https://www.googleapis.com/oauth2/v2/userinfo",
131
+ app_id: "...",
132
+ app_secret: "...",
133
+ scope: "email profile",
126
134
  },
127
- "secureConnection": true
128
- },
135
+ },
136
+ });
129
137
  ```
130
138
 
131
- ## oAuth
132
-
133
- If you want users to be able to log in with an oAuth 2 provider, such as Google or Facebook, you would set it up as follows.
134
-
135
- We have a `success_uri` and `fail_uri` that will direct the user to your front end's success and failure page, respectively.
139
+ Routes: `GET /login/oauth/:provider` and `GET /login/oauth/callback/:provider`. See [Authentication](authentication.md).
136
140
 
137
- Then, for each service you'd like to support, define the `app_id`, `app_secret`, `scope`, `auth_uri`, `token_uri` and `api_uri`.
141
+ ## Tests
138
142
 
139
- Here are some examples. We used to support LinkedIn but they made some weird decisions that effectively broke their oAuth2 support.
140
-
141
- ```
142
- "oauth": {
143
- "success_uri": "https://my.workshop17.co.za/login/oauth",
144
- "fail_uri": "https://my.workshop17.co.za/login/oauth/fail",
145
- "facebook": {
146
- "app_id": "<your app id>",
147
- "app_secret": "<your app secret>",
148
- "scope": "email",
149
- "auth_uri": "https://www.facebook.com/dialog/oauth",
150
- "token_uri": "https://graph.facebook.com/v2.3/oauth/access_token",
151
- "api_uri": "https://graph.facebook.com/me?fields=id,name,about,age_range,email,picture"
152
- },
153
- "google": {
154
- "app_id": "<your app id>",
155
- "app_secret": "<your app secret>",
156
- "auth_uri": "https://accounts.google.com/o/oauth2/auth",
157
- "scope": "email+profile",
158
- "api_uri": "https://www.googleapis.com/oauth2/v1/userinfo?alt=json",
159
- "token_uri": "https://www.googleapis.com/oauth2/v3/token"
160
- }
161
- },
162
- ```
143
+ Tests load `.env.test` via `test/env.js` (see `npm test`). Set `NODE_ENV=test` and `MONGO_CONNECTION_STRING` for your local Mongo instance.
package/docs/hooks.md CHANGED
@@ -1,8 +1,8 @@
1
1
  # Hooks
2
2
 
3
- In `/bin/server.js` you will find some pre-operation hooks. These will run after `auth` but before the operation.
3
+ In your app's server bootstrap (eg. `dist/bin/server.js` compiled from `src/bin/server.ts`), configure `pre_hooks` and `post_hooks` on the object passed to `JXP()`. These run after authentication but before (or after, for post-hooks) the operation.
4
4
 
5
- We've used callbacks to add parameters on `get` and `getOne` to our queries to limit what certain users see in the results. We've also use the `post`, `put` and `delete` to fire emits on an external Websocket server.
5
+ We've used pre-hooks on `get` and `getOne` to add query constraints so certain users only see allowed rows. We've also used `post`, `put`, and `delete` hooks alongside JXP's built-in WebSocket emitter to notify subscribed clients of changes.
6
6
 
7
7
  ```javascript
8
8
  config.pre_hooks = {
@@ -21,19 +21,23 @@ config.pre_hooks = {
21
21
  put: (req, res, next) => {
22
22
  next();
23
23
  },
24
+ update: (req, res, next) => {
25
+ next();
26
+ },
24
27
  delete: (req, res, next) => {
25
28
  next();
26
29
  },
27
30
  };
28
31
  ```
29
32
 
30
- For _login_ there's also a post-hook, eg. if you need to inject more information into the login result.
33
+ For _login_ there is also a post-hook if you need to inject more information into the login result:
31
34
 
32
35
  ```javascript
33
- config.post_hooks = (req, res, next) => {
34
- res.result.some_field = "Some Value";
35
- next();
36
- }
36
+ config.post_hooks = {
37
+ login: (req, res) => {
38
+ res.result.some_field = "Some Value";
39
+ },
40
+ };
37
41
  ```
38
42
 
39
- To suppress a callback, pass `_silence=true` as a parameter. This helps avoid infinite loops, for instance when your PUT updates the same model.
43
+ To suppress model callbacks (eg. to avoid infinite loops when a PUT updates the same model), pass `_silence=true` as a query parameter or in the request body.
package/docs/index.md CHANGED
@@ -1,26 +1,26 @@
1
- # Welcome to JXP
1
+ # JXP API
2
2
 
3
- JXP is an opinionated API framework that you can change just by defining schemas.
3
+ JXP is an opinionated REST framework: define Mongoose schemas and get a production-ready API with authentication, permissions, population, filtering, and more.
4
+
5
+ JXP 4 requires **Node.js 22+**, ships as compiled TypeScript, and uses environment-based configuration (see [Installation](installation.md) and [TypeScript](typescript.md)).
4
6
 
5
7
  ## Features
6
8
 
7
- * Takes care of the plumbing, like authentication and password encryption
8
- * Get all the data you need with smart link population
9
- * Search, sort, filter, populate... all on the url
10
- * Get just the data you need with field selection
11
- * Super-fast and easy to add and change models
12
- * An obvious and standard API interface
13
- * CSV output to quickly grab your data
14
- * Permissioning and groups baked in
15
- * Ability to handle business logic
16
- * Simple to deploy
17
- * Lightning-fast
18
- * Stateless
19
- * Subscribe to changes through Websocket
9
+ * Authentication and password encryption handled for you
10
+ * Smart link population on read
11
+ * Search, sort, filter, and populate via the URL
12
+ * Field selection to return only what you need
13
+ * Fast model changes schemas drive the API
14
+ * Standard REST interface (GET, POST, PUT, DELETE)
15
+ * CSV export for collections
16
+ * Groups and per-model permissions
17
+ * Hooks for business logic
18
+ * Stateless and easy to deploy
19
+ * WebSocket change subscriptions
20
20
 
21
21
  ## Links
22
22
 
23
23
  * [JXP on NPM](https://www.npmjs.com/package/jxp)
24
- * [GitHub Repository](https://github.com/j-norwood-young/jexpress-2)
25
- * [JXP Helper](https://www.npmjs.com/package/jxp-helper) - A super-helpful client library for JXP
24
+ * [GitHub Repository](https://github.com/WorkSpaceMan/jxp)
25
+ * [JXP Helper](https://www.npmjs.com/package/jxp-helper) client library for JXP
26
26
  * [Documentation on ReadTheDocs](https://jxp.readthedocs.io)