jxp 3.0.0 → 4.1.0

package/docs/aggregations.md

@@ -1,25 +1,39 @@
 # Aggregation Queries
 
-You can apply an aggregation pipeline through the `/aggregate/<modelname>` endpoint. See [MongoDB's aggregation documentation](https://docs.mongodb.com/manual/aggregation/).
+You can apply an aggregation pipeline through the `/aggregate/<modelname>` endpoint. See [MongoDB's aggregation documentation](https://www.mongodb.com/docs/manual/aggregation/).
 
-POST the query as JSON, and wrap it with a "query" variable, like so *:
-```JSON
+Only allowlisted stages are permitted (e.g. `$match`, `$group`, `$lookup`, `$project`, `$sort`, `$limit`). Stages such as `$out`, `$merge`, and `$function` require an admin user. Disable per model with `advanced_queries: { aggregate: false }`.
+
+POST the pipeline as a **JSON array**, or wrap it in a `"query"` property. Both forms are accepted:
+
+```json
+[
+    {
+        "$group": {
+            "_id": null,
+            "count": {
+                "$sum": 1
+            }
+        }
+    }
+]
+```
+
+```json
 {
     "query": [
         {
-            "$group": { 
+            "$group": {
                 "_id": null,
-                "count": { 
-                    "$sum": 1 
-                } 
-            } 
+                "count": {
+                    "$sum": 1
+                }
+            }
         }
     ]
 }
 ```
 
-* Note that embedding in a "query" is no longer necessary
-
 ### Aggregations with ObjectIds
 
 Because we can't define ObjectIds in our aggregate functions, we need to send the ObjectIds as strings and then convert them in the pipeline.
@@ -27,12 +41,13 @@ Because we can't define ObjectIds in our aggregate functions, we need to send th
 There are two ways of doing this. We can embed `"ObjectId(\"<your object id>\")"` or you can convert in the pipeline. Embedding will be faster on execution.
 
 Embedding:
-```JSON
+
+```json
 {
     "query": [
         {
             "$match": {
-                "$campaign_id", "ObjectId(\"5fd45d05f2b93af8d59588fb\")"
+                "campaign_id": "ObjectId(\"5fd45d05f2b93af8d59588fb\")"
             }
         }
     ]
@@ -40,13 +55,14 @@ Embedding:
 ```
 
 Using a pipeline to add a field:
-```JSON
+
+```json
 {
     "query": [
-        { 
+        {
             "$addFields": {
                 "campaign_id_obj": {
-                    "$toObjectId":  "5fd45d05f2b93af8d59588fb"
+                    "$toObjectId": "5fd45d05f2b93af8d59588fb"
                 }
             }
         },
@@ -70,12 +86,13 @@ Because we can't define Date objects in our aggregate functions, we need to send
 There are two ways of doing this. We can embed `"new Date(\"<your date>\")"` or you can convert in the pipeline. Embedding will be faster on execution.
 
 Embedding:
-```JSON
+
+```json
 {
     "query": [
         {
             "$match": {
-                "$timestamp": {
+                "timestamp": {
                     "$gte": "new Date(\"2021-03-03T00:00:00.0Z\")"
                 }
             }
@@ -85,7 +102,8 @@ Embedding:
 ```
 
 Using a pipeline to add a date field:
-```JSON
+
+```json
 {
     "query": [
         {
@@ -115,12 +133,13 @@ Using a pipeline to add a date field:
 Say you want to use a date relative to today's date, you can use `relative_date(offset, offset_unit, startof_unit, endof_unit)`, similar to the method of embedding in a string above. It will also take `null` as a value.
 
 Eg. to get the beginning of this month:
-```JSON
+
+```json
 {
     "query": [
         {
             "$match": {
-                "$timestamp": {
+                "timestamp": {
                     "$gte": "relative_date(null, null, \"month\")"
                 }
             }
@@ -131,4 +150,4 @@ Eg. to get the beginning of this month:
 
 ### AllowDiskUse
 
-***Tip*** Aggregates can use a lot of memory. If you're unable to complete your query, try using the disk. To enable allowDiskUse, add `?allowDiskUse=true` as a query parameter to the calling url.
+Aggregates can use a lot of memory. If you're unable to complete your query, try using the disk. To enable `allowDiskUse`, add `?allowDiskUse=true` as a query parameter to the calling URL.

package/docs/api.md

@@ -92,6 +92,41 @@ You can paginate with the `page=<page number>` parameter. Page count starts at 1
 
 If you go beyond the total number of pages, you will get an empty `data` array.
 
+#### Default limit
+
+When `query_limits.enabled` is true (default), list and query endpoints apply **`?limit=100`** if you omit `limit`. Use `?limit=` up to the configured maximum (default **1000**) for larger pages.
+
+#### Large collections (explicit limit)
+
+For collections with at least **10,000** documents (by default), you must pass an explicit `?limit=` between 1 and the maximum. Omitting `limit` on a large collection returns **400 Bad Request** (the default limit does not satisfy this rule).
+
+Use `GET /count/<model>` for totals. Add **`?count=true`** (or **`?page=`**) when you need `count` / `page_count` in the list response without paginating.
+
+#### Totals in list responses
+
+By default, list responses **omit** `count` unless you pass **`?count=true`** or **`?page=`**, to avoid expensive `countDocuments` on every request.
+
+Configure globally when starting JXP:
+
+```js
+query_limits: {
+  enabled: true,
+  large_collection_threshold: 10000,
+  max: 1000,
+  default: 100,
+  skip_count_unless_paginated: true,
+}
+```
+
+Override per model in schema options:
+
+```js
+const ReaderSchema = new JXPSchema({ ... }, {
+  perms: { ... },
+  query_limits: { large_collection_threshold: 5000, max: 500 },
+});
+```
+
 *Note* If you have more than 100,000 items in your collection, `count` will return as -1, else doing a full-dataset filtered count becomes too expensive and can cause serious performance issues.
 
 ### Counting
@@ -149,7 +184,7 @@ To ensure a full-text index across all fields on your model, add this to your sc
 
 `MySchema.index( { "$**": "text" } );`
 
-See [https://docs.mongodb.com/manual/core/index-text/](MongoDB Text Indexes) for more options, such as weighted indexing.
+See [MongoDB Text Indexes](https://www.mongodb.com/docs/manual/core/index-text/) for more options, such as weighted indexing.
 
 Note that you can only declare one index per collection (and hence schema).
 
@@ -168,28 +203,33 @@ As with PUT, we need to reference a specific item, so the endpoint needs to incl
 
 ## Advanced queries
 
-If you need to send an advanced query, such as a combined $and/$or, you can _POST_ a `{query}` variable to the `/query/{modelname}` endpoint. Most of the other features you'd use for `/get/{modelname}` (except for _search_ since it's a query) will still work.
+If you need to send an advanced query, such as a combined `$and`/`$or`, POST JSON to `/query/{modelname}` with a **`query` object** (required). Most URL features from `GET /api/{modelname}` still apply (`limit`, `page`, `sort`, `populate`, `filter`, `autopopulate`, `fields`) except `search`.
 
 Eg.
-```javascript
-query = {
-    "$and": [
-        { 
-            "foo": {
-                "$regex": "foo",
-                "$options": "i"
+
+```json
+{
+    "query": {
+        "$and": [
+            {
+                "foo": {
+                    "$regex": "foo",
+                    "$options": "i"
+                }
+            },
+            {
+                "bar": "Bar"
             }
-        },
-        {	
-            "bar": "Bar"
-        }
-    ]
+        ]
+    }
 }
 ```
 
+See [Queries](queries.md) for more examples.
+
 ## Aggregate queries
 
-You can perform an aggregate query by POSTing your query to `/aggregate/{modelname}`. The aggregation pipeline must be wrapped in an array
+You can perform an aggregate query by POSTing a pipeline array to `/aggregate/{modelname}`, or wrapping it in `{ "query": [ ... ] }`. See [Aggregations](aggregations.md).
 
 Eg.
 ```javascript

package/docs/authentication.md

@@ -1,19 +1,17 @@
 # Authentication
 
-## API Key
-
 There are four ways of authenticating:
 
 * Basic Auth
 * A Bearer Token
 * An API Key
-* A Javascript Web Token
+* A Javascript Web Token (JWT)
 
-We always use `email` and `password` to identify the user. The passwords are always one-way encrypted using bcrypt.
+We always use `email` and `password` to identify the user. Passwords are one-way encrypted using bcrypt.
 
-In a typical application, your front-end site would present a login page asking for the user's email and password. In addition, you would present a "Forgotten Password" link. 
+In a typical application, your front-end site would present a login page asking for the user's email and password. In addition, you would present a "Forgotten Password" link.
 
-When the user submits their username and password, you would POST that data to the `/login` endpoint. If the login succeeds, the page will return their API key and bearer token. 
+When the user submits their username and password, you would POST that data to the `/login` endpoint. If the login succeeds, the response includes their API key and bearer token.
 
 ## Login endpoints
 
@@ -22,7 +20,9 @@ When the user submits their username and password, you would POST that data to t
 Logging in will always delete the previous token and give you a new one.
 
 POST `http://localhost:4001/login`
+
 Data:
+
 ```json
 {
     "email": "blah@blah.com",
@@ -31,6 +31,7 @@ Data:
 ```
 
 Successful Response (Status 200):
+
 ```json
 {
     "user_id": "5dadbd7e2384ad419975e4a1",
@@ -43,6 +44,7 @@ Successful Response (Status 200):
 ```
 
 Failed Response (Status 401):
+
 ```json
 {
     "status": "fail",
@@ -53,19 +55,22 @@ Failed Response (Status 401):
 
 ### Refresh Token
 
-You can use your refresh_token to refresh a token, even if it's expired. By default, refresh tokens last 30 days, whereas tokens last 24 hours.
+You can use your refresh_token to refresh a token, even if it's expired. By default, refresh tokens last 30 days, whereas tokens last 24 hours (configurable via `REFRESH_TOKEN_EXPIRY` and `TOKEN_EXPIRY`).
 
 Note that the response is almost identical to the `/login` endpoint, except it doesn't have the `apikey`.
 
 POST `http://localhost:4001/refresh`
-Header: 
+
+Header:
+
 ```json
 {
-    "Authorization": "Bearer <refresh token>",
+    "Authorization": "Bearer <refresh token>"
 }
 ```
 
 Successful Response (Status 200):
+
 ```json
 {
     "user_id": "5dadbd7e2384ad419975e4a1",
@@ -77,6 +82,7 @@ Successful Response (Status 200):
 ```
 
 Failed Response (Status 401):
+
 ```json
 {
     "status": "fail",
@@ -87,65 +93,74 @@ Failed Response (Status 401):
 
 ### Logout
 
-This will immeditately expire the token.
+This will immediately expire the token. You must be authenticated (Bearer token or other method).
+
+GET `http://localhost:4001/login/logout`
+
+GET `http://localhost:4001/logout`
 
-POST or GET `http://localhost:4001/login/logout`
+Successful Response (200):
+
+```json
+{
+    "status": "ok",
+    "message": "User logged out"
+}
+```
 
 ### Recover Password
 
-Send the user an email with their JWT embedded so that they can reset their password
+Send the user an email with a JWT embedded so that they can reset their password. Requires SMTP settings on the `JXP()` config object (see [Configuration](configuration.md#smtp-and-password-recovery)).
 
 POST `http://localhost:4001/login/recover`
 
 Data:
-```
+
+```json
 {
     "email": "blah@blah.com"
 }
 ```
 
-Successful Response (200): 
-```
+Successful Response (200):
+
+```json
 {
     "status": "ok",
     "message": "Sent recovery email"
 }
 ```
 
-Failed Response (403):
-```
-{
-    "status": "fail",
-    "message": "Unauthorized",
-    "err": "Could not find email"
-}
-```
+Failed responses include 400 (missing email), 404 (user not found), or 401/500 depending on configuration errors.
 
-***Note:*** You will still have to build the password reset page on your front end.
+***Note:*** You will still have to build the password reset page on your front end. The recovery link uses `password_recovery_url` from config plus the JWT token.
 
 ### JWT
 
-A Javascript Web Token can be used to log the user in through a URL. 
+A Javascript Web Token can be used to log the user in through a URL.
 
 POST `http://localhost:4001/login/getjwt`
 
 Data:
-```
+
+```json
 {
     "email": "blah@blah.com"
 }
 ```
 
-Successful Response (200): 
-```
+Successful Response (200):
+
+```json
 {
     "status": "ok",
     "jwt": "<jwt>"
 }
 ```
 
-Failed Response (403):
-```
+Failed Response (403 or 404):
+
+```json
 {
     "status": "fail",
     "message": "Unauthorized",
@@ -153,7 +168,16 @@ Failed Response (403):
 }
 ```
 
-## Authenticating
+### OAuth2
+
+OAuth login is configured programmatically (see [Configuration](configuration.md#oauth)).
+
+- `GET /login/oauth/:provider` — redirects to the provider's authorization URL
+- `GET /login/oauth/callback/:provider` — handles the callback and redirects to `oauth.success_uri` or `oauth.fail_uri` with a token or error
+
+The provider name (`:provider`) must match a key under `oauth` in your config (excluding `success_uri` and `fail_uri`).
+
+## Authenticating API requests
 
 ### Basic Auth
 
@@ -161,11 +185,11 @@ Basic auth encodes (NOTE: ***NOT*** encrypts) your username and password and sen
 
 ***WARNING:*** You must only use basic auth over SSL, since it is trivial to decode the username and password. In fact, you should use SSL for everything, anyway.
 
-A basic auth is created by base64-encoding your username and password, separated by a colon. 
+A basic auth token is created by base64-encoding your username and password, separated by a colon.
 
-Eg. `echo "blah@blah.com:password" | base64` would generate a basic auth token on the command line. (In this case, it would encode to `YmxhaEBibGFoLmNvbTpwYXNzd29yZAo=`.)
+Eg. `echo -n "blah@blah.com:password" | base64` would generate a basic auth token on the command line.
 
-However, `echo "YmxhaEBibGFoLmNvbTpwYXNzd29yZAo=" | base64 --decode` would reveal the uesrname and password, which is why it's not safe to use it on an unencrypted connection.
+However, `echo "YmxhaEBibGFoLmNvbTpwYXNzd29yZA==" | base64 --decode` would reveal the username and password, which is why it's not safe to use it on an unencrypted connection.
 
 Header: `Authorization: Basic <your basic token>`
 
@@ -173,11 +197,10 @@ Header: `Authorization: Basic <your basic token>`
 
 _This is the preferred method of authenticating._
 
-Bearer tokens are ephemeral tokens that will expire in a certain time period. When a user logs out of a session, they will be destroyed.
+Bearer tokens are ephemeral tokens that will expire after a configured period. When a user logs out of a session, they are revoked.
 
 Header: `Authorization: Bearer <your bearer token>`
 
 ### API Key
 
-The API Key is a permanent key that doesn't expire. It can be used by adding `?apikey=<apikey>` to the end of any request, or sending `x-api-key: <apikey>` in the header. 
-
+The API Key is a permanent key that doesn't expire. It can be used by adding `?apikey=<apikey>` to the end of any request, or sending `x-api-key: <apikey>` in the header.

package/docs/bulk_writes.md

@@ -1,20 +1,23 @@
 # Bulk Writes
 
-You can bulk writes through the `/bulkwrite/<modelname>` endpoint. Note that authorisation works a little differently - the user must have permissions to Create, Edit, Update and Read.
+You can perform bulk writes through the `/bulkwrite/<modelname>` endpoint. **Disabled by default** unless the model opts in with `advanced_queries: { bulkwrite: true }`. Only allowlisted operations (`insertOne`, `updateOne`, `replaceOne`, `deleteOne`) are accepted; `updateMany` / `deleteMany` require admin.
+
+Authorisation works a little differently — the user must have **create, read, update, and delete** permissions on the model.
 
 ***WARNING*** Be cautious when using bulk writes because you can destroy your data. Don't forget to back up!
 
 Eg:
-```JSON
+
+```json
 [
     {
         "insertOne": {
             "document": {
                 "foo": "Foo2",
                 "bar": "Bar2",
-                "yack": { "yack": "yack2", "shmack": 2 },
+                "yack": { "yack": "yack2", "shmack": 2 }
             }
-        },
+        }
     },
     {
         "updateOne": {
@@ -36,10 +39,10 @@ Eg:
             "update": {
                 "foo": "Foo3",
                 "bar": "Bar3",
-                "yack": { "yack": "yack3", "shmack": 3 },
+                "yack": { "yack": "yack3", "shmack": 3 }
             },
             "upsert": true
-        },
+        }
     }
-];
-```
+]
+```

package/docs/caching.md

@@ -1,13 +1,30 @@
 # Caching
 
-Enable caching to speed up your queries. Caching is disabled by default. Enable caching through the config:
-
-```JSON
-{
-    "cache": {
-        "enabled": true,
-        "debug": false,
-        "ttl": 3600
-    }
-}
-```
+Enable caching to speed up repeated GET requests. Caching is disabled by default. Enable via environment variables (or pass `cache` in the object to `JXP()`):
+
+```
+CACHE_ENABLED=true
+CACHE_DEBUG=false
+CACHE_TTL=3600
+```
+
+## How it works
+
+JXP uses an in-process [node-cache](https://www.npmjs.com/package/node-cache) store (not Memcached or Redis). When caching is enabled:
+
+- Successful GET responses for list and single-item routes may be stored under a key derived from the model name, item id, and query string.
+- Response headers include:
+  - `jxp-cache`: `hit` or `miss`
+  - `jxp-cache-key`: the cache key used for that request
+- POST, PUT, and DELETE operations clear cached keys for the affected model.
+
+Set `CACHE_DEBUG=true` to log cache expiry and flush events to the console.
+
+## Admin endpoints
+
+Requires **admin** authentication (same as other admin-only routes):
+
+- `GET /cache/stats` — returns node-cache statistics when caching is enabled, or `{ "cache_enabled": false }` when disabled
+- `GET /cache/clear` — flushes all cached entries
+
+See also [Configuration](configuration.md#cache).