integreat 1.6.7 → 1.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -5,10 +5,9 @@ An integration layer written in TypeScript.
5
5
  [![npm Version](https://img.shields.io/npm/v/integreat.svg)](https://www.npmjs.com/package/integreat)
6
6
  [![Maintainability](https://api.codeclimate.com/v1/badges/a5bd9841a47ff9f74577/maintainability)](https://codeclimate.com/github/integreat-io/integreat/maintainability)
7
7
 
8
- The basic idea of Integreat is to make it easy to define how to send data to
9
- and receive data from a set of [**services**](#services), and expose them
10
- through a well defined interface, abstracting away the specifics of each
11
- service.
8
+ The basic idea of Integreat is to make it easy to define how to send data to and
9
+ receive data from a set of [**services**](#services), and expose them through a
10
+ well defined interface, abstracting away the specifics of each service.
12
11
 
13
12
  There are a few concepts that makes this possible:
14
13
 
@@ -18,8 +17,8 @@ There are a few concepts that makes this possible:
18
17
  deal with familiar JavasScript objects, arrays, and primitive data types,
19
18
  regardless of what the service expects.
20
19
  - [**Mutation pipelines**](#mutations) let you define how the data coming from
21
- or going to a service should be transformed. This includes changing the
22
- overal structure, renaming properties, transforming and filtering values with
20
+ or going to a service should be transformed. This includes changing the overal
21
+ structure, renaming properties, transforming and filtering values with
23
22
  transformer functions, etc. You may also provide your own transformer
24
23
  functions.
25
24
  - [**Schemas**](#schemas) serve as a common normalization of data between
@@ -28,9 +27,9 @@ There are a few concepts that makes this possible:
28
27
  may send it to any service where you have set up the right mutations for this
29
28
  schema, again abstracting away all service details.
30
29
 
31
- All configuration is done through basic JSON-friendly structures, and you
32
- define your services with different endpoints, mutation pipelines,
33
- authentication schemes, etc.
30
+ All configuration is done through basic JSON-friendly structures, and you define
31
+ your services with different endpoints, mutation pipelines, authentication
32
+ schemes, etc.
34
33
 
35
34
  Your configuration is spun up as an Integreat instance. To send and retrieve
36
35
  data, you dispatch [**actions**](#actions) to your instance and get
@@ -81,7 +80,8 @@ though, depends on how you define your services.
81
80
  Requires node v18.
82
81
 
83
82
  Note: This package is native [ESM](https://nodejs.org/api/esm.html). See this
84
- guide on how to [convert to or use ESM packages](https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c).
83
+ guide on how to
84
+ [convert to or use ESM packages](https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c).
85
85
 
86
86
  Install from npm:
87
87
 
@@ -101,9 +101,8 @@ usefulness of Integreat, but it shows you the simplest setup possible.
101
101
 
102
102
  Here, we fetch cat facts from the API endpoint
103
103
  'https://cat-fact.herokuapp.com/facts', which returns data in JSON and requires
104
- no authentication. The returned list of facts are mutated and cast to the
105
- `fact` schema. We only fetch data _from_ the service, and no data is sent _to_
106
- it.
104
+ no authentication. The returned list of facts are mutated and cast to the `fact`
105
+ schema. We only fetch data _from_ the service, and no data is sent _to_ it.
107
106
 
108
107
  ```javascript
109
108
  import Integreat from 'integreat'
@@ -188,8 +187,8 @@ The `response` object will look like this:
188
187
 
189
188
  # Integreat concepts
190
189
 
191
- As mentioned in the introduction, the building blocks of Integreat are
192
- services, transporters and adapters, mutation pipelines, and schemas.
190
+ As mentioned in the introduction, the building blocks of Integreat are services,
191
+ transporters and adapters, mutation pipelines, and schemas.
193
192
 
194
193
  ## Services
195
194
 
@@ -197,28 +196,27 @@ A service is the API, database, FTP server, queue, etc. that you want to get
197
196
  data from and/or set data to. We pass on a set of service definitions to
198
197
  Integreat, specifying what transporter, adapters, authentication schemas it
199
198
  requires, in adition to defining the different endpoints available on the
200
- service, how they should be called, and how data should be mutated in each
201
- case.
199
+ service, how they should be called, and how data should be mutated in each case.
202
200
 
203
201
  We'll get back to the details of all of this in turn, but first we want to
204
202
  highlight how central the concept of a service is to Integreat. Basically, in
205
203
  Integreat "everything is a service". A simple REST/JSON API is a service, a
206
204
  database is a service, and everything external you want to communicate with are
207
- services. Want to set up a queue to handle actions one by one? That's a
208
- service. Want to cache data in a memory store? That's a service. Want to
209
- schedule actions to run on intervals? That's a service to. By simply defining
210
- services and their specifics, you may set up a variety of different types of
211
- configurations with the same few building blocks. This is very powerful as
212
- soon as you get into the right mindset.
205
+ services. Want to set up a queue to handle actions one by one? That's a service.
206
+ Want to cache data in a memory store? That's a service. Want to schedule actions
207
+ to run on intervals? That's a service to. By simply defining services and their
208
+ specifics, you may set up a variety of different types of configurations with
209
+ the same few building blocks. This is very powerful as soon as you get into the
210
+ right mindset.
213
211
 
214
- Services are configured by service definitions, and tells Integreat how to
215
- fetch data from a service, how to mutate this data to schemas, and how to send
216
- data back to the service.
212
+ Services are configured by service definitions, and tells Integreat how to fetch
213
+ data from a service, how to mutate this data to schemas, and how to send data
214
+ back to the service.
217
215
 
218
216
  The service definition object includes the transporter id, adapter ids, any
219
217
  authentication method, the endpoints for fetching from and sending to the
220
- service, mutations that data to all endpoints will pass through, and options
221
- for transporters, adapters, etc.
218
+ service, mutations that data to all endpoints will pass through, and options for
219
+ transporters, adapters, etc.
222
220
 
223
221
  ```javascript
224
222
  {
@@ -245,17 +243,16 @@ pipeline for a service.
245
243
 
246
244
  The `auth` property should normally be set to the id of an
247
245
  [auth definition](#service-authentication), if the service requires
248
- authentication. In cases where the service is authenticated by other means,
249
- e.g. by including username and password in the uri, set the `auth` property to
250
- `true` to signal that this is an authenticated service. For services accepting
251
- incoming actions, `auth` should be set to an object with
246
+ authentication. In cases where the service is authenticated by other means, e.g.
247
+ by including username and password in the uri, set the `auth` property to `true`
248
+ to signal that this is an authenticated service. For services accepting incoming
249
+ actions, `auth` should be set to an object with
252
250
  `{ outgoing: <auth id | true>, incoming: <auth id | true>}`. To accept several
253
251
  incoming actions, provide an array of `<auth id | true>`, and they will be run
254
252
  from first to last until one of them returns an ident or an error other than
255
253
  `noaccess`.
256
254
 
257
- > [!NOTE]
258
- > When connecting to a service for listening, the `outgoing` auth is
255
+ > [!NOTE] When connecting to a service for listening, the `outgoing` auth is
259
256
  > used. `incoming` is only used for validating the actions being dispatched
260
257
  > "back" from the service.
261
258
 
@@ -267,8 +264,8 @@ merged with the `options` object on the endpoint. See
267
264
 
268
265
  A service will have at least one endpoint, but often there will be several.
269
266
  Endpoints are the definitions of the different ways Integreat may interact with
270
- a service. You decide how you want to set up the endpoints and what is the
271
- right "endpoint design" for a service, but there might be one endpoint for each
267
+ a service. You decide how you want to set up the endpoints and what is the right
268
+ "endpoint design" for a service, but there might be one endpoint for each
272
269
  operation that can be done with a type of data.
273
270
 
274
271
  For example, let's say you have a simple REST API with blog articles and
@@ -287,11 +284,10 @@ one, creating, updating, and the same all over for users. Instead of urls and
287
284
  http verbs, as for REST, these endpoints will address different databases and
288
285
  different database operations (through the transporter).
289
286
 
290
- > [!NOTE]
291
- > This is not to say that Integreat requires you to set up endpoints
292
- > exactly as described in these examples, it might be that you would like to
293
- > set up an endpoint that handles many of these cases. The intention here is
294
- > just to give you an understanding of what an endpoint is in Integreat.
287
+ > [!NOTE] This is not to say that Integreat requires you to set up endpoints
288
+ > exactly as described in these examples, it might be that you would like to set
289
+ > up an endpoint that handles many of these cases. The intention here is just to
290
+ > give you an understanding of what an endpoint is in Integreat.
295
291
 
296
292
  When you dispatch an action, Integreat will figure out what service and what
297
293
  endpoint to send the action to. The target service is often specified in the
@@ -301,14 +297,13 @@ property, will be used.
301
297
 
302
298
  The `targetService` property will be set on the the action payload when it is
303
299
  sent to the transporter, before it goes through the middleware. There are two
304
- exceptions to this, however. `targetService` will not be set for a queued
305
- action (and it will not be removed if it is already set), and it will not be
306
- set for actions where the `meta.options.doSetTargetService` flag is set to
307
- `false`.
300
+ exceptions to this, however. `targetService` will not be set for a queued action
301
+ (and it will not be removed if it is already set), and it will not be set for
302
+ actions where the `meta.options.doSetTargetService` flag is set to `false`.
308
303
 
309
- The matching to an endpoint is done by finding the endpoint whose `match`
310
- object matches the action with most accuracy. The rules of the endpoint
311
- matching is describe in more details [below](#match-properties).
304
+ The matching to an endpoint is done by finding the endpoint whose `match` object
305
+ matches the action with most accuracy. The rules of the endpoint matching is
306
+ describe in more details [below](#match-properties).
312
307
 
313
308
  Here's the format of an endpoint definition:
314
309
 
@@ -345,27 +340,26 @@ response coming back. This might be what you need, but often you'll want to
345
340
  specify a few things:
346
341
 
347
342
  - `id`: The endpoint may have an id, which you may use to specify that you want
348
- an action to go to this particular id. However, most of the time you'll set
349
- up the `match` object so that Integreat will decide what endpoint to use for
350
- the action you dispatch.
343
+ an action to go to this particular id. However, most of the time you'll set up
344
+ the `match` object so that Integreat will decide what endpoint to use for the
345
+ action you dispatch.
351
346
  - `match`: The match object is used to decide the right endpoint for an action.
352
347
  More one this in the [Match properties](#match-properties) section.
353
348
  - `validate`: This is an array of condition that have to be met in order for
354
- Integreat to proceed with the endpoint. The `condition` is a mutation
355
- pipeline that should return a truthy value for the validation to pass. Any
356
- falsy value will cause the validation to fail. If `validate` is missing or
357
- an empty array, no validation will be done. This may sound similar to `match
358
- `, but `validate` is only processed after a match is found, and if the
359
- validation fails, no other endpoint is considered. On a failing validation,
360
- the `failResponse` is returned as the response from this action, or a
361
- `badrequest` response if no `failResponse` is provided. There's also a
362
- shorthand, where you set `failResponse` to a string, which will be the
363
- `error` message of the `badrequest` response. The response is passed through
364
- the mutation pipeline.
349
+ Integreat to proceed with the endpoint. The `condition` is a mutation pipeline
350
+ that should return a truthy value for the validation to pass. Any falsy value
351
+ will cause the validation to fail. If `validate` is missing or an empty array,
352
+ no validation will be done. This may sound similar to `match `, but `validate`
353
+ is only processed after a match is found, and if the validation fails, no
354
+ other endpoint is considered. On a failing validation, the `failResponse` is
355
+ returned as the response from this action, or a `badrequest` response if no
356
+ `failResponse` is provided. There's also a shorthand, where you set
357
+ `failResponse` to a string, which will be the `error` message of the
358
+ `badrequest` response. The response is passed through the mutation pipeline.
365
359
  - `mutate`: A mutation pipeline for the endpoint. The pipeline is run for both
366
360
  actions going to a service and the response coming back, so keep this in mind
367
- when you set up this pipeline. See [Mutation pipelines](#mutations)
368
- for more on how to define the mutation. `mutation` is an alias for `mutate`.
361
+ when you set up this pipeline. See [Mutation pipelines](#mutations) for more
362
+ on how to define the mutation. `mutation` is an alias for `mutate`.
369
363
  - `adapters`: An array of adapter ids that will be appended to the array of
370
364
  adapters set on the service.
371
365
  - `auth`: Auth config that will override the `auth` config on the service. See
@@ -383,10 +377,10 @@ specify a few things:
383
377
  data is not typed. The default is `false`, expcept for incoming endpoints
384
378
  (endpoints where `match` object has `incoming: true`) where the default value
385
379
  is `true`.
386
- - `castWithoutDefaults`: Set to `true` when you don't want to set default
387
- values on casted data. This also means no `id` will be generated and no
388
- `createdAt` or `updatedAt` will be set – when any of these are missing in
389
- the data. Default is `false`.
380
+ - `castWithoutDefaults`: Set to `true` when you don't want to set default values
381
+ on casted data. This also means no `id` will be generated and no `createdAt`
382
+ or `updatedAt` will be set – when any of these are missing in the data.
383
+ Default is `false`.
390
384
  - `options`: This object is merged with the `options` object on the service
391
385
  definition, and provide options for transporters and adapters. See
392
386
  [the `options` object](#options-object) for more on this.
@@ -396,8 +390,8 @@ specify a few things:
396
390
  An endpoint may specify none or more of the following match properties:
397
391
 
398
392
  - `type`: When set, the endpoint will only be used for actions with the
399
- specified schema type (the schema's id). `type` may also be an array of
400
- types, matching any one of the schemas in the list.
393
+ specified schema type (the schema's id). `type` may also be an array of types,
394
+ matching any one of the schemas in the list.
401
395
  - `scope`: May be `member`, `members`, `collection`, or `all`, to specify that
402
396
  the endpoint should be used to request one item (member) by id, several items
403
397
  by ids (members), or an entire collection of items. Setting this to `member`
@@ -409,32 +403,28 @@ An endpoint may specify none or more of the following match properties:
409
403
  match. `action` may also be a list of action types, matching any of these.
410
404
  - `params`: This object should list all params that this endpoint supports. A
411
405
  param in this context is any property on the action payload except `type`,
412
- `id`, or `data`. Use the param name as key on this object and set the value
413
- to `true` if it is required, and `false` if it is optional. When matching
414
- endpoints, an action will only match if it has all the required params, and
415
- in case several match, the endpoint with more specified params will be
416
- preferred.
417
- - `incoming`: If this is `true`, it will only match incoming actions, if
418
- `false` only outgoing, and if not set, it will match both.
419
- - `conditions`: An array of mutation pipelines that will be run on the action
420
- to see if it's a fit for this endpoint. If all pipelines return a truthy
421
- value, the endpoint is chosen (given that the other match properties also
422
- match). We rely on JavaScript definition of 'truthy' here, so any value that
423
- is not `false`, `null`, `undefined`, `0`, `NaN`, or `''` will be considered
424
- truthy.
425
-
426
- > [!NOTE]
427
- > There used to be a `filters` property on the endpoint match object. It is
428
- > still supported, but it's deprecated and will be removed in v1.1. Please use
429
- > `conditions` instead.
430
-
431
- > [!NOTE]
432
- > Editor's note: Describe what incoming actions are, and give more details on
433
- > filters.
434
-
435
- There might be cases where several endpoints match an action, and in these
436
- cases the endpoint with the highest level of specificity will be used. E.g.,
437
- for a `GET` action asking for resources of type `entry`, an endpoint with both
406
+ `id`, or `data`. Use the param name as key on this object and set the value to
407
+ `true` if it is required, and `false` if it is optional. When matching
408
+ endpoints, an action will only match if it has all the required params, and in
409
+ case several match, the endpoint with more specified params will be preferred.
410
+ - `incoming`: If this is `true`, it will only match incoming actions, if `false`
411
+ only outgoing, and if not set, it will match both.
412
+ - `conditions`: An array of mutation pipelines that will be run on the action to
413
+ see if it's a fit for this endpoint. If all pipelines return a truthy value,
414
+ the endpoint is chosen (given that the other match properties also match). We
415
+ rely on JavaScript definition of 'truthy' here, so any value that is not
416
+ `false`, `null`, `undefined`, `0`, `NaN`, or `''` will be considered truthy.
417
+
418
+ > [!NOTE] There used to be a `filters` property on the endpoint match object. It
419
+ > is still supported, but it's deprecated and will be removed in v1.1. Please
420
+ > use `conditions` instead.
421
+
422
+ > [!NOTE] Editor's note: Describe what incoming actions are, and give more
423
+ > details on filters.
424
+
425
+ There might be cases where several endpoints match an action, and in these cases
426
+ the endpoint with the highest level of specificity will be used. E.g., for a
427
+ `GET` action asking for resources of type `entry`, an endpoint with both
438
428
  `action: 'GET'` and `type: 'entry'` is picked over an endpoint matching all
439
429
  `GET` actions regardless of type. For `params` and `filters` this is decided by
440
430
  the highest number of properties on these objects.
@@ -505,8 +495,8 @@ Example of an `options` object set on the service definition:
505
495
  ```
506
496
 
507
497
  Any properties set directly on the `options` object or on a `transporter`
508
- property, are treated as options for the transporter. If there are properties
509
- on both the `options` and a `transporter` object, they will be merged, with the
498
+ property, are treated as options for the transporter. If there are properties on
499
+ both the `options` and a `transporter` object, they will be merged, with the
510
500
  `transporter` object having priority if conflicts. This is a shallow merge, so
511
501
  objects used in the options will not be merged.
512
502
 
@@ -525,9 +515,9 @@ for clarity, but both will work.
525
515
  Adapter options may be given in an `adapters` object, where each adapter may
526
516
  have its own options, set with the id of the adapter as a key. In the example
527
517
  above, the `xml` adapter will be given the `namespaces` object. A requirement
528
- for this, is that the adapter actually have an id. Adapters provided directly
529
- on service definition may not have an id, but all adapters that are referenced
530
- by an id, will also be given options set on that id, which is the common behavior.
518
+ for this, is that the adapter actually have an id. Adapters provided directly on
519
+ service definition may not have an id, but all adapters that are referenced by
520
+ an id, will also be given options set on that id, which is the common behavior.
531
521
 
532
522
  Finally, when all this sorting have been done on options from both the service
533
523
  definition and an endpoint, the two options structures are merged before being
@@ -608,12 +598,11 @@ Integreat supports getting and setting metadata for a service. The most common
608
598
  use of this is to keep track of when data of a certain type was last synced.
609
599
 
610
600
  Some services may have support for storing their own metadata, but usually you
611
- set up a dedicated service for storing other services' metadata. A few
612
- different pieces goes into setting up a meta store:
601
+ set up a dedicated service for storing other services' metadata. A few different
602
+ pieces goes into setting up a meta store:
613
603
 
614
604
  - A meta schema with the fields available as metadata
615
- - A service for storing metadata, with an endpoint suporting the metadata
616
- schema
605
+ - A service for storing metadata, with an endpoint suporting the metadata schema
617
606
  - Possible a metadata mutation for the metadata service
618
607
 
619
608
  When all of this is set up, you activate the metadata on the service the
@@ -657,13 +646,13 @@ Integreat has transporters for some common cases, and more may come:
657
646
  - [MQTT](https://github.com/integreat-io/integreat-transporter-mqtt)
658
647
  - [Redis](https://github.com/integreat-io/integreat-transporter-redis)
659
648
 
660
- You may write your own transporters if your case is not covered by any of
661
- these. Documentation on developing transporters are coming.
649
+ You may write your own transporters if your case is not covered by any of these.
650
+ Documentation on developing transporters are coming.
662
651
 
663
652
  Integreat will handle the transporters based on you configurations, but there
664
653
  are some specifics to each transporter, like HTTP needing an `uri` option or
665
- MongoDb needing a `collection` option. See the documentation of each
666
- transporter for more.
654
+ MongoDb needing a `collection` option. See the documentation of each transporter
655
+ for more.
667
656
 
668
657
  ## Adapters
669
658
 
@@ -703,10 +692,10 @@ with the service or a third-party service, like with OAuth2.
703
692
  When [setting up a service](#services), you may provide it with an auth id that
704
693
  refers to a [service authentication definition](#service-authentication), that
705
694
  again refers to an authenticator by id. The service auth definition also holds
706
- options for the authenticator, so when assigning an auth id to a service,
707
- you're assigning it an authenticator with those specific options. Another
708
- service may use the same authenticator, but with different options, and you
709
- would set this up with a different service authentication definition.
695
+ options for the authenticator, so when assigning an auth id to a service, you're
696
+ assigning it an authenticator with those specific options. Another service may
697
+ use the same authenticator, but with different options, and you would set this
698
+ up with a different service authentication definition.
710
699
 
711
700
  Authentication for outgoing actions are done when sending the action. When
712
701
  authenticated, an auth object is retrieved with the auth-as method specified on
@@ -714,11 +703,11 @@ the transporter (e.g. `asHttpHeaders` for the http transporter), or on the
714
703
  `overrideAuthAsMethod` in [auth options](#service-authentication) if set. The
715
704
  auth object is passed to the transporter on the action `meta.auth` prop. It is
716
705
  applied just before sending it, though, so it will be available to service
717
- middleware, but not to the mutation pipeline. This is done to expose
718
- credentials in as few places as possible. If you however _want_ to have the
719
- auth object in mutations, set `authInData` to `true` on the service or
720
- endpoint options, and authentication will be done in the `preflightAction`
721
- step instead, making it available on `meta.auth` throughout the entire mutation pipeline.
706
+ middleware, but not to the mutation pipeline. This is done to expose credentials
707
+ in as few places as possible. If you however _want_ to have the auth object in
708
+ mutations, set `authInData` to `true` on the service or endpoint options, and
709
+ authentication will be done in the `preflightAction` step instead, making it
710
+ available on `meta.auth` throughout the entire mutation pipeline.
722
711
 
723
712
  For incoming actions, authentication is done when a listening action calls the
724
713
  `authenticate()` callback. The `validate()` method on the authenticator is used
@@ -727,30 +716,29 @@ here, which will provide the transporter with an authorized ident.
727
716
  Available authenticators:
728
717
 
729
718
  - `action`: Will dispatch an action and use the response data to create an
730
- authentication. The `options` should have the action type as `action`, and
731
- the entire payload as `payload`. The response data should have an `auth`
732
- object, that will be used directly, and an optional `expire` that is a
733
- timestamp on which the auth will expire, in milleseconds since Epoc
734
- (1970-01-01). If no `expire` is returned, the `expireIn` option will be used
735
- if present. It is given as milliseconds before the auth should expire, or a
736
- `ms` string like `'1h'`.
719
+ authentication. The `options` should have the action type as `action`, and the
720
+ entire payload as `payload`. The response data should have an `auth` object,
721
+ that will be used directly, and an optional `expire` that is a timestamp on
722
+ which the auth will expire, in milleseconds since Epoc (1970-01-01). If no
723
+ `expire` is returned, the `expireIn` option will be used if present. It is
724
+ given as milliseconds before the auth should expire, or a `ms` string like
725
+ `'1h'`.
737
726
  - `http`: Supports http native authentications, like `Basic` and `Bearer`. It's
738
727
  included with the
739
728
  [HTTP transporter](https://github.com/integreat-io/integreat-transporter-http).
740
729
  - `ident`: Will always grant access and `validate()` will return an ident with
741
730
  the id provided in `identId` on the `options` object, or `'anonymous'` if no
742
731
  `identId` is provided. This is built into Integreat.
743
- - `options`: Will pass on the options as authentication, so whatever you
744
- provide here is the authentication. What options to provide, then, is
745
- depending on what the relevant transporter requires. For outgoing actions,
746
- the options are provided as is. Incoming action are validated agains the
747
- values given in the options (the keys may be dot notation paths in this
748
- case, and `identId` is excluded). An ident with the `identId` from the
749
- options as `id`, is returned if the action matches. This is built into
750
- Integreat.
751
- - `token`: A simple way of authenticating with a given token. For HTTP
752
- requests, the token will be provided as a `Authorization` header, and a
753
- configurable prefix like `Basic` or `Bearer`. This is built into Integreat.
732
+ - `options`: Will pass on the options as authentication, so whatever you provide
733
+ here is the authentication. What options to provide, then, is depending on
734
+ what the relevant transporter requires. For outgoing actions, the options are
735
+ provided as is. Incoming action are validated agains the values given in the
736
+ options (the keys may be dot notation paths in this case, and `identId` is
737
+ excluded). An ident with the `identId` from the options as `id`, is returned
738
+ if the action matches. This is built into Integreat.
739
+ - `token`: A simple way of authenticating with a given token. For HTTP requests,
740
+ the token will be provided as a `Authorization` header, and a configurable
741
+ prefix like `Basic` or `Bearer`. This is built into Integreat.
754
742
  - [`jwt`](https://github.com/integreat-io/integreat-authenticator-jwt): Will
755
743
  generate and encode a JavaScript Web Token (JWT) based on the options.
756
744
  - [`oauth2`](https://github.com/integreat-io/integreat-authenticator-oauth2):
@@ -759,23 +747,23 @@ Available authenticators:
759
747
 
760
748
  ## Mutations
761
749
 
762
- Both on the service and on endpoints, you define mutation pipelines. The
763
- service mutation is run before the endpoint mutation for data coming from a
764
- service, and in the oposite order when going to a service.
750
+ Both on the service and on endpoints, you define mutation pipelines. The service
751
+ mutation is run before the endpoint mutation for data coming from a service, and
752
+ in the oposite order when going to a service.
765
753
 
766
- A nice - but sometimes complicated - thing about mutations, is that they are
767
- run in both directions. They are by default defined for mutating data coming
768
- _from_ a service, and will be run in reverse for data going _to_ a service. In
769
- some cases this reversing of the pipeline will work as expected without
770
- modifications -- you define the mutation pipeline for data coming _from_ the
771
- service, and the reversed pipeline works _to_ as well. But many times you need
772
- to make adjustments and sometimes you'll have to have separate steps based on
773
- the direction. We'll get into more details in the following.
754
+ A nice - but sometimes complicated - thing about mutations, is that they are run
755
+ in both directions. They are by default defined for mutating data coming _from_
756
+ a service, and will be run in reverse for data going _to_ a service. In some
757
+ cases this reversing of the pipeline will work as expected without modifications
758
+ -- you define the mutation pipeline for data coming _from_ the service, and the
759
+ reversed pipeline works _to_ as well. But many times you need to make
760
+ adjustments and sometimes you'll have to have separate steps based on the
761
+ direction. We'll get into more details in the following.
774
762
 
775
- A mutation pipeline consists of one or more steps that the data will go
776
- through, before coming out on the other in the desired shape. It helps
777
- picturing this as an actual pipeline. After each step, data will be in a
778
- different shape, and this is the input to the next step.
763
+ A mutation pipeline consists of one or more steps that the data will go through,
764
+ before coming out on the other in the desired shape. It helps picturing this as
765
+ an actual pipeline. After each step, data will be in a different shape, and this
766
+ is the input to the next step.
779
767
 
780
768
  You define a pipeline in Integreat with an array, although for a pipeline with
781
769
  only one step, you may skip the array for simplicity.
@@ -784,18 +772,18 @@ Each step may be one of the following:
784
772
 
785
773
  - [**A dot notation path**](#dot-notation-paths), e.g. `path.to.data`. The data
786
774
  at that path will be extracted, and will be provided as the data to the next
787
- step in the pipeline. When going in reverse, the data will be set on that
788
- path instead.
775
+ step in the pipeline. When going in reverse, the data will be set on that path
776
+ instead.
789
777
  - **A mutation object** is an object that basically describes the object you
790
778
  want as a result, where the keys are dot notation paths and the values are
791
779
  mutation pipelines. Each pipeline on the mutation object will be run on the
792
780
  data, and then set on the path, resulting in an object that will be passed on
793
781
  to the next step. Setting `$iterate: true` on the object will cause it to
794
782
  iterate over items in an array, otherwise it will be applied to the array.
795
- Setting `$modify: true` will cause any properties on an object in the
796
- pipeline not set in the mutation, to be included, much like the spread in
797
- JavaScript. Setting `$modify` to a path works the same, but you will spread
798
- from the object at the path (`$modify: true` is equal to `$modify: '.'`).
783
+ Setting `$modify: true` will cause any properties on an object in the pipeline
784
+ not set in the mutation, to be included, much like the spread in JavaScript.
785
+ Setting `$modify` to a path works the same, but you will spread from the
786
+ object at the path (`$modify: true` is equal to `$modify: '.'`).
799
787
  - **A transform object** letting you run a transformer function on the data,
800
788
  e.g. `{ $transform: 'number' }` to transform the value into a number, or
801
789
  `undefined` if not possible.
@@ -854,8 +842,8 @@ return `[{ id: 'john' }]`. Arrays count as one level, so after
854
842
  A double carret `^^` takes you to the top -- the root -- so after
855
843
  `content.articles[0].id`, `^^.content.authors` returns `[{ id: 'john' }]`.
856
844
 
857
- Carret notations -- parents and roots -- does not currently work in reverse,
858
- but they might in a future version.
845
+ Carret notations -- parents and roots -- does not currently work in reverse, but
846
+ they might in a future version.
859
847
 
860
848
  ### Non-values
861
849
 
@@ -885,16 +873,16 @@ configure how data from A will be mutated to a schema, and then have data in
885
873
  that schema will be mutated and sent to B.
886
874
 
887
875
  This is a useful abstraction, and if you ever need to change one side, you can
888
- do so without involving the other side. If you need to switch out service B
889
- with service C, you can do so without involving the configuration of service
890
- A, or you can send data to both B and C, using the same setup for service A.
876
+ do so without involving the other side. If you need to switch out service B with
877
+ service C, you can do so without involving the configuration of service A, or
878
+ you can send data to both B and C, using the same setup for service A.
891
879
 
892
880
  To be clear, you can setup flows without schemas in Integreat, but then you may
893
881
  loose this flexibility and maintainability.
894
882
 
895
- A schema describe the data you expected to get out of Integreat, or send
896
- through it. You basically define the fields and their types, and may then cast
897
- data to that shape. Note that data on an action for a specified type, will be
883
+ A schema describe the data you expected to get out of Integreat, or send through
884
+ it. You basically define the fields and their types, and may then cast data to
885
+ that shape. Note that data on an action for a specified type, will be
898
886
  automatically cast to that type.
899
887
 
900
888
  ```javascript
@@ -924,13 +912,12 @@ automatically cast to that type.
924
912
  `plural: 'entries'`. This is not used by Integreat right now, but it may be
925
913
  used in the future for error messages, generating APIs from schemas, etc.
926
914
  - `service`: You may specify a default service for the schema when it makes
927
- sense. This allows you to dispatch an action for a type without specifying
928
- the target service, e.g. `{ type: 'GET', payload: { type: 'article' } }`,
929
- and have Integreat use the default service. This is a way of hiding
930
- configuration details from the code dispatching the actions, and you may
931
- also change the default service without changing the dispatching code if
932
- need be. You may always override this by specifying a `service` on the
933
- action payload.
915
+ sense. This allows you to dispatch an action for a type without specifying the
916
+ target service, e.g. `{ type: 'GET', payload: { type: 'article' } }`, and have
917
+ Integreat use the default service. This is a way of hiding configuration
918
+ details from the code dispatching the actions, and you may also change the
919
+ default service without changing the dispatching code if need be. You may
920
+ always override this by specifying a `service` on the action payload.
934
921
  - `shape`: This is where you define all the fields, see
935
922
  [the section below](#the-shape-of-a-schema).
936
923
  - `generateId`: Set this to `true` to generate a unique id for the `id` field
@@ -941,15 +928,15 @@ automatically cast to that type.
941
928
  of data cast to a schema will then be controlled by the rules you set here.
942
929
  See [Access rules](#access-rules) below for details on these rules. Note that
943
930
  `access` is optional, but when you get data from a service where any form of
944
- authentication is used to access the data, you will not be able to do
945
- anything with the data unless you cast it to a schema with `access` set up
946
- (or specifically says that you allow raw data from that endpoint).
931
+ authentication is used to access the data, you will not be able to do anything
932
+ with the data unless you cast it to a schema with `access` set up (or
933
+ specifically says that you allow raw data from that endpoint).
947
934
 
948
935
  ### The shape of a schema
949
936
 
950
- The shape is defined by an object where each key is the id of a field, which
951
- may contain only alphanumeric characters, and may not start with a digit. A
952
- schema cannot have the same id as a primitive type (see list below).
937
+ The shape is defined by an object where each key is the id of a field, which may
938
+ contain only alphanumeric characters, and may not start with a digit. A schema
939
+ cannot have the same id as a primitive type (see list below).
953
940
 
954
941
  The values on this object define the types of the fields and a few other
955
942
  optional features:
@@ -965,10 +952,9 @@ optional features:
965
952
  The `$type` prop sets the type of the field. The available primitive types, are
966
953
  `string`, `integer`, `float` (or `number`), `boolean`, and `date`. A field may
967
954
  also have another schema as its type, in which case the id of the schema is set
968
- in `$type`. An example can be an
969
- `article` schema with an `author` field of type `user`, referring to a schema
970
- with id `user`. When casting the `article`, data on the `author` prop will be
971
- cast with the `user` schema.
955
+ in `$type`. An example can be an `article` schema with an `author` field of type
956
+ `user`, referring to a schema with id `user`. When casting the `article`, data
957
+ on the `author` prop will be cast with the `user` schema.
972
958
 
973
959
  The `default` value will be used when the field is `undefined`, `null`, or not
974
960
  preset in data object being cast to this schema. If `default` is set to a
@@ -976,9 +962,9 @@ function, the function will be run with no argument, and the returned value is
976
962
  used as the default value. When no `default` is given, `undefined` is used.
977
963
 
978
964
  The `const` value override any value you provide to the field. It may be useful
979
- if you want a field to always have a fixed value. Just as for `default`, you
980
- may set it to a function, in which case the function will be run without
981
- arguments and the returned value will be used.
965
+ if you want a field to always have a fixed value. Just as for `default`, you may
966
+ set it to a function, in which case the function will be run without arguments
967
+ and the returned value will be used.
982
968
 
983
969
  If both `const` and `default` are set, `const` will be used.
984
970
 
@@ -1032,20 +1018,20 @@ When data is cast to a schema, the data will be in the following format:
1032
1018
  universally unique id will be generated for you.
1033
1019
  - `$type`: Set to the id of the schema by Integreat. This is a signal that the
1034
1020
  data has been cast.
1035
- - `createdAt`: This is not mandatory, but has special meaning. When a schema
1036
- has a `createdAt` field, but the date is not set in the data, it will be set
1037
- to the same as `updatedAt` (if provided) or to the current date/time.
1038
- - `updatedAt`: Just as `createdAt`, this is not mandatory. When a schema has
1039
- an `updatedAt` field, and the date is not set in the data, it will be set to
1040
- the same as `createdAt` (if provided) or the current date/time.
1021
+ - `createdAt`: This is not mandatory, but has special meaning. When a schema has
1022
+ a `createdAt` field, but the date is not set in the data, it will be set to
1023
+ the same as `updatedAt` (if provided) or to the current date/time.
1024
+ - `updatedAt`: Just as `createdAt`, this is not mandatory. When a schema has an
1025
+ `updatedAt` field, and the date is not set in the data, it will be set to the
1026
+ same as `createdAt` (if provided) or the current date/time.
1041
1027
  - `<key>`: Then follows the values of all the fields specified in the schema.
1042
1028
  Any value not provided in the data will be set to their default value, unless
1043
1029
  `castWithoutDefaults` is set to `true` in
1044
1030
  [the endpoint definition](#endpoints). When casting a value results in
1045
1031
  `undefined`, it will not be included on the returned object. Fields that has
1046
1032
  the `id` of other schemas as their type, will be objects. If only the id is
1047
- provided in the data, the `{ id: <string>, $ref: <schema id> }` format will
1048
- be used, with `$ref` being the id of the field type schema. When more data is
1033
+ provided in the data, the `{ id: <string>, $ref: <schema id> }` format will be
1034
+ used, with `$ref` being the id of the field type schema. When more data is
1049
1035
  provided, Integreat will cast it to the target schema and provide the entire
1050
1036
  data object, or array of objects, with the relevant `$type`.
1051
1037
 
@@ -1054,11 +1040,10 @@ When data is cast to a schema, the data will be in the following format:
1054
1040
  Set the `access` property on a schema to enforce permission checking. This
1055
1041
  applies to any service that provides data in this schema.
1056
1042
 
1057
- The simplest access rule is `auth`, which means that anyone can do anything
1058
- with the data of this schema, as long as they are authenticated. Being
1059
- authenticated, in this context, means that the dispatched action has an
1060
- `ident` in the `meta` object. See [the section on idents](#idents) for more on
1061
- this.
1043
+ The simplest access rule is `auth`, which means that anyone can do anything with
1044
+ the data of this schema, as long as they are authenticated. Being authenticated,
1045
+ in this context, means that the dispatched action has an `ident` in the `meta`
1046
+ object. See [the section on idents](#idents) for more on this.
1062
1047
 
1063
1048
  Example of a schema with an access rule:
1064
1049
 
@@ -1072,11 +1057,10 @@ Example of a schema with an access rule:
1072
1057
  }
1073
1058
  ```
1074
1059
 
1075
- To signal that the schema really has no need for authorization, use `all`.
1076
- This is not the same as not setting the `auth` prop, as `all` will override
1077
- Integreat's principle of not letting authorized data out of Integreat without
1078
- an access rule. `all` allows anybody to access the data, even the
1079
- unauthenticated.
1060
+ To signal that the schema really has no need for authorization, use `all`. This
1061
+ is not the same as not setting the `auth` prop, as `all` will override
1062
+ Integreat's principle of not letting authorized data out of Integreat without an
1063
+ access rule. `all` allows anybody to access the data, even the unauthenticated.
1080
1064
 
1081
1065
  On the other end of the spectrum, `none` will allow no one to access data cast
1082
1066
  to this schema, no matter who they are.
@@ -1093,16 +1077,16 @@ The following access rule props are available:
1093
1077
  above, where we provided this string instead of a access rule object.
1094
1078
  - `role`: Authorize only idents with this `role`. May also be an array.
1095
1079
  - `ident`: Authorize only idents with this precise `id`. May also be an array.
1096
- - `roleFromField`: Same as `role`, except the role is gotten from a field in
1097
- the schema. When authorizing data cast to this schema, the value of the role
1098
- field needs to be identical to (one of) the role(s) of the ident.
1080
+ - `roleFromField`: Same as `role`, except the role is gotten from a field in the
1081
+ schema. When authorizing data cast to this schema, the value of the role field
1082
+ needs to be identical to (one of) the role(s) of the ident.
1099
1083
  - `identFromField` - The same as `roleFromField`, but for an ident id.
1100
1084
 
1101
- In addition, you may override the general access rules of a schema with
1102
- specific rules for a type of action, by setting an `action` object with access
1103
- rules for action types. Here's an example of an access definition for allowing
1104
- all authorized idents to `GET` data in a certain shema, requiring the role
1105
- `admin` for `SET`s, and disallowing all other actions with the general rule
1085
+ In addition, you may override the general access rules of a schema with specific
1086
+ rules for a type of action, by setting an `action` object with access rules for
1087
+ action types. Here's an example of an access definition for allowing all
1088
+ authorized idents to `GET` data in a certain shema, requiring the role `admin`
1089
+ for `SET`s, and disallowing all other actions with the general rule
1106
1090
  `allow: 'none'`:
1107
1091
 
1108
1092
  ```javascript
@@ -1122,12 +1106,12 @@ all authorized idents to `GET` data in a certain shema, requiring the role
1122
1106
  ```
1123
1107
 
1124
1108
  Note that these action specific rules only applies to actions being sent to a
1125
- service. Some actions will never reach a service, but will instead trigger
1126
- other actions, and access will be granted or rejected only for the actions
1127
- that are about to be sent to a service. E.g. when you dispatch a `SYNC`
1128
- action, it starts off by dispatching one or more `GET` actions. The `SYNC`
1129
- action is not subjected to any access rules, but the `GET` actions are, and so
1130
- the `SYNC` will fail if one of the `GET` is rejected.
1109
+ service. Some actions will never reach a service, but will instead trigger other
1110
+ actions, and access will be granted or rejected only for the actions that are
1111
+ about to be sent to a service. E.g. when you dispatch a `SYNC` action, it starts
1112
+ off by dispatching one or more `GET` actions. The `SYNC` action is not subjected
1113
+ to any access rules, but the `GET` actions are, and so the `SYNC` will fail if
1114
+ one of the `GET` is rejected.
1131
1115
 
1132
1116
  Another example, intended for authorizing only the ident matching a user:
1133
1117
 
@@ -1142,8 +1126,8 @@ Another example, intended for authorizing only the ident matching a user:
1142
1126
  ```
1143
1127
 
1144
1128
  Here, only actions where the ident id is the same as the id of the user data,
1145
- will be allowed. This means that authenticated users (idents) may only
1146
- only access their own user data.
1129
+ will be allowed. This means that authenticated users (idents) may only only
1130
+ access their own user data.
1147
1131
 
1148
1132
  ## Actions
1149
1133
 
@@ -1163,13 +1147,14 @@ An action looks like this:
1163
1147
  }
1164
1148
  ```
1165
1149
 
1166
- - `type`: This is the id of one [of the action handlers](#available-action-handlers)
1167
- that comes with Integreat, e.g. `GET`. When you dispatch an action, it is
1168
- handed off to this handler (after some inital preperation). You may write
1169
- your own action handlers as well.
1170
- - `payload`: Holds parameters and [data](#typed-data) for this action. There
1171
- are some reserved [payload properties](#payload-properties), and the rest
1172
- will be made available to you in the mutation pipeline.
1150
+ - `type`: This is the id of one
1151
+ [of the action handlers](#available-action-handlers) that comes with
1152
+ Integreat, e.g. `GET`. When you dispatch an action, it is handed off to this
1153
+ handler (after some inital preperation). You may write your own action
1154
+ handlers as well.
1155
+ - `payload`: Holds parameters and [data](#typed-data) for this action. There are
1156
+ some reserved [payload properties](#payload-properties), and the rest will be
1157
+ made available to you in the mutation pipeline.
1173
1158
  - `meta`: Holds information about the action that does not belong in the
1174
1159
  payload, like the ident of the user dispatching, action id, etc. There are
1175
1160
  some reserved [meta properties](#meta-properties), but you may add your own
@@ -1178,42 +1163,41 @@ An action looks like this:
1178
1163
  When an action is dispatched, it returns a [response object](#action-response)
1179
1164
  with status, data, error message, etc.
1180
1165
 
1181
- Note that in a mutation pipeline, action handler, or middleware, the
1182
- response object is provided as a fourth property on the action. You will most
1183
- likely meet this at least when setting up mutations.
1166
+ Note that in a mutation pipeline, action handler, or middleware, the response
1167
+ object is provided as a fourth property on the action. You will most likely meet
1168
+ this at least when setting up mutations.
1184
1169
 
1185
1170
  Integreat will keep track of how many actions have been dispatched and are
1186
1171
  currently being process. The Instance object (`great` in
1187
1172
  [the example](#basic-example) at the beginning of this README) has a
1188
1173
  `dispatchedCount` property that gives you the number dispatched actions waiting
1189
- to be completed. Every time all dispatched are completed, a `done` event will
1190
- be emitted.
1174
+ to be completed. Every time all dispatched are completed, a `done` event will be
1175
+ emitted.
1191
1176
 
1192
1177
  ### Payload properties
1193
1178
 
1194
1179
  The payload is, together with the action `type`, a description to Integreat and
1195
1180
  the service of what to do. A design principle of Integreat has been to have as
1196
1181
  little specifics in these payload, so actions may be discpatched to service
1197
- without knowing how the service works. This is not always possible, at least
1198
- not yet, but it's a good principle to follow, also when you configure services
1199
- and plan what props need to be sent in the action payload.
1182
+ without knowing how the service works. This is not always possible, at least not
1183
+ yet, but it's a good principle to follow, also when you configure services and
1184
+ plan what props need to be sent in the action payload.
1200
1185
 
1201
1186
  You may set any properties on the payload, and they will be be available to you
1202
- in the service endpoint match and in the service mutations. Some properties
1203
- have special meanings, though, and you should avoid using them for anything
1204
- else:
1187
+ in the service endpoint match and in the service mutations. Some properties have
1188
+ special meanings, though, and you should avoid using them for anything else:
1205
1189
 
1206
1190
  - `type`: The type of the data the action sends and/or receives. This refers to
1207
- the `id` of a schema, and you will usually want to set this. Data provided
1208
- in the payload `data` and response `data` will be cast to this schema. If
1209
- you're dealing with several types in one action, you may set an array of
1210
- types, but will have to cast the data in the mutation yourself. Integreat
1211
- will validate that the data you send and receive is indeed of that type, and
1212
- will give you an auth error if not. (See
1191
+ the `id` of a schema, and you will usually want to set this. Data provided in
1192
+ the payload `data` and response `data` will be cast to this schema. If you're
1193
+ dealing with several types in one action, you may set an array of types, but
1194
+ will have to cast the data in the mutation yourself. Integreat will validate
1195
+ that the data you send and receive is indeed of that type, and will give you
1196
+ an auth error if not. (See
1213
1197
  [`allowRawRequest` and `allowRawResponse` on endpoints](#endpoints) for an
1214
1198
  exception.)
1215
- - `id`: You provide an id when you want to address a specific data item,
1216
- usually when you want to fetch one data item with an action like
1199
+ - `id`: You provide an id when you want to address a specific data item, usually
1200
+ when you want to fetch one data item with an action like
1217
1201
  `{ type: 'GET', payload: { type: 'article', id: '12345' } }`. You may also
1218
1202
  supply an array of ids to fetch several data items by id. When setting data,
1219
1203
  the id will instead be specified in the `data` when appropriate.
@@ -1226,11 +1210,11 @@ else:
1226
1210
  - `targetService`: An alias of `service`.
1227
1211
  - `sourceService`: When data comes from a different service and has not been
1228
1212
  mutated and cast yet, the `sourceService` property will tell Integreat to run
1229
- the data through the source service configuration before passing the action
1230
- on to an action handler. An example may be data coming in through an API,
1231
- where the API is configured as a service in Integreat. Note that this
1232
- property is usually set by transporters in their `listen()` methods, but you
1233
- may also set it directly on the action when it makes sense.
1213
+ the data through the source service configuration before passing the action on
1214
+ to an action handler. An example may be data coming in through an API, where
1215
+ the API is configured as a service in Integreat. Note that this property is
1216
+ usually set by transporters in their `listen()` methods, but you may also set
1217
+ it directly on the action when it makes sense.
1234
1218
  - `endpoint`: Set this to the `id` of a service endpoint when you want to
1235
1219
  override the endpoint match rules of Integreat. This should only be used when
1236
1220
  it is really necessary. Normally, you should instead design the match
@@ -1249,13 +1233,13 @@ page at a time, the following properties may be supported:
1249
1233
  of items specified in `pageSize`. If you ask for 500 items, the first action
1250
1234
  should have `pageOffset: 0` (or not specified), the next action
1251
1235
  `pageOffset: 500`, then `pageOffset: 1000`, and so on.
1252
- - `page`: The index of the page to fetch. Unlike most other indexes, this
1253
- starts with `1` being the first page. The effect is the same as
1254
- `pageOffset`, it's just a different way of specifying it. `page: 1` is the
1255
- same as `pageOffset: 0`, and `page: 2` is the same as `pageOffset: 500`,
1256
- given a `pageSize: 500`. Integreat will actually calculate both before
1257
- sending it to the transporter, as different types of services support
1258
- different types of pagination.
1236
+ - `page`: The index of the page to fetch. Unlike most other indexes, this starts
1237
+ with `1` being the first page. The effect is the same as `pageOffset`, it's
1238
+ just a different way of specifying it. `page: 1` is the same as
1239
+ `pageOffset: 0`, and `page: 2` is the same as `pageOffset: 500`, given a
1240
+ `pageSize: 500`. Integreat will actually calculate both before sending it to
1241
+ the transporter, as different types of services support different types of
1242
+ pagination.
1259
1243
  - `pageAfter`: As an alternative to specifying the number of items to skip, you
1260
1244
  may ask for the items after the item with the id you provide as `pageAfter`.
1261
1245
  If the last item of the first page is `'12345'`, you may set
@@ -1269,15 +1253,14 @@ page at a time, the following properties may be supported:
1269
1253
  that this id may hold internal logic from the transporter, but you should
1270
1254
  never rely on this logic and simply use it as an id.
1271
1255
 
1272
- > [!IMPORTANT]
1273
- > Pagination has to be supported by the service and your
1274
- > service configuration, and sometimes also the transporter. Integreat prepares
1275
- > and passes on these pagination properties, but if the service disregards
1276
- > them, there is little Integreat can do – except limiting the number of items
1256
+ > [!IMPORTANT] Pagination has to be supported by the service and your service
1257
+ > configuration, and sometimes also the transporter. Integreat prepares and
1258
+ > passes on these pagination properties, but if the service disregards them,
1259
+ > there is little Integreat can do except limiting the number of items
1277
1260
  > returned. It's up to you to figure out how to configure pagination for a
1278
1261
  > service, but youshould use these pagination properties to support it, to make
1279
- > this predictable. It also lets you use actions such as `GET_ALL`, that
1280
- > support pagination.
1262
+ > this predictable. It also lets you use actions such as `GET_ALL`, that support
1263
+ > pagination.
1281
1264
 
1282
1265
  Finally, there are some properties that has no special meaning to Integreat
1283
1266
  itself, but that may be set on incoming actions from transporters. These should
@@ -1302,16 +1285,16 @@ ideally be used in the same way or avoided:
1302
1285
 
1303
1286
  ### Meta properties
1304
1287
 
1305
- The action meta object is for information about an action that does not
1306
- directly define the action itself. The difference may be subtle in some cases,
1307
- but the general rule is a piece of information affects how the action is run,
1308
- it should be in the payload. E.g. the type of items to fetch is in the
1309
- payload, while the time the action was dispatched would go in the meta.
1288
+ The action meta object is for information about an action that does not directly
1289
+ define the action itself. The difference may be subtle in some cases, but the
1290
+ general rule is a piece of information affects how the action is run, it should
1291
+ be in the payload. E.g. the type of items to fetch is in the payload, while the
1292
+ time the action was dispatched would go in the meta.
1310
1293
 
1311
- This rule does not always hold, e.g. for information on the user dispatching
1312
- the action in `ident` on the meta object. Different idents may result in
1313
- different data being returned from the service, but still the action to
1314
- perform is the same, so it makes sense to have the ident in the meta object.
1294
+ This rule does not always hold, e.g. for information on the user dispatching the
1295
+ action in `ident` on the meta object. Different idents may result in different
1296
+ data being returned from the service, but still the action to perform is the
1297
+ same, so it makes sense to have the ident in the meta object.
1315
1298
 
1316
1299
  You may set your own meta properties, but in most cases you'll probably rather
1317
1300
  set payload properties.
@@ -1319,43 +1302,41 @@ set payload properties.
1319
1302
  Current meta properties reserved by Integreat:
1320
1303
 
1321
1304
  - `ident`: The ident to authorize the action with. May hold an `id`, `roles`,
1322
- `tokens`, and a few other options. See
1323
- [the section on idents](#idents).
1305
+ `tokens`, and a few other options. See [the section on idents](#idents).
1324
1306
  - `id`: The id of the action itself. You may set this yourself or let Integreat
1325
1307
  generate a universally unique id for you. Useful for logging and may be used
1326
1308
  by queues.
1327
1309
  - `cid`: Correlation id. When dispatching an action without `meta.cid`,
1328
1310
  Integreat will set it to the same as the `meta.id`. All actions that are then
1329
- dispatched as a consequence of that action (e.g. a `SYNC` or `GET_META`),
1330
- will have the same `cid`. The `cid` may then be used to group actions
1331
- belonging together, e.g. when displaying logs. The dispatching code set the
1332
- `cid` on an action, e.g. to correlate an action and the actions it
1333
- dispatches, with other operations outside Integreat.
1334
- - `gid`: Group id. This has some of the same purpose as `cid`, as it may be
1335
- used to group actions that belong together, but `gid` is not always set, and
1336
- will be used for smaller groups of actions than `cid`. Right now, `RUN`,
1337
- `SYNC`, and `GET_ALL` will use the `id` of the original action as `gid` for
1338
- all actions they dispatched.
1311
+ dispatched as a consequence of that action (e.g. a `SYNC` or `GET_META`), will
1312
+ have the same `cid`. The `cid` may then be used to group actions belonging
1313
+ together, e.g. when displaying logs. The dispatching code set the `cid` on an
1314
+ action, e.g. to correlate an action and the actions it dispatches, with other
1315
+ operations outside Integreat.
1316
+ - `gid`: Group id. This has some of the same purpose as `cid`, as it may be used
1317
+ to group actions that belong together, but `gid` is not always set, and will
1318
+ be used for smaller groups of actions than `cid`. Right now, `RUN`, `SYNC`,
1319
+ and `GET_ALL` will use the `id` of the original action as `gid` for all
1320
+ actions they dispatched.
1339
1321
  - `dispatchedAt`: Timestamp for when the action was dispatched (set by
1340
1322
  Integreat).
1341
- - `queue`: Signals to Integreat that an action may be queued. Set to `true`
1342
- when you want the action to be queued, but executed as soon as possible. Set
1343
- to a UNIX timestamp (number) to schedule for a later time. If no queue is
1344
- set up, the action will be dispatched right away. More on this under
1323
+ - `queue`: Signals to Integreat that an action may be queued. Set to `true` when
1324
+ you want the action to be queued, but executed as soon as possible. Set to a
1325
+ UNIX timestamp (number) to schedule for a later time. If no queue is set up,
1326
+ the action will be dispatched right away. More on this under
1345
1327
  [the section on queues](#queues).
1346
1328
  - `queuedAt`: Timestamp for when the action was pushed to the queue (set by
1347
1329
  Integreat).
1348
- - `options`: Used for passing the processed service endpoint options object to
1349
- a transporter. The `options` object is available through mutations, so that
1350
- you may modify it futher before it goes to the transporter. Note that only
1351
- the transporter options are provided here, not the adapter options.
1330
+ - `options`: Used for passing the processed service endpoint options object to a
1331
+ transporter. The `options` object is available through mutations, so that you
1332
+ may modify it futher before it goes to the transporter. Note that only the
1333
+ transporter options are provided here, not the adapter options.
1352
1334
  - `authorized`: An internal flag signaling that the action has been authorized.
1353
1335
  Will be removed from any dispatched actions.
1354
1336
 
1355
1337
  ### Action response
1356
1338
 
1357
- When you dispatch an action, you will get a response object back in this
1358
- format:
1339
+ When you dispatch an action, you will get a response object back in this format:
1359
1340
 
1360
1341
  ```javascript
1361
1342
  {
@@ -1389,17 +1370,16 @@ format:
1389
1370
  this property will hold a code for where the error originated. The goal is to
1390
1371
  set it as close to the actual origin as possible. See
1391
1372
  [list of origin codes](#origin-codes) below.
1392
- - `access`: An object holding the `ident` that was actually being used. This
1393
- may be different than the `meta.ident` on the action, as the ident may also
1394
- be mutated or completed with roles etc. along the way.
1373
+ - `access`: An object holding the `ident` that was actually being used. This may
1374
+ be different than the `meta.ident` on the action, as the ident may also be
1375
+ mutated or completed with roles etc. along the way.
1395
1376
  - `paging`: For services and transporters that support
1396
1377
  [pagination](#pagination), this object will hold information about how to get
1397
1378
  the next or previous page, in a `next` or `prev` object. These objects are
1398
- essentially the payloads you need to dispatch (with the same action `type`
1399
- and meta), to get the next or previous page. If there is no next or previous
1400
- page, the corresponding prop will not be set on the `paging` object. When
1401
- pagination is not relevant or used, the `paging` object may be missing
1402
- completely.
1379
+ essentially the payloads you need to dispatch (with the same action `type` and
1380
+ meta), to get the next or previous page. If there is no next or previous page,
1381
+ the corresponding prop will not be set on the `paging` object. When pagination
1382
+ is not relevant or used, the `paging` object may be missing completely.
1403
1383
  - `params`: Integreat never sets this, but you may set it in your mutations to
1404
1384
  provide parameters from a service that does not belong in the `data`.
1405
1385
  - `headers`: Integreat never sets this, but you may set it in your mutations to
@@ -1409,8 +1389,7 @@ format:
1409
1389
  `SYNC` or `RUN`. The action handlers _may_ then provide an array of all the
1410
1390
  sub-response objects here.
1411
1391
 
1412
- > [!NOTE]
1413
- > Editor's note: Is it correct that queues return the id in the data?
1392
+ > [!NOTE] Editor's note: Is it correct that queues return the id in the data?
1414
1393
 
1415
1394
  When the status is `queued`, the id of the queued action may found in
1416
1395
  `response.data.id`. This is the id assigned by the queue, and not necessarily
@@ -1418,17 +1397,20 @@ the same as `action.meta.id`.
1418
1397
 
1419
1398
  ### Status codes
1420
1399
 
1421
- The `status` property on the action response will be one of the following
1422
- status codes:
1400
+ The `status` property on the action response will be one of the following status
1401
+ codes:
1423
1402
 
1424
1403
  - `ok`: Everything is well, data is returned as expected
1425
1404
  - `queued`: The action has been queued. This is regarded as a success status
1405
+ - `pending`: May be used in cases where we need to signal that an action is
1406
+ still running or haven't started yet. This is mostly useful for loggers that
1407
+ needs to log progress.
1426
1408
  - `noaction`: The action did nothing, e.g. when a `SYNC` action has no data to
1427
1409
  sync
1428
1410
  - `notfound`: Tried to get or modify a resource that does not exist
1429
1411
  - `timeout`: The attempt to perform the action timed out
1430
- - `toomany`: Too many actions/requests were sent. This is normally used for
1431
- rate limiting errors from a service.
1412
+ - `toomany`: Too many actions/requests were sent. This is normally used for rate
1413
+ limiting errors from a service.
1432
1414
  - `autherror`: An authentication request failed
1433
1415
  - `noaccess`: Authentication is required or the provided auth is not enough
1434
1416
  - `badrequest`: Request data is not as expected
@@ -1437,8 +1419,8 @@ status codes:
1437
1419
 
1438
1420
  ### Origin codes
1439
1421
 
1440
- The `origin` property is not exclusively defined, but these are some of the
1441
- more common codes:
1422
+ The `origin` property is not exclusively defined, but these are some of the more
1423
+ common codes:
1442
1424
 
1443
1425
  - `service:<service id>`: The error originated in service. There may also be
1444
1426
  third level of detail here, if the service sets an origin code of its own.
@@ -1451,16 +1433,16 @@ more common codes:
1451
1433
  - `mutate:request`: The error was set in a request mutation pipeline.
1452
1434
  - `mutate:response`: The error was set in a response mutation pipeline.
1453
1435
  - `auth:action`: The error occured while attempting to authorize the action.
1454
- - `auth:data`: The error occured while attempting to authorize data in an
1455
- action payload or a response.
1436
+ - `auth:data`: The error occured while attempting to authorize data in an action
1437
+ payload or a response.
1456
1438
  - `auth:service:<service id>`: The error occured while attempting to authorize
1457
1439
  the service with the given id.
1458
1440
  - `auth:service:<service id>:<authenticator id>`: The error occured while
1459
1441
  attempting to authorize the service with the given id, through the given
1460
1442
  authenticator.
1461
- - `handler:<handler id>`: The error occurred with the handler with the given
1462
- id, e.g. `'handler:GET'`. This means the error did happen in the service or
1463
- the mutation pipelines, but in the internal workings of then handler.
1443
+ - `handler:<handler id>`: The error occurred with the handler with the given id,
1444
+ e.g. `'handler:GET'`. This means the error did happen in the service or the
1445
+ mutation pipelines, but in the internal workings of then handler.
1464
1446
  - `validate:service:<service id>:endpoint:<endpoint id>`: Validation of an
1465
1447
  action against an endpoint failed. Note that not all endpoints has an id, in
1466
1448
  which case that part of the origin code is left out.
@@ -1486,26 +1468,26 @@ Example ident:
1486
1468
  }
1487
1469
  ```
1488
1470
 
1489
- - `id`: A unique string identifying the ident. The actual value is irrelevant
1490
- to Integreat, as long as it is a string with A-Z, a-z, 0-9, \_, and -, and
1491
- it's unique within one Integreat configuration. This means that mapped
1492
- values from services may be used as ident ids, as long as they are unique
1493
- among these services.
1494
- - `tokens`: A list of values that may identify this ident in other services.
1495
- For example, an api that uses GitHub OAuth to identify its users, may provide
1496
- the `'github|23456'` token in the example above, which will be replaced with
1497
- this ident when it enters Integreat.
1471
+ - `id`: A unique string identifying the ident. The actual value is irrelevant to
1472
+ Integreat, as long as it is a string with A-Z, a-z, 0-9, \_, and -, and it's
1473
+ unique within one Integreat configuration. This means that mapped values from
1474
+ services may be used as ident ids, as long as they are unique among these
1475
+ services.
1476
+ - `tokens`: A list of values that may identify this ident in other services. For
1477
+ example, an api that uses GitHub OAuth to identify its users, may provide the
1478
+ `'github|23456'` token in the example above, which will be replaced with this
1479
+ ident when it enters Integreat.
1498
1480
  - `roles`: A list of roles or permissions given to this ident. The roles are
1499
1481
  custom defined per setup, and may be mapped to roles from other systems. When
1500
- setting the auth rules for a schema, you specify required rules so that to
1501
- get data cast in this schema, an ident with e.g. the role `admin` must be
1482
+ setting the auth rules for a schema, you specify required rules so that to get
1483
+ data cast in this schema, an ident with e.g. the role `admin` must be
1502
1484
  provided.
1503
1485
  - `type`: An optional string to signal when this ident is `'ROOT'`, `'SCHED'`,
1504
- `'SYST'`, or `'ANON'`. This is used internally by Integreat, but in some
1505
- cases you may want to set this yourself. Make sure, however, that you don't
1506
- let third-parties set `'ROOT'`. Make sure to also set the id, typically to
1507
- `'root'` or `'anonymous'`. Not setting any `type` is the same as setting it
1508
- to `'CUST'`, which is the default.
1486
+ `'SYST'`, or `'ANON'`. This is used internally by Integreat, but in some cases
1487
+ you may want to set this yourself. Make sure, however, that you don't let
1488
+ third-parties set `'ROOT'`. Make sure to also set the id, typically to
1489
+ `'root'` or `'anonymous'`. Not setting any `type` is the same as setting it to
1490
+ `'CUST'`, which is the default.
1509
1491
  - `isCompleted`: A flag to signal that an ident has already been completed, so
1510
1492
  that it won't be completed again. Used by the `completeIdent` middleware. You
1511
1493
  should normally not need to set this yourself.
@@ -1514,10 +1496,10 @@ The ident may also hold other custom properties, defined in the
1514
1496
  `identConfig.mapping` in the definitions passed when created a new Integreat
1515
1497
  instance. See description under [Idents](#idents).
1516
1498
 
1517
- Actions are authenticated by setting an ident on the `meta.ident` property.
1518
- It's up to the code dispatching an action to get hold of the properties of an
1519
- ident in a secure way. Once Integreat receives an ident through a dispatch, it
1520
- will assume this is accurate information and uphold its part of the security
1499
+ Actions are authenticated by setting an ident on the `meta.ident` property. It's
1500
+ up to the code dispatching an action to get hold of the properties of an ident
1501
+ in a secure way. Once Integreat receives an ident through a dispatch, it will
1502
+ assume this is accurate information and uphold its part of the security
1521
1503
  agreement and only return data and execute actions that the ident have
1522
1504
  permissions for.
1523
1505
 
@@ -1552,13 +1534,12 @@ is fetched, if it exists:
1552
1534
  }
1553
1535
  ```
1554
1536
 
1555
- See [the section on payload properties](#payload-properties) for more
1556
- properties that may be used with the `GET` action.
1537
+ See [the section on payload properties](#payload-properties) for more properties
1538
+ that may be used with the `GET` action.
1557
1539
 
1558
1540
  #### `GET_ALL`
1559
1541
 
1560
- Will run as many `GET` actions as needed to the get all available pages of
1561
- data.
1542
+ Will run as many `GET` actions as needed to the get all available pages of data.
1562
1543
 
1563
1544
  The action ...
1564
1545
 
@@ -1596,8 +1577,8 @@ response if one is not set.
1596
1577
 
1597
1578
  Send data to a service. The data to send is provided in the payload `data`
1598
1579
  property. Recomended practice is to provide the data as
1599
- [typed data](#typed-data), i.e. data objects cast to a schema, and let
1600
- mutations on the service endpoint modify it to the format the service expects.
1580
+ [typed data](#typed-data), i.e. data objects cast to a schema, and let mutations
1581
+ on the service endpoint modify it to the format the service expects.
1601
1582
 
1602
1583
  Any data coming back from the service, will be provided on `response.data` and
1603
1584
  may be mutated through service endpoint mutations, just as for [`GET`](#get)
@@ -1622,32 +1603,31 @@ Example `SET` action:
1622
1603
 
1623
1604
  Update data on a service. The idea is that while `SET` is used for setting data
1624
1605
  to a service – with no regard to what is actually set in the service already,
1625
- `UPDATE` is used for updating data, possibly not overwriting all properties.
1626
- If `UPDATE` provides data with only a few properties, the expectation is that
1627
- only these properties will be updated in the service. The `UPDATE` action is
1628
- also expected to fail when the item being updated does not exist, unlike `SET`,
1629
- that will usually create it.
1606
+ `UPDATE` is used for updating data, possibly not overwriting all properties. If
1607
+ `UPDATE` provides data with only a few properties, the expectation is that only
1608
+ these properties will be updated in the service. The `UPDATE` action is also
1609
+ expected to fail when the item being updated does not exist, unlike `SET`, that
1610
+ will usually create it.
1630
1611
 
1631
1612
  Note that the actual behavior is up to how you set up the service and what the
1632
1613
  service itself supports, but the `UPDATE` action will provide you with a way of
1633
1614
  doing this.
1634
1615
 
1635
- An `UPDATE` action may be handled in one of two ways, where the first is just
1636
- to run it against a service endpoint, much like a `SET` action (except it will
1616
+ An `UPDATE` action may be handled in one of two ways, where the first is just to
1617
+ run it against a service endpoint, much like a `SET` action (except it will
1637
1618
  match different endpoints). Data provided in the payload `data` is mutated and
1638
- sent to the service according to the endpoint configuration, and any data
1639
- coming back, will be provided on `response.data` and mutated.
1619
+ sent to the service according to the endpoint configuration, and any data coming
1620
+ back, will be provided on `response.data` and mutated.
1640
1621
 
1641
1622
  What makes `UPDATE` different from `SET`, though, is the second way we may
1642
1623
  handle `UPDATE` actions. Whenever there is no maching `UPDATE` endpoint,
1643
- Integreat will run the action as a `GET` and then a `SET`, to mimick and
1644
- update. The `GET` action will have the same `payload` and `meta` as the
1645
- original action. The same goes for the `SET` action, but the `payload.data`
1646
- will be the data returned from `GET` merged with the data on the original
1647
- `UPDATE` action. This will be a deep merge, prioritizing properties from the
1648
- `UPDATE` action, but any `createdAt` date in the data from `GET` will be kept.
1649
- If there's an `updatedAt` in the merged data, it will be overriden by the
1650
- current time.
1624
+ Integreat will run the action as a `GET` and then a `SET`, to mimick and update.
1625
+ The `GET` action will have the same `payload` and `meta` as the original action.
1626
+ The same goes for the `SET` action, but the `payload.data` will be the data
1627
+ returned from `GET` merged with the data on the original `UPDATE` action. This
1628
+ will be a deep merge, prioritizing properties from the `UPDATE` action, but any
1629
+ `createdAt` date in the data from `GET` will be kept. If there's an `updatedAt`
1630
+ in the merged data, it will be overriden by the current time.
1651
1631
 
1652
1632
  A requirement for this approach to work as expected, is that the data is casted
1653
1633
  to the same schema, but that should normally be the case when you use
@@ -1674,9 +1654,9 @@ Example `UPDATE` action:
1674
1654
  #### `DELETE` / `DEL`
1675
1655
 
1676
1656
  Delete one or more items from a service. Set the data for the items to delete,
1677
- in the payload `data` property as an array of [typed data](#typed-data).
1678
- In most cases, you only need to provide the `id` and the `$type`, but the way
1679
- you set up the service may require more properties.
1657
+ in the payload `data` property as an array of [typed data](#typed-data). In most
1658
+ cases, you only need to provide the `id` and the `$type`, but the way you set up
1659
+ the service may require more properties.
1680
1660
 
1681
1661
  Any data coming back from the service, will be provided on `response.data` and
1682
1662
  may be mutated through service endpoint mutations, just as for [`GET`](#get)
@@ -1714,8 +1694,7 @@ You may also `DELETE` one item like this:
1714
1694
  #### `GET_META`
1715
1695
 
1716
1696
  Get metadata for a service. See
1717
- [the section on metadata](#configuring-service-metadata) for how to set this
1718
- up.
1697
+ [the section on metadata](#configuring-service-metadata) for how to set this up.
1719
1698
 
1720
1699
  The `data` of the response from this aciton contains the `service` (the service
1721
1700
  id) and `meta` object with the metadata set as properties.
@@ -1803,8 +1782,8 @@ See [the section on jobs](#jobs) for more on how to configure jobs.
1803
1782
 
1804
1783
  The `SYNC` action will `GET` items from one service and `SET` them on another.
1805
1784
  There are different options for how to retrieve items, ranging from a crude
1806
- retrieval of all items on every sync, to a more fine grained approach where
1807
- only items that have been updated or created since last sync, will be synced.
1785
+ retrieval of all items on every sync, to a more fine grained approach where only
1786
+ items that have been updated or created since last sync, will be synced.
1808
1787
 
1809
1788
  The simplest action definition would look like this, where all items would be
1810
1789
  retrieved from the service and set on the target:
@@ -1826,31 +1805,30 @@ dispatch a `SET_META` action to update the `lastSyncedAt` date on the service.
1826
1805
  The `SET` actions to update the target service is added to the queue if one is
1827
1806
  configured.
1828
1807
 
1829
- To retrieve only new items, change the `retrieve` property to `updated`. In
1830
- this case, the action will dispatch `GET_META` to get the `lastSyncedAt` from
1831
- the `from` service, and get only newer items, by passing it the `updatedAfter`
1832
- param. The action will also filter out older items, in case the service does
1833
- not support `updatedAfter`.
1808
+ To retrieve only new items, change the `retrieve` property to `updated`. In this
1809
+ case, the action will dispatch `GET_META` to get the `lastSyncedAt` from the
1810
+ `from` service, and get only newer items, by passing it the `updatedAfter`
1811
+ param. The action will also filter out older items, in case the service does not
1812
+ support `updatedAfter`.
1834
1813
 
1835
1814
  By setting `retrieve` to `created`, you accomplish the same, but with
1836
1815
  `createdAfter`.
1837
1816
 
1838
- If you need to include more params in the actions to get from the `from`
1839
- service or set to the `to` service, you may provide a params object for the
1840
- `from` or `to` props, with the service id set as a `service` param. You may
1841
- also provide different action types than `GET` and `SET`, by setting the
1842
- `action` prop on the `from` or `to` objects respectively.
1817
+ If you need to include more params in the actions to get from the `from` service
1818
+ or set to the `to` service, you may provide a params object for the `from` or
1819
+ `to` props, with the service id set as a `service` param. You may also provide
1820
+ different action types than `GET` and `SET`, by setting the `action` prop on the
1821
+ `from` or `to` objects respectively.
1843
1822
 
1844
1823
  By default, `SYNC` will send every item gotten from the `from` service to the
1845
1824
  `to` service. You can split an array of items into several sets, by setting the
1846
- `maxPerSet` on the payload object, to a max number of items per set. If you
1847
- need to have one set per individual item, you may set `setMember` to `true` on
1848
- the payload object. (This is almost the same as setting `maxPerSet` to `1`,
1849
- except it won't be wrapped in an array.)
1825
+ `maxPerSet` on the payload object, to a max number of items per set. If you need
1826
+ to have one set per individual item, you may set `setMember` to `true` on the
1827
+ payload object. (This is almost the same as setting `maxPerSet` to `1`, except
1828
+ it won't be wrapped in an array.)
1850
1829
 
1851
- > [!NOTE]
1852
- > There are more options than these, and the documentation will be updated to
1853
- > include them later.
1830
+ > [!NOTE] There are more options than these, and the documentation will be
1831
+ > updated to include them later.
1854
1832
 
1855
1833
  #### `EXPIRE`
1856
1834
 
@@ -1889,22 +1867,21 @@ Here's an example of an `EXPIRE` action that will dispatch a `DELETE` directly:
1889
1867
  }
1890
1868
  ```
1891
1869
 
1892
- The `GET` action (or the `DELETE` action when `deleteWithParams` is `true`)
1893
- will have a `timestamp` property with the current time as microseconds since
1894
- epoc (Januar 1, 1970 UTC), and `isodate` as the current time in the extended
1895
- ISO 8601 format(`YYYY-MM-DDThh:mm:ss.sssZ`).
1870
+ The `GET` action (or the `DELETE` action when `deleteWithParams` is `true`) will
1871
+ have a `timestamp` property with the current time as microseconds since epoc
1872
+ (Januar 1, 1970 UTC), and `isodate` as the current time in the extended ISO 8601
1873
+ format(`YYYY-MM-DDThh:mm:ss.sssZ`).
1896
1874
 
1897
1875
  To have `timestamp` and `isodate` be a time in the future instead, set
1898
1876
  `msFromNow` to a positive number of milliseconds. This will be added to the
1899
- current time. To have a time in the past, use a negative number for
1900
- `msFromNow`.
1877
+ current time. To have a time in the past, use a negative number for `msFromNow`.
1901
1878
 
1902
1879
  #### `SERVICE`
1903
1880
 
1904
1881
  A `SERVICE` action will be sent directly to the specified service without any
1905
1882
  intervention by Integreat. This allows for running specialized actions on the
1906
- service that goes beyond what Integreat supports. It's up to each transporter
1907
- to support such actions, describe what they'll do, and define their payload
1883
+ service that goes beyond what Integreat supports. It's up to each transporter to
1884
+ support such actions, describe what they'll do, and define their payload
1908
1885
  properties.
1909
1886
 
1910
1887
  An example of an action that will tell a
@@ -1933,29 +1910,28 @@ Action handler signature:
1933
1910
  async function (action, { dispatch, getService, setProgress, options }) { ... }
1934
1911
  ```
1935
1912
 
1936
- - `action`: This is the dispatched action after it has been modified a bit
1937
- by the `dispatch()` method and possible after running an incoming mutation on
1938
- it. The modifications include cleaning up alias fields (e.g. `service` will
1939
- be set as `targetService`), removing sensitive or forbidden fields, and
1940
- setting a few default or internal fields (like the `dispatchedAt` meta).
1913
+ - `action`: This is the dispatched action after it has been modified a bit by
1914
+ the `dispatch()` method and possible after running an incoming mutation on it.
1915
+ The modifications include cleaning up alias fields (e.g. `service` will be set
1916
+ as `targetService`), removing sensitive or forbidden fields, and setting a few
1917
+ default or internal fields (like the `dispatchedAt` meta).
1941
1918
  - `dispatch`: From the handler, you may dispatch your own sub actions to the
1942
- provided `dispatch()` method. Note that this is an "internal dispatch
1943
- method", so it will return an action with the `response` object on it,
1944
- instead of just the `response` object. It's good practice to set the `cid`
1945
- meta prop for the actions you dispatch, to the `cid` meta prop on the
1946
- `action` you're handling. You should also use the same `ident` unless you
1947
- have very good reasons to do otherwise, to make sure you don't create
1948
- security holes.
1919
+ provided `dispatch()` method. Note that this is an "internal dispatch method",
1920
+ so it will return an action with the `response` object on it, instead of just
1921
+ the `response` object. It's good practice to set the `cid` meta prop for the
1922
+ actions you dispatch, to the `cid` meta prop on the `action` you're handling.
1923
+ You should also use the same `ident` unless you have very good reasons to do
1924
+ otherwise, to make sure you don't create security holes.
1949
1925
  - `getService`: This is a convenience method that will return the relevant
1950
- service object when you provide it with a type and optional a service id.
1951
- With a service id, you'll get the service with that id, with only the type,
1952
- you'll get the default service for that type. E.g.: `getService('article')`.
1926
+ service object when you provide it with a type and optional a service id. With
1927
+ a service id, you'll get the service with that id, with only the type, you'll
1928
+ get the default service for that type. E.g.: `getService('article')`.
1953
1929
  - `setProgress`: For long running tasks, you may want to set the progress along
1954
1930
  the way. Progress is specified as a number between `0` and `1`, e.g.
1955
- `setProgress(.5)` to signal that you're halfway through. When the your
1956
- handler is finished, the progress will automatically be set to `1`. This may
1957
- be used by queue implementations etc., to give progress feedback to users
1958
- and to know the action has not gone stale.
1931
+ `setProgress(.5)` to signal that you're halfway through. When the your handler
1932
+ is finished, the progress will automatically be set to `1`. This may be used
1933
+ by queue implementations etc., to give progress feedback to users and to know
1934
+ the action has not gone stale.
1959
1935
  - `options`: This is an object with a few settings: `queueService` is the id of
1960
1936
  the service set up as the default queue, and `identConfig` is the config
1961
1937
  object used for mapping ident schemas to ids, roles, and tokens (see
@@ -2059,11 +2035,11 @@ const flowJob = {
2059
2035
  ```
2060
2036
 
2061
2037
  Several things are going on here: First, we have a flow with two actions. We
2062
- imagine here that we are going to fetch an entry with an id that is found in
2063
- the `store` service, and use the `otherId` retrieved from that service to get
2064
- the entry from `otherService`. The two steps in `flow` look a lot like a job,
2065
- and in one way they are the same, with some differences, that we will get back
2066
- to. They are run sequentally in the order they appear in the `flow` array.
2038
+ imagine here that we are going to fetch an entry with an id that is found in the
2039
+ `store` service, and use the `otherId` retrieved from that service to get the
2040
+ entry from `otherService`. The two steps in `flow` look a lot like a job, and in
2041
+ one way they are the same, with some differences, that we will get back to. They
2042
+ are run sequentally in the order they appear in the `flow` array.
2067
2043
 
2068
2044
  Secondly, we have a `premutation` on each step. This is given the `action` and
2069
2045
  may mutate it before it is dispatched. As for endpoint mutations, the top level
@@ -2071,22 +2047,22 @@ has `$modify: true` as default, but we need to modify the sub-objects we
2071
2047
  include, when that is what we want. In the first step, we set the payload `id`
2072
2048
  to the `id` provided in the action that called this job. This job is passed to
2073
2049
  mutations on jobs and steps under the name `'action'`, and we prepend it with
2074
- the root path (`^^`) as it is found on the top level of the data structure
2075
- we're mutating.
2050
+ the root path (`^^`) as it is found on the top level of the data structure we're
2051
+ mutating.
2076
2052
 
2077
2053
  The second step is similar, but here we set the payload `id` to the `otherId`
2078
- found in the response data of the first step. The action and the response from
2079
- a step is available to all following steps by the id of the step, in this case
2054
+ found in the response data of the first step. The action and the response from a
2055
+ step is available to all following steps by the id of the step, in this case
2080
2056
  `'getFromStore'`. We have to prepend with the root path (`^^`) here as well.
2081
- When we say the action and response is available, we find it as an action
2082
- object with any response on a `response` property.
2057
+ When we say the action and response is available, we find it as an action object
2058
+ with any response on a `response` property.
2083
2059
 
2084
2060
  With the action
2085
2061
  `{ type: 'RUN', payload: { jobId: 'getEntryFromOtherService', id: '12345' } }`,
2086
- the first step will dispatch a `GET` for the id `12345`. If that action
2087
- succeeds with the data `{ id: '12345', otherId: '67890' }`, the second step
2088
- will dispatch a `GET` for the id `67890`. The response from the last action is
2089
- returned by default.
2062
+ the first step will dispatch a `GET` for the id `12345`. If that action succeeds
2063
+ with the data `{ id: '12345', otherId: '67890' }`, the second step will dispatch
2064
+ a `GET` for the id `67890`. The response from the last action is returned by
2065
+ default.
2090
2066
 
2091
2067
  If any job step fails, the entire job will fail and the error will be returned,
2092
2068
  unless you set up any `preconditions`, `postconditions`, or `postmutations` to
@@ -2095,9 +2071,9 @@ alter this default behavior. More on that in the following sections.
2095
2071
  ### Job and step mutations
2096
2072
 
2097
2073
  Jobs and job steps provide two mechanisms for mutations: `premutation` and
2098
- `postmutation`. They work in a similar way, but `premutation` is used to
2099
- mutate the action of a job or a job step _before_ it is dispatched, and
2100
- `postmutations` is used to mutate the response from a job or a step.
2074
+ `postmutation`. They work in a similar way, but `premutation` is used to mutate
2075
+ the action of a job or a job step _before_ it is dispatched, and `postmutations`
2076
+ is used to mutate the response from a job or a step.
2101
2077
 
2102
2078
  Note that `premutation` will not have an effect on a job with a `flow`, but
2103
2079
  `postmutation` may be used with the response from a flow.
@@ -2118,34 +2094,34 @@ used as the response from the job or step.
2118
2094
 
2119
2095
  ### Job and step conditions
2120
2096
 
2121
- By default, if a step in a flow fails, no more steps are run, and the entire
2122
- job will fail with the response from the failed step. You may, however,
2123
- provide your own conditions for when a step should be run and when a step
2124
- should be regarded as having failed.
2125
-
2126
- `preconditions` may be set to an array of condition objects, that must all
2127
- pass in order for the step to be run. Each condition object must have a
2128
- `condition` property with an array of mutation pipelines, and all these
2129
- pipelines must return truthy for the condition to be regarded as passed. Each
2130
- pipeline is given the same object that is given to `premutation` with action
2131
- and responses from previous steps, but without the action for this step. See
2097
+ By default, if a step in a flow fails, no more steps are run, and the entire job
2098
+ will fail with the response from the failed step. You may, however, provide your
2099
+ own conditions for when a step should be run and when a step should be regarded
2100
+ as having failed.
2101
+
2102
+ `preconditions` may be set to an array of condition objects, that must all pass
2103
+ in order for the step to be run. Each condition object must have a `condition`
2104
+ property with an array of mutation pipelines, and all these pipelines must
2105
+ return truthy for the condition to be regarded as passed. Each pipeline is given
2106
+ the same object that is given to `premutation` with action and responses from
2107
+ previous steps, but without the action for this step. See
2132
2108
  [the section on mutating jobs](#job-and-step-mutations) for more on this.
2133
2109
 
2134
2110
  The condition object may also have a `failResponse` property with a response
2135
2111
  object that will be used as the response from the step if the condition fails.
2136
2112
 
2137
2113
  Finally, the `condition` object may have a `break` property set to `true`, to
2138
- signal that the entire job should fail if the condition fails. If `break` is
2139
- not set, the step will just be skipped and the job flow continues.
2114
+ signal that the entire job should fail if the condition fails. If `break` is not
2115
+ set, the step will just be skipped and the job flow continues.
2140
2116
 
2141
- > [!NOTE]
2142
- > The default behavior of breaking when a step returns an error, is implemented
2143
- > as a default pre-condition, so if you set your own `preconditions`, you
2144
- > will also have to handle breaking on error.
2117
+ > [!NOTE] The default behavior of breaking when a step returns an error, is
2118
+ > implemented as a default pre-condition, so if you set your own
2119
+ > `preconditions`, you will also have to handle breaking on error.
2145
2120
  >
2146
2121
  > This is not very intuitive, so we have introduced a
2147
2122
  > `failOnErrorInPostconditions` flag that will change this to what will be the
2148
- > default in the next major version. [See more below.](#conditions-and-the-failOnErrorInPostconditions-flag)
2123
+ > default in the next major version.
2124
+ > [See more below.](#conditions-and-the-failOnErrorInPostconditions-flag)
2149
2125
 
2150
2126
  `postconditions` is also an array of condition objects, but this is used to
2151
2127
  decide if the step should be regarded as having failed after its action or flow
@@ -2158,19 +2134,19 @@ describe above.
2158
2134
 
2159
2135
  Post-conditions specify what is required for a step to be succeessful, and
2160
2136
  sometimes you consider a certain error to be an indication of success. An
2161
- example may be when you're checking a cache and will only continue if a value
2162
- is _not_ cached, meaning a `notfound` response status is a success in this
2163
- case. The condition pipeline for this should be straight forward, but as you
2164
- cannot specify the response that will be used when the condition _passes_, you
2165
- may wonder what happens with the error response. When an error response causes
2166
- a post-condition to pass, Integreat will set the status of the response to
2167
- `ok` and any `error` property will be changed to a `warning` property. Apart
2168
- from that, the response will be unchanged.
2137
+ example may be when you're checking a cache and will only continue if a value is
2138
+ _not_ cached, meaning a `notfound` response status is a success in this case.
2139
+ The condition pipeline for this should be straight forward, but as you cannot
2140
+ specify the response that will be used when the condition _passes_, you may
2141
+ wonder what happens with the error response. When an error response causes a
2142
+ post-condition to pass, Integreat will set the status of the response to `ok`
2143
+ and any `error` property will be changed to a `warning` property. Apart from
2144
+ that, the response will be unchanged.
2169
2145
 
2170
2146
  Jobs may have `preconditions` too, and it will work in the same way as for a
2171
- step, determining whether a job will run of not. `break` will have no effect
2172
- for a condition on a job. Jobs may not have `postconditions`, but you can
2173
- mutate the response with `postmutation` and accomplish the same.
2147
+ step, determining whether a job will run of not. `break` will have no effect for
2148
+ a condition on a job. Jobs may not have `postconditions`, but you can mutate the
2149
+ response with `postmutation` and accomplish the same.
2174
2150
 
2175
2151
  #### Conditions and the `failOnErrorInPostconditions` flag
2176
2152
 
@@ -2178,38 +2154,36 @@ The way `preconditions` controls whether the previous step will break on error,
2178
2154
  is not very intuitive, so we have introduced a feature flag to change this
2179
2155
  behavior. The flag is called `failOnErrorInPostconditions`, and is set on the
2180
2156
  `flags` object in the definitions given to `Integreat.create()`). In the next
2181
- major version of Integreat, this will be the default behavior, so it may
2182
- be a good idea to start using it now.
2157
+ major version of Integreat, this will be the default behavior, so it may be a
2158
+ good idea to start using it now.
2183
2159
 
2184
2160
  When the flag is set to `true`, the default break on error behavior is set in
2185
2161
  `postconditions` instead. This means that you may set your own `preconditions`
2186
2162
  and still have the previous step break on error, the way you probably would
2187
- expect it to work. If you set your own `postconditions`, on the other hand,
2188
- you need to take care of how an error response from that step is handled, but
2189
- that is often why you would set your own `postconditions` anyway.
2163
+ expect it to work. If you set your own `postconditions`, on the other hand, you
2164
+ need to take care of how an error response from that step is handled, but that
2165
+ is often why you would set your own `postconditions` anyway.
2190
2166
 
2191
- Also, `break` will be `true` by default for `postconditions` with this flag,
2192
- and you need to set it to `break: false` if you have a post-condition and you
2193
- don't want it to stop the entire flow.
2167
+ Also, `break` will be `true` by default for `postconditions` with this flag, and
2168
+ you need to set it to `break: false` if you have a post-condition and you don't
2169
+ want it to stop the entire flow.
2194
2170
 
2195
2171
  For `preconditions`, the `break` flag is `false` by default in all cases.
2196
2172
 
2197
2173
  ### Dispatching several actions by iterating over an array
2198
2174
 
2199
2175
  Sometimes you will want to dispatch several actions based on a data array, e.g.
2200
- when you have an array of data items, but the relevant endpoint only accepts
2201
- one data item. This may be done with `iterate`, which is a special mutation
2202
- that is must return an array, and the job action will be dispatched once for
2203
- every item in this array. The item will be set as payload `data`.
2204
- `premutation` may be used to modify the action before it is dispatched as
2205
- usual, but note that the mutation is applied to every single action, after the
2206
- `iterate`, so to speak.
2176
+ when you have an array of data items, but the relevant endpoint only accepts one
2177
+ data item. This may be done with `iterate`, which is a special mutation that is
2178
+ must return an array, and the job action will be dispatched once for every item
2179
+ in this array. The item will be set as payload `data`. `premutation` may be used
2180
+ to modify the action before it is dispatched as usual, but note that the
2181
+ mutation is applied to every single action, after the `iterate`, so to speak.
2207
2182
 
2208
2183
  This applies to both a job with an action and a step with an action in a flow.
2209
2184
 
2210
2185
  The responses of each action are combined and set as a `response` object on the
2211
- step action (before the iteration), making an iterated step just like any
2212
- other.
2186
+ step action (before the iteration), making an iterated step just like any other.
2213
2187
 
2214
2188
  When all actions are successful, the response will have status `ok`, and the
2215
2189
  response `data` will be an array of the data from each response in the order
@@ -2256,23 +2230,31 @@ const great = Integreat.create(
2256
2230
  ```
2257
2231
 
2258
2232
  To queue an action instead of dispatching it right away, you set `queue: true`
2259
- on the `meta` object. If everything is set up correctly, Integreat will push
2260
- the action to the queue. When the action is later pulled from the queue, it
2261
- will be dispatched again, but without the `queue` property.
2233
+ on the `meta` object. If everything is set up correctly, Integreat will push the
2234
+ action to the queue. When the action is later pulled from the queue, it will be
2235
+ dispatched again, but without the `queue` property.
2262
2236
 
2263
- You may also set the meta `queue` property to a Unix timestamp, and if the
2264
- queue transporter supports it, it will be run at this time instead of being
2265
- processed as soon as it is next in line in the queue.
2237
+ You may also set the meta `queue` property to a Unix timestamp, and if the queue
2238
+ transporter supports it, it will be run at this time instead of being processed
2239
+ as soon as it is next in line in the queue.
2266
2240
 
2267
2241
  When a queue is not set up, a dispatched action with `queue: true` will just be
2268
- run right away as a normal action.
2242
+ run right away as a normal action. In this case, the `queue` property is removed
2243
+ from the action before it is dispatched.
2244
+
2245
+ If you want to handle queuing yourself, outside of Integreat, you may set
2246
+ `disableQueuing: true` on the definition object (at the same level as
2247
+ `queueService`). Integreat will then never push any action to a queue, and –
2248
+ unlike when no queue is set up – it will leave the `queue` property untouched on
2249
+ the action. This lets your own middleware or service detect `queue` and route
2250
+ the action to your queue implementation. When `disableQueuing` is `true`, it
2251
+ takes precedence over `queueService`.
2269
2252
 
2270
2253
  You may also use queues directly, by dispatching to it as a server and getting
2271
2254
  incoming actions from its `listen()` method. In that case, it's just as any
2272
2255
  other service with no need for any special handling.
2273
2256
 
2274
- > [!NOTE]
2275
- > Queueing actions are actually done through an action handler, but this
2257
+ > [!NOTE] Queueing actions are actually done through an action handler, but this
2276
2258
  > handler is not available from outside Integreat.
2277
2259
 
2278
2260
  ## Middleware
@@ -2292,8 +2274,8 @@ Integreat supports middleware, and there are two different middleware
2292
2274
  _before_ it is mutated, giving you access to the data as it comes from the
2293
2275
  service.
2294
2276
 
2295
- To set up a logger of what we recieve from and send to a service, you'll use
2296
- the second middleware "pipeline", while a logger of dispatched actions would be
2277
+ To set up a logger of what we recieve from and send to a service, you'll use the
2278
+ second middleware "pipeline", while a logger of dispatched actions would be
2297
2279
  placed in the first.
2298
2280
 
2299
2281
  When actions pass through middleware, they may modifiy the actions as
@@ -2327,8 +2309,8 @@ information held somewhere else, e.g. in a database, you need to configure a
2327
2309
  user schema and set up a service to fetch this information.
2328
2310
 
2329
2311
  Integreat uses schemas and services to store idents. In the definition object
2330
- passed to `Integreat.create()`, you may provide an `identConfig` property with
2331
- a definition object looking something like this:
2312
+ passed to `Integreat.create()`, you may provide an `identConfig` property with a
2313
+ definition object looking something like this:
2332
2314
 
2333
2315
  ```javascript
2334
2316
  const great = Integreat.create(
@@ -2352,11 +2334,10 @@ const great = Integreat.create(
2352
2334
  - `type`: This is the id of the schema used for getting ident data. This schema
2353
2335
  needs to have a `service` specified.
2354
2336
  - `props`: You may provide alternative field names for the `id`, `roles`, and
2355
- `tokens` for an ident in the schema specified on `type`. When the prop and
2356
- the field has the same name, it may be omitted, though it doesn't hurt to
2357
- specify it anyway for clarity. For setups that don't need `roles` and/or
2358
- `tokens`, you may set these to `null`. Omitting them will result in the
2359
- default field names.
2337
+ `tokens` for an ident in the schema specified on `type`. When the prop and the
2338
+ field has the same name, it may be omitted, though it doesn't hurt to specify
2339
+ it anyway for clarity. For setups that don't need `roles` and/or `tokens`, you
2340
+ may set these to `null`. Omitting them will result in the default field names.
2360
2341
  - `mapping`: When `id`, `roles`, or `tokens` is found on other paths in the
2361
2342
  returned user data than the field names set in `props` (or by default), you
2362
2343
  may set additional mapping here, to specify at what dot notation path to find
@@ -2368,8 +2349,8 @@ const great = Integreat.create(
2368
2349
 
2369
2350
  Note that in the example above, the `id` of the data will be used as the ident
2370
2351
  `id`. When the id is not suited for this, you will need another field on the
2371
- schema that may act as the ident id. In cases where you need to transform the
2372
- id from the data in some way, this must be set up as a separate field and the
2352
+ schema that may act as the ident id. In cases where you need to transform the id
2353
+ from the data in some way, this must be set up as a separate field and the
2373
2354
  mutation will dictate how to transform it. In most cases, the `id` will do,
2374
2355
  though.
2375
2356
 
@@ -2387,8 +2368,8 @@ actions like this:
2387
2368
  }
2388
2369
  ```
2389
2370
 
2390
- In this case, `user` is the schema mapped to idents, and the `tokens`
2391
- property on the ident is mapped to the `tokens` field on the schema.
2371
+ In this case, `user` is the schema mapped to idents, and the `tokens` property
2372
+ on the ident is mapped to the `tokens` field on the schema.
2392
2373
 
2393
2374
  To make Integreat complete idents on actions with the persisted data, set it up
2394
2375
  with the `completeIdent` middleware:
@@ -2400,21 +2381,19 @@ const great = Integreat.create(defs, resources, [
2400
2381
  ```
2401
2382
 
2402
2383
  This middleware will intercept any action with `meta.ident` and replace it with
2403
- the ident item loaded from the designated schema. If the ident has an `id`,
2404
- the ident with this id is loaded, otherwise a `withToken` is used to load the
2405
- ident with the specified token. If no ident is found, the original ident is
2406
- kept.
2384
+ the ident item loaded from the designated schema. If the ident has an `id`, the
2385
+ ident with this id is loaded, otherwise a `withToken` is used to load the ident
2386
+ with the specified token. If no ident is found, the original ident is kept.
2407
2387
 
2408
- > [!NOTE]
2409
- > You may now alternatively set `identConfig.completeIdent` to `true` to have
2410
- > Integreat complete idents on dispatch and from the authentication callback
2411
- > used on incoming actions.
2388
+ > [!NOTE] You may now alternatively set `identConfig.completeIdent` to `true` to
2389
+ > have Integreat complete idents on dispatch and from the authentication
2390
+ > callback used on incoming actions.
2412
2391
 
2413
2392
  ### Writing middleware
2414
2393
 
2415
- You may write middleware to intercept dispatched actions. This may be useful
2416
- for logging, debugging, and situations where you need to make adjustments to
2417
- certain actions.
2394
+ You may write middleware to intercept dispatched actions. This may be useful for
2395
+ logging, debugging, and situations where you need to make adjustments to certain
2396
+ actions.
2418
2397
 
2419
2398
  A middleware is a function that accepts a `next()` function as only argument,
2420
2399
  and returns an async function that will be called with the action on dispatch.