aiiinotate 0.8.3 → 0.8.4

package/README.md

@@ -2,6 +2,14 @@
 
 aiiinotate is a fast and lightweight annotation server for IIIF. It relies on `nodejs/fastify` and `mongodb` and provides an API to read/write/update/delete IIIF annotations and index manifests.
 
+NOTE: currently, only annotations following the IIIF presentation API 2.0 and 2.1 are supported.
+
+---
+
+## API
+
+See the [docs on the aiiinotate API](https://github.com/Aikon-platform/aiiinotate/blob/dev/docs/endpoints.md).
+
 ---
 
 ## PROD USAGE
@@ -138,9 +146,6 @@ Remember to have your `mongodb` service running: `sudo systemctl start mongod` !
 ```bash
 # reload enabled
 npm run dev
-
-# reload disabled
-npm run prod
 ```
 
 - **Test the app**. NOTE: the tests will probably fail if you set the env variable `AIIINOTATE_STRICT_MODE` to `true`.
@@ -153,7 +158,7 @@ npm run test
 
 
 ```bash
-npm cli
+npm run cli
 ```
 
 - **Process migrations**. (see [the CLI docs](https://github.com/Aikon-platform/aiiinotate/blob/dev/docs/cli.md) for more info)

package/docs/db.md

@@ -4,15 +4,35 @@
 
 ## Collections
 
+- `annotations2`: a collection of annotations following the IIIF Presentation API 2.x (OpenAnnotation specification)
+- `annotations3`: a collection of annotations following the IIIF Presentation API 3.x (WebAnnotation specification)
+- `manifests2`: an index of manifests (manifest ID and array of canvas IDs of the manifest) following the IIIF Presentation API 2.x (OpenAnnotation specification)
+- `manifests3`: an index of manifests (manifest ID and array of canvas IDs of the manifest) following the IIIF Presentation API 3.x (WebAnnotation specification)
+
+Conversion between IIIF Presentation APIs 2 and 3 is not implemented.
+
 ---
 
 ## Validation rules
 
+Two levels of validation rules are implemented:
+- route-level validation
+- database-level validation
+
+At route-level, we ensure that the data has the absolutely necessary data to work with annotations.
+
+At database-level (see [`src/schema`](https://github.com/Aikon-platform/aiiinotate/blob/main/src/schemas/) module, we define a complete model (in JSONSchema) to describe annotations.
+
+**TLDR**: when inserting annotations (i.e., from an AnnotationList)
+1. at [route-level](https://github.com/Aikon-platform/aiiinotate/blob/main/src/data/annotations/routes.js), we ensure that the AnnotationList is generally valid (contains a `resource` with annotations, each annotations contains a `on`...)
+2. in the [`Annotations2`](https://github.com/Aikon-platform/aiiinotate/blob/main/src/data/annotations/annotations2.js) module, we do some extra checks and extra minimal reformatting to ensure that our annotations all have the same structure (i.e., convert `annotation.on` values to an array of `SpecificResources`). After finishing the cleanup, we are certain that an annotation can be inserted
+3. when inserting an annotation in the MongDB `annotations2`, MongoDB uses the `annotation` schema defined in [`src/schemas/schemasPresentation2`](https://github.com/Aikon-platform/aiiinotate/blob/main/src/schemas/schemasPresentation2.js) to validate annotations before inserting.
+
 ---
 
 ## Migrations
 
-Migrations are used to specify changes to the database's structure (creation and deletion of collections and indexes, changes to collection options such as validation rules etc.). Migration is done using [`migrate-mongo`](https://github.com/seppevs/migrate-mongo), a command-line interface for npm. 
+Migrations are used to specify changes to the database's structure (creation and deletion of collections and indexes, changes to collection options such as validation rules etc.). Migration is done using [`migrate-mongo`](https://github.com/seppevs/migrate-mongo), a command-line interface for npm.
 
 *Note that here, migrations only concern changes to the structure, not changes to the data.*
 

package/docs/endpoints.md

@@ -13,10 +13,9 @@ In the docs below,
 
 **aiiinotate** is meant to be able to handle both IIIF presentation APIs: the most common [2.x](https://iiif.io/api/presentation/2.1) and the more recent [3.x](https://iiif.io/api/presentation/3.0). Both APIs define a data structure for manifests, annotations, lists of annotations and collections of manifests.
 
-**HOWEVER, in aiiinotate, IIIF Presentation v2 and v3 data are isolated**: they form two separate collections, and no conversion is done between IIIF 2.x and 3.x data. This means that:
+**HOWEVER, in aiiinotate, v2 and v3 data are isolated**: they form two separate collections, and no conversion is done between IIIF 2.x and 3.x data. This means that:
 - **when communicating with aiiinotate**, you must specify a **IIIF presentation version in the query URL**. In the docs, this is described by the  `iiif_version` keyword.
 - **when inserting/updating data**, the data structure you provide must match the URL's `iiif_version`: you can't insert an annotation in v3 if your `iiif_version` is `2`.
-- **when searching for data**, if you inserted an annotation in v3, you must search for it with `iiif_version = 3`.
 - **TLDR**:
     - your data must match the `iiif_version` argument
     - if you insert an Annotation following the API v3.x, you can't search for it using `iiif_version=2`.
@@ -289,6 +288,7 @@ Batch insert multiple annotations.
 
 #### Notes
 
+- Calling this route using parallel processes (i.e., `Promise.all`) can cause data races which will cause inserts to fail
 - Be wary of maximum body size, especially when sending AnnotationLists in your body. If possible, using `{ uri: string }` is better.
 - All annotations within a single AnnotationList/Page may have different target canvases or manifests.
 - See **Create/update an annotation**.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "aiiinotate",
-  "version": "0.8.3",
+  "version": "0.8.4",
   "description": "a fast IIIF-compliant annotation server",
   "main": "./cli/index.js",
   "type": "module",
@@ -18,7 +18,8 @@
     "lint": "npx eslint --fix",
     "migrate": "npm run cli -- migrate",
     "update_version": "python3 scripts/update_version.py",
-    "get_version": "python3 scripts/get_version.py"
+    "get_version": "python3 scripts/get_version.py",
+    "ttt": "echo 'HELLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLO'"
   },
   "pre-commit": [
     "lint"

package/docs/progress.md

@@ -1,38 +0,0 @@
-# Dev progress
-
-We mostly talk about which routes are done here
-
----
-
-## Done
-
-### Routes
-
-Routes are only implemented with IIIF Presentation API 2.x, not with the 3.0 version.
-
-#### Generic routes
-
-- `GET /search-api/:iiifSearchVersion/manifests/:manifestShortId/search`: search API
-
-#### Annotations routes
-
-- `GET /annotations/:iiifPresentationVersion/search`: get all annotations for a canvas URI
-- `POST /annotations/:iiifPresentationVersion/create`: create 1 annotation
-- `POST /annotations/:iiifPresentationVersion/createMany`: create several annotations
-- `POST /annotations/:iiifPresentationVersion/update`: update 1 annotation
-- `DELETE /annotations/:iiifPresentationVersion/delete`: delete annotations, either by their `@id`, trget canvas URI (`on.full`), or their `on.manifestShortId`
-
-=> all create/update/delete annotation routes are done !
-
-#### Manifests routes
-
-- `POST /manifests/:iiifPresentationVersion/create`: create a single manifest, either by including the manifest in the body or its URI
-- `DELETE /manifests/:iiifPresentationVersion/delete`: delete a single manifest
-- `GET /manifests/:iiifPresentationVersion`: return an index of all manifests as a collection
-
-### Non-routes
-
-- `manifests2`: `insert`, `insertMany` internal behaviours
-- fetching and inserting manifests related to an annotation when using inserting annotations.
-
----

package/docs/todo.md

@@ -1,19 +0,0 @@
-# TODO
-
-## Short term
-
-- **pagination**
-    - add pagination to responses (`next` field in AnnotationLists)
-    - add `pageNumber` and `limit` parameters to routes to be able to specify page numbers and number of items in page from GET queries
-- **update Mongo indexes** 
-    - to reflect database changes (`annotation.on` is now an array)
-    - on frequently queried fields (`annotation.on.canvasIdx`, `annotation.on.manifestShortId`... => see what else is used in GET routes)
-- **sort annotations collections**
-    - by `canvasIdx` 
-    - by position on the page (y-offset then x-offset, see `xywh`)
-
-
-## Mid term
-
-- **aiiinotate CLI** to import and export data easily
-- **IIIF presentation 3**