arocapi 5.0.0-beta.1 → 5.0.0-beta.10

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (234) hide show
  1. package/LICENSE.md +1 -1
  2. package/README.md +37 -802
  3. package/dist/app.d.ts +38 -20
  4. package/dist/app.js +62 -40
  5. package/dist/app.js.map +1 -1
  6. package/dist/routes/capabilities.d.ts +7 -0
  7. package/dist/routes/capabilities.js +13 -0
  8. package/dist/routes/capabilities.js.map +1 -0
  9. package/dist/routes/entities.d.ts +5 -2
  10. package/dist/routes/entities.js +37 -57
  11. package/dist/routes/entities.js.map +1 -1
  12. package/dist/routes/entity.d.ts +7 -2
  13. package/dist/routes/entity.js +13 -29
  14. package/dist/routes/entity.js.map +1 -1
  15. package/dist/routes/file.d.ts +8 -2
  16. package/dist/routes/file.js +37 -72
  17. package/dist/routes/file.js.map +1 -1
  18. package/dist/routes/files.d.ts +5 -2
  19. package/dist/routes/files.js +24 -39
  20. package/dist/routes/files.js.map +1 -1
  21. package/dist/routes/metadata.d.ts +11 -5
  22. package/dist/routes/metadata.js +24 -68
  23. package/dist/routes/metadata.js.map +1 -1
  24. package/dist/routes/search.d.ts +8 -7
  25. package/dist/routes/search.js +90 -119
  26. package/dist/routes/search.js.map +1 -1
  27. package/dist/search/advanced.d.ts +8 -0
  28. package/dist/search/advanced.js +198 -0
  29. package/dist/search/advanced.js.map +1 -0
  30. package/dist/search/backend.d.ts +95 -0
  31. package/dist/search/backend.js +5 -0
  32. package/dist/search/backend.js.map +1 -0
  33. package/dist/search/fields.d.ts +63 -0
  34. package/dist/search/fields.js +203 -0
  35. package/dist/search/fields.js.map +1 -0
  36. package/dist/search/opensearch/advanced.d.ts +4 -0
  37. package/dist/search/opensearch/advanced.js +28 -0
  38. package/dist/search/opensearch/advanced.js.map +1 -0
  39. package/dist/search/opensearch/opensearch.d.ts +16 -0
  40. package/dist/search/opensearch/opensearch.js +187 -0
  41. package/dist/search/opensearch/opensearch.js.map +1 -0
  42. package/dist/search/opensearch/visibility.d.ts +3 -0
  43. package/dist/search/opensearch/visibility.js +27 -0
  44. package/dist/search/opensearch/visibility.js.map +1 -0
  45. package/dist/search/postgres/advanced.d.ts +7 -0
  46. package/dist/search/postgres/advanced.js +54 -0
  47. package/dist/search/postgres/advanced.js.map +1 -0
  48. package/dist/search/postgres/facets.d.ts +2 -0
  49. package/dist/search/postgres/facets.js +27 -0
  50. package/dist/search/postgres/facets.js.map +1 -0
  51. package/dist/search/postgres/geo.d.ts +6 -0
  52. package/dist/search/postgres/geo.js +42 -0
  53. package/dist/search/postgres/geo.js.map +1 -0
  54. package/dist/search/postgres/postgres.d.ts +19 -0
  55. package/dist/search/postgres/postgres.js +157 -0
  56. package/dist/search/postgres/postgres.js.map +1 -0
  57. package/dist/search/postgres/values.d.ts +8 -0
  58. package/dist/search/postgres/values.js +20 -0
  59. package/dist/search/postgres/values.js.map +1 -0
  60. package/dist/search/postgres/visibility.d.ts +4 -0
  61. package/dist/search/postgres/visibility.js +38 -0
  62. package/dist/search/postgres/visibility.js.map +1 -0
  63. package/dist/search/requestFilters.d.ts +39 -0
  64. package/dist/search/requestFilters.js +125 -0
  65. package/dist/search/requestFilters.js.map +1 -0
  66. package/dist/search/visibility.d.ts +51 -0
  67. package/dist/search/visibility.js +100 -0
  68. package/dist/search/visibility.js.map +1 -0
  69. package/dist/storage/backend.d.ts +58 -0
  70. package/dist/storage/backend.js +2 -0
  71. package/dist/storage/backend.js.map +1 -0
  72. package/dist/storage/deliver.d.ts +12 -0
  73. package/dist/storage/deliver.js +34 -0
  74. package/dist/storage/deliver.js.map +1 -0
  75. package/dist/storage/directory/directory.d.ts +6 -0
  76. package/dist/storage/directory/directory.js +24 -0
  77. package/dist/storage/directory/directory.js.map +1 -0
  78. package/dist/storage/entity-meta/entity-meta.d.ts +5 -0
  79. package/dist/storage/entity-meta/entity-meta.js +14 -0
  80. package/dist/storage/entity-meta/entity-meta.js.map +1 -0
  81. package/dist/storage/location.d.ts +19 -0
  82. package/dist/storage/location.js +36 -0
  83. package/dist/storage/location.js.map +1 -0
  84. package/dist/storage/ocfl/ocfl.d.ts +6 -0
  85. package/dist/storage/ocfl/ocfl.js +45 -0
  86. package/dist/storage/ocfl/ocfl.js.map +1 -0
  87. package/dist/storage/results.d.ts +7 -0
  88. package/dist/storage/results.js +22 -0
  89. package/dist/storage/results.js.map +1 -0
  90. package/dist/storage/s3/s3.d.ts +10 -0
  91. package/dist/storage/s3/s3.js +47 -0
  92. package/dist/storage/s3/s3.js.map +1 -0
  93. package/dist/transformers/default.d.ts +11 -22
  94. package/dist/transformers/default.js +41 -15
  95. package/dist/transformers/default.js.map +1 -1
  96. package/dist/types/access.d.ts +9 -0
  97. package/dist/types/access.js +2 -0
  98. package/dist/types/access.js.map +1 -0
  99. package/dist/types/context.d.ts +6 -0
  100. package/dist/types/context.js +2 -0
  101. package/dist/types/context.js.map +1 -0
  102. package/dist/types/transformers.d.ts +4 -10
  103. package/dist/types/visibility.d.ts +6 -0
  104. package/dist/types/visibility.js +2 -0
  105. package/dist/types/visibility.js.map +1 -0
  106. package/dist/utils/access.d.ts +17 -0
  107. package/dist/utils/access.js +41 -0
  108. package/dist/utils/access.js.map +1 -0
  109. package/dist/utils/errors.d.ts +19 -10
  110. package/dist/utils/errors.js +44 -8
  111. package/dist/utils/errors.js.map +1 -1
  112. package/dist/utils/headers.d.ts +1 -1
  113. package/dist/utils/headers.js +10 -1
  114. package/dist/utils/headers.js.map +1 -1
  115. package/dist/utils/validation.d.ts +2 -0
  116. package/dist/utils/validation.js +8 -0
  117. package/dist/utils/validation.js.map +1 -0
  118. package/dist/utils/visibility.d.ts +18 -0
  119. package/dist/utils/visibility.js +16 -0
  120. package/dist/utils/visibility.js.map +1 -0
  121. package/package.json +82 -40
  122. package/prisma/models/entity.prisma +9 -8
  123. package/prisma/models/entitySearch.prisma +21 -0
  124. package/prisma/models/file.prisma +3 -3
  125. package/src/app.ts +168 -95
  126. package/src/routes/capabilities.ts +22 -0
  127. package/src/routes/entities.ts +48 -67
  128. package/src/routes/entity.ts +21 -35
  129. package/src/routes/file.ts +53 -103
  130. package/src/routes/files.ts +31 -47
  131. package/src/routes/metadata.ts +42 -98
  132. package/src/routes/search.ts +131 -151
  133. package/src/search/advanced.ts +258 -0
  134. package/src/search/backend.ts +91 -0
  135. package/src/search/fields.ts +320 -0
  136. package/src/search/opensearch/advanced.ts +36 -0
  137. package/src/search/opensearch/opensearch.ts +290 -0
  138. package/src/search/opensearch/visibility.ts +38 -0
  139. package/src/search/postgres/advanced.ts +76 -0
  140. package/src/search/postgres/facets.ts +34 -0
  141. package/src/search/postgres/geo.ts +58 -0
  142. package/src/search/postgres/postgres.ts +232 -0
  143. package/src/search/postgres/values.ts +33 -0
  144. package/src/search/postgres/visibility.ts +54 -0
  145. package/src/search/requestFilters.ts +178 -0
  146. package/src/search/visibility.ts +193 -0
  147. package/src/storage/backend.ts +75 -0
  148. package/src/storage/deliver.ts +65 -0
  149. package/src/storage/directory/directory.ts +39 -0
  150. package/src/storage/entity-meta/entity-meta.ts +25 -0
  151. package/src/storage/location.ts +78 -0
  152. package/src/storage/ocfl/ocfl.ts +71 -0
  153. package/src/storage/results.ts +32 -0
  154. package/src/storage/s3/s3.ts +79 -0
  155. package/src/transformers/default.ts +115 -85
  156. package/src/types/access.ts +25 -0
  157. package/src/types/context.ts +15 -0
  158. package/src/types/transformers.ts +8 -34
  159. package/src/types/visibility.ts +21 -0
  160. package/src/utils/access.ts +80 -0
  161. package/src/utils/errors.ts +68 -11
  162. package/src/utils/headers.ts +22 -2
  163. package/src/utils/validation.ts +11 -0
  164. package/src/utils/visibility.ts +56 -0
  165. package/dist/app.test.d.ts +0 -1
  166. package/dist/app.test.js +0 -82
  167. package/dist/app.test.js.map +0 -1
  168. package/dist/index.dev.d.ts +0 -1
  169. package/dist/index.dev.js +0 -156
  170. package/dist/index.dev.js.map +0 -1
  171. package/dist/routes/crate.test.d.ts +0 -1
  172. package/dist/routes/crate.test.js +0 -342
  173. package/dist/routes/crate.test.js.map +0 -1
  174. package/dist/routes/entities.test.d.ts +0 -1
  175. package/dist/routes/entities.test.js +0 -273
  176. package/dist/routes/entities.test.js.map +0 -1
  177. package/dist/routes/entity.test.d.ts +0 -1
  178. package/dist/routes/entity.test.js +0 -130
  179. package/dist/routes/entity.test.js.map +0 -1
  180. package/dist/routes/file.test.d.ts +0 -1
  181. package/dist/routes/file.test.js +0 -315
  182. package/dist/routes/file.test.js.map +0 -1
  183. package/dist/routes/files.test.d.ts +0 -1
  184. package/dist/routes/files.test.js +0 -222
  185. package/dist/routes/files.test.js.map +0 -1
  186. package/dist/routes/search.test.d.ts +0 -1
  187. package/dist/routes/search.test.js +0 -780
  188. package/dist/routes/search.test.js.map +0 -1
  189. package/dist/test/helpers/fastify.d.ts +0 -9
  190. package/dist/test/helpers/fastify.js +0 -18
  191. package/dist/test/helpers/fastify.js.map +0 -1
  192. package/dist/test/integration.setup.d.ts +0 -29
  193. package/dist/test/integration.setup.js +0 -187
  194. package/dist/test/integration.setup.js.map +0 -1
  195. package/dist/test/integration.test.d.ts +0 -1
  196. package/dist/test/integration.test.js +0 -396
  197. package/dist/test/integration.test.js.map +0 -1
  198. package/dist/transformers/default.test.d.ts +0 -1
  199. package/dist/transformers/default.test.js +0 -163
  200. package/dist/transformers/default.test.js.map +0 -1
  201. package/dist/transformers/transformer.test.d.ts +0 -1
  202. package/dist/transformers/transformer.test.js +0 -187
  203. package/dist/transformers/transformer.test.js.map +0 -1
  204. package/dist/types/fileHandlers.d.ts +0 -41
  205. package/dist/types/fileHandlers.js +0 -2
  206. package/dist/types/fileHandlers.js.map +0 -1
  207. package/dist/utils/errors.test.d.ts +0 -1
  208. package/dist/utils/errors.test.js +0 -66
  209. package/dist/utils/errors.test.js.map +0 -1
  210. package/dist/utils/queryBuilder.d.ts +0 -35
  211. package/dist/utils/queryBuilder.js +0 -98
  212. package/dist/utils/queryBuilder.js.map +0 -1
  213. package/prisma.config.ts +0 -9
  214. package/src/app.test.ts +0 -112
  215. package/src/index.dev.ts +0 -200
  216. package/src/reset.d.ts +0 -1
  217. package/src/routes/__snapshots__/entities.test.ts.snap +0 -96
  218. package/src/routes/__snapshots__/entity.test.ts.snap +0 -74
  219. package/src/routes/__snapshots__/search.test.ts.snap +0 -165
  220. package/src/routes/crate.test.ts +0 -423
  221. package/src/routes/entities.test.ts +0 -320
  222. package/src/routes/entity.test.ts +0 -160
  223. package/src/routes/file.test.ts +0 -395
  224. package/src/routes/files.test.ts +0 -278
  225. package/src/routes/search.test.ts +0 -903
  226. package/src/test/__snapshots__/integration.test.ts.snap +0 -18
  227. package/src/test/helpers/fastify.ts +0 -27
  228. package/src/test/integration.setup.ts +0 -209
  229. package/src/test/integration.test.ts +0 -482
  230. package/src/transformers/default.test.ts +0 -184
  231. package/src/transformers/transformer.test.ts +0 -252
  232. package/src/types/fileHandlers.ts +0 -65
  233. package/src/utils/errors.test.ts +0 -75
  234. package/src/utils/queryBuilder.ts +0 -132
package/README.md CHANGED
@@ -1,818 +1,53 @@
1
- # An RO-Crate API for Data Collections : Arocapi
1
+ # arocapi
2
2
 
3
- This project a prototype implementation of an RO-Crate based collections
4
- archive and data portal based on the Portland Common Data model of collections
5
- suitable use in long-term archival repository systems that aim to follow the
6
- Protocols for Long Term Archival Repository Services
7
- [PILARS](https://w3id.org/ldac/pilars).
3
+ arocapi implements the [RO-Crate API](https://ro-crate-api.crate-works.org) for data collections, packaged as a
4
+ Fastify plugin. You mount it into your own Fastify app and bring your storage, search index and access
5
+ rules; arocapi serves the routes for entities, files, search and RO-Crate metadata, following the
6
+ [PILARS](https://w3id.org/ldac/pilars) protocols for long-term archival repositories.
8
7
 
9
- The library is a Fastify plugin that mounts the API endpoints into your own
10
- Fastify app, or into an Express app via Fastify (see
11
- [Using with Express](#using-with-express)).
8
+ **Documentation: <https://arocapi.crate-works.org>**
12
9
 
13
- The aim is to provide an API-centred reference implementation of a repository
14
- with an access-controlled web portal for data dissemination and deposit:
10
+ - [What is arocapi](https://arocapi.crate-works.org/introduction/what-is-arocapi/)
11
+ - [Getting started](https://arocapi.crate-works.org/getting-started/install/)
12
+ - [Configuration](https://arocapi.crate-works.org/configuration/data-sources/)
13
+ - [Routes](https://arocapi.crate-works.org/reference/routes/)
15
14
 
16
- - Backend-agnostic RO-Crate storage using pluggable storage-layers
17
- - [OCFL] - the Oxford Common File Layout specification on disk or cloud-based
18
- object storage
19
- - Object Storage as used in [PARADISEC]
20
- - A simple RO-Crate file layout (TODO - likely a super-simple protocol for
21
- directory hierarchy with the presence on RO-Crate used to indicate that
22
- everything under that directory is a single RO-Crate)
23
- - Authorisation based on the concept of Access (and Deposit, TODO) Licenses
24
- using an arms-length process with an external authority such as a REMS
25
- instance or other license/group management system
26
- - Based on the principle that all data MUST have a implementation-neutral
27
- natural-language description of its access conditions stored with it
28
- - Licenses may be based on 'traditional' access control lists maintained in
29
- software such as PARADISEC or a license manager such as REMS, (TODO: an
30
- example implementation using GitHub groups to manage license access, a
31
- simple ACL manager?)).
15
+ ## Install
32
16
 
33
- ## Prerequisites
34
-
35
- - Node.js (version 22 or higher)
36
- - pnpm package manager
37
- - Database: MySQL or PostgreSQL
38
- - OpenSearch for search functionality
39
- - Docker and Docker Compose (for development)
40
-
41
- ## Quick Start
42
-
43
- ### 1. Installation
44
-
45
- ```bash
46
- # Install the package
17
+ ```sh
47
18
  pnpm add arocapi
48
-
49
- # Install required peer dependencies
50
- pnpm add @prisma/client @opensearch-project/opensearch
51
- pnpm add -D prisma
52
- ```
53
-
54
- Changes merged to `main` are published as prereleases under the `beta` tag: `pnpm add arocapi@beta`.
55
-
56
- ### 2. Environment Setup
57
-
58
- Create a `.env` file in your project root:
59
-
60
- ```env
61
- # Database Configuration (You can use an DB that Prisma supports)
62
- DATABASE_URL="mysql://root:password@localhost:3306/catalog"
63
-
64
- # OpenSearch Configuration
65
- OPENSEARCH_URL="http://localhost:9200"
66
- ```
67
-
68
- ## Creating Your API
69
-
70
- Create your Fastify application with Typescript support:
71
-
72
- ```typescript
73
- // src/index.ts
74
- import { Client } from '@opensearch-project/opensearch';
75
- import arocapi, { AllPublicAccessTransformer, AllPublicFileAccessTransformer } from 'arocapi';
76
- import Fastify from 'fastify';
77
- import { PrismaClient } from './generated/prisma/client.js';
78
- import { Readable } from 'stream';
79
-
80
- const prisma = new PrismaClient();
81
-
82
- if (!process.env.OPENSEARCH_URL) {
83
- throw new Error('OPENSEARCH_URL environment variable is not set');
84
- }
85
- const opensearchUrl = process.env.OPENSEARCH_URL;
86
- const opensearch = new Client({ node: opensearchUrl });
87
-
88
- const fastify = Fastify({
89
- logger: true,
90
- });
91
-
92
- // For fully public datasets
93
- fastify.register(arocapi, {
94
- prisma,
95
- opensearch,
96
- accessTransformer: AllPublicAccessTransformer,
97
- fileAccessTransformer: AllPublicFileAccessTransformer,
98
- // Required: File handler for serving File entity content
99
- fileHandler: {
100
- get: async (file) => {
101
- const fileUrl = `https://storage.example.com/${file.meta.storagePath}`;
102
- return { type: 'redirect', url: fileUrl };
103
- },
104
- head: async (file) => ({
105
- contentType: file.mediaType,
106
- contentLength: file.size,
107
- }),
108
- },
109
- // Required: RO-Crate handler for serving RO-Crate metadata
110
- roCrateHandler: {
111
- get: async (entity) => {
112
- const jsonString = JSON.stringify(entity.rocrate, null, 2);
113
- return {
114
- type: 'stream',
115
- stream: Readable.from([jsonString]),
116
- metadata: {
117
- contentType: 'application/ld+json',
118
- contentLength: Buffer.byteLength(jsonString),
119
- },
120
- };
121
- },
122
- head: async (entity) => ({
123
- contentType: 'application/ld+json',
124
- contentLength: Buffer.byteLength(JSON.stringify(entity.rocrate)),
125
- }),
126
- },
127
- });
128
-
129
- try {
130
- await fastify.listen({ port: 3000 });
131
- } catch (err) {
132
- fastify.log.error(err);
133
- process.exit(1);
134
- }
135
- ```
136
-
137
- ### CORS
138
-
139
- arocapi doesn't set CORS headers; CORS is up to your application. To let
140
- browsers on other origins call the API, register
141
- [`@fastify/cors`](https://github.com/fastify/fastify-cors) on the root
142
- instance:
143
-
144
- ```bash
145
- pnpm add @fastify/cors
146
- ```
147
-
148
- ```typescript
149
- import cors from '@fastify/cors';
150
-
151
- fastify.register(cors);
152
- fastify.register(arocapi, { /* ... */ });
153
- ```
154
-
155
- The defaults allow all origins with `GET`, `HEAD` and `POST`, matching what
156
- arocapi registered before 5.0.0. See the `@fastify/cors` docs to restrict them.
157
-
158
- ## Database Setup with Prisma
159
-
160
- ### 1. Initialize Prisma
161
-
162
- ```bash
163
- # Initialize prisma with your preferred database provider
164
- npx prisma init --datasource-provider mysql
165
- # or for PostgreSQL: npx prisma init --datasource-provider postgresql
166
- ```
167
-
168
- ### 2. Configure Prisma Multi-Model Setup
169
-
170
- Modify the `prisma.config.ts` file in your project root to support folder based schemas:
171
-
172
- ```typescript
173
- // prisma.config.ts
174
- import 'dotenv/config';
175
- import { defineConfig } from 'prisma/config';
176
-
177
- export default defineConfig({
178
- // Adds support for folder based schemas
179
- schema: 'prisma',
180
- });
181
- ```
182
-
183
- ### 3. Set Up Model Structure
184
-
185
- Create the directory structure and link to arocapi models:
186
-
187
- ```bash
188
- # Create prisma models directory
189
- mkdir -p prisma/models
190
-
191
- # Link to the upstream arocapi models
192
- ln -s ../../node_modules/arocapi/prisma/models prisma/arocapi
193
- ```
194
-
195
- ### 4. Generate Prisma Client and Run Migrations
196
-
197
- ```bash
198
- # Create database migrations for this version of arocapi
199
- npx prisma migrate dev --name="arocapi-$(pnpm list | grep arocapi | awk '{ print $2 }')"
200
-
201
- # Generate the Prisma client
202
- npx prisma generate
203
-
204
- ```
205
-
206
- ## Available API Routes
207
-
208
- The arocapi provides the following endpoints:
209
-
210
- - `GET /entities` - List all entities with pagination and filtering
211
- - `GET /entity/:id` - Get a specific entity by ID
212
- - `GET /entity/:id/rocrate` - Download RO-Crate metadata
213
- - `GET /files` - List files with pagination and filtering
214
- - `GET /file/:id` - Download or access file content
215
- - `POST /search` - Search entities using OpenSearch
216
-
217
- ## Customising OpenSearch query
218
-
219
- The OpenSearch query builder used in the `/search` endpoint can be customised to match any
220
- structure of the actual OpenSearch data. The default OpensearchQueryBuilder class is used to
221
- build all the queries for the search and an optional config can be passed on
222
- to customise the class behaviour.
223
-
224
- ### Query Builder (Optional)
225
- Use the `queryBuilderClass` optional parameter to specify a custom builder class.
226
- If not specified, the default `OpensearchQueryBuilder` is used.
227
- The custom class should extend the `OpensearchQueryBuilder` class and
228
- may override any or all of the methods as required:
229
-
230
- - `buildQuery`: For generating the base search query.
231
- - `buildAggregations`: For generating the facets or aggregations related query.
232
- - `buildSort`: For generating the ordering/sorting part of the query.
233
-
234
- ### Query Builder Options (Optional)
235
- The `queryBuilderOptions` optional parameter is passed as a constructor argument
236
- when instantiating the queryBuilderClass. The default `OpensearchQueryBuilder` can be
237
- configured with:
238
-
239
- - `aggregations`: This field controls the aggregation and will be passed verbatim as the `aggs` field in the generated search query.
240
-
241
- ## Customising Entity Responses
242
-
243
- The API provides a flexible transformer system for customising entity responses
244
- through two types of transformers:
245
-
246
- ### Access Transformer (Required)
247
-
248
- The `accessTransformer` parameter is **required** for security. You must explicitly
249
- choose how access control is handled for your repository.
250
-
251
- **For fully public datasets**, use `AllPublicAccessTransformer`:
252
-
253
- ```typescript
254
- import arocapi, { AllPublicAccessTransformer } from 'arocapi';
255
-
256
- await server.register(arocapi, {
257
- prisma,
258
- opensearch,
259
- accessTransformer: AllPublicAccessTransformer,
260
- });
261
- ```
262
-
263
- **For restricted content**, implement a custom access transformer:
264
-
265
- ```typescript
266
- const accessTransformer = async (entity, { request }) => {
267
- // Custom logic to determine access
268
- const user = await authenticateUser(request);
269
- const canAccessContent = await checkLicense(entity.contentLicenseId, user);
270
-
271
- return {
272
- ...entity,
273
- access: {
274
- metadata: true, // Metadata always visible
275
- content: canAccessContent,
276
- contentAuthorisationUrl: canAccessContent
277
- ? undefined
278
- : 'https://rems.example.com/request-access',
279
- },
280
- };
281
- };
282
- ```
283
-
284
- > [!WARNING]
285
- > The `accessTransformer` is required to prevent accidental exposure of
286
- > restricted content. You must make an explicit choice about access control for
287
- > your repository.
288
-
289
- ### Entity Transformers
290
-
291
- Optional transformations for enriching or modifying response data. Multiple
292
- transformers can be chained together.
293
-
294
- ```typescript
295
- await server.register(arocapi, {
296
- prisma,
297
- opensearch,
298
- accessTransformer: AllPublicAccessTransformer,
299
- entityTransformers: [
300
- // Add computed fields
301
- (entity) => ({
302
- ...entity,
303
- displayName: `${entity.name} [${entity.entityType.split('/').pop()}]`,
304
- }),
305
- // Add counts
306
- async (entity) => {
307
- const objectCount = entity.memberOf
308
- ? await prisma.entity.count({
309
- where: { memberOf: entity.id },
310
- })
311
- : 0;
312
-
313
- return {
314
- ...entity,
315
- counts: {
316
- objects: objectCount,
317
- }
318
- };
319
- },
320
- ],
321
- });
322
- ```
323
-
324
- ### Transformation Pipeline
325
-
326
- Every entity response flows through this three-stage pipeline:
327
-
328
- 1. **Base transformer** - Converts database entities to standard format
329
- 2. **Access transformer** - Adds access control information
330
- 3. **Entity transformers** - Optional additional transformations
331
-
332
- ### Transformer Context
333
-
334
- Access and entity transformers receive a `TransformerContext` as their second argument with the shape `{ request, fastify }`. The `request` is the Fastify request for per-call data (headers, auth, query). The `fastify` instance is provided as an extension point — arocapi does not decorate it with anything itself, but consumers who decorate `fastify` with their own helpers (e.g. via `fastify.decorate('userService', ...)`) can read those decorations through `context.fastify` from inside their transformers. For database access, prefer closing over your own client reference as shown in the examples below.
335
-
336
- ### Common Use Cases
337
-
338
- **Access control for restricted content:**
339
-
340
- ```typescript
341
- accessTransformer: async (entity, { request }) => {
342
- const hasAccess = await checkUserPermissions(request, entity.contentLicenseId);
343
- return {
344
- ...entity,
345
- access: {
346
- metadata: true,
347
- content: hasAccess,
348
- },
349
- };
350
- }
351
- ```
352
-
353
- **Adding computed or derived fields:**
354
-
355
- ```typescript
356
- entityTransformers: [
357
- (entity) => ({
358
- ...entity,
359
- shortId: entity.id.split('/').pop(),
360
- year: extractYear(entity.description),
361
- }),
362
- ]
363
- ```
364
-
365
- **Fetching related data asynchronously:**
366
-
367
- ```typescript
368
- entityTransformers: [
369
- async (entity) => ({
370
- ...entity,
371
- stats: await fetchEntityStats(entity.id, prisma),
372
- }),
373
- ]
374
- ```
375
-
376
- ## Customising File Responses
377
-
378
- The API provides a separate transformer system for file responses, similar to entity transformers but specifically designed for files.
379
-
380
- ### File Access Transformer (Required)
381
-
382
- The `fileAccessTransformer` parameter is **required** for security. You must explicitly choose how access control is handled for file content.
383
-
384
- **Key Difference from Entity Transformers**: File metadata (filename, size, mediaType, etc.) is always accessible. Only content access is controlled via the `access.content` field.
385
-
386
- **For fully public datasets**, use `AllPublicFileAccessTransformer`:
387
-
388
- ```typescript
389
- import arocapi, { AllPublicAccessTransformer, AllPublicFileAccessTransformer } from 'arocapi';
390
-
391
- await server.register(arocapi, {
392
- prisma,
393
- opensearch,
394
- accessTransformer: AllPublicAccessTransformer,
395
- fileAccessTransformer: AllPublicFileAccessTransformer,
396
- });
397
- ```
398
-
399
- **For restricted content**, implement a custom file access transformer:
400
-
401
- ```typescript
402
- const fileAccessTransformer = async (file, { request }) => {
403
- // Custom logic to determine file content access
404
- const user = await authenticateUser(request);
405
- const canAccessContent = await checkLicense(file.contentLicenseId, user);
406
-
407
- return {
408
- ...file,
409
- access: {
410
- content: canAccessContent,
411
- contentAuthorizationUrl: canAccessContent
412
- ? undefined
413
- : 'https://rems.example.com/request-access',
414
- },
415
- };
416
- };
417
-
418
- await server.register(arocapi, {
419
- prisma,
420
- opensearch,
421
- accessTransformer: AllPublicAccessTransformer,
422
- fileAccessTransformer,
423
- });
424
- ```
425
-
426
- > [!WARNING]
427
- > The `fileAccessTransformer` is required to prevent accidental exposure of
428
- > restricted file content. You must make an explicit choice about access control
429
- > for your files.
430
-
431
- ### File Transformers
432
-
433
- Optional transformations for enriching or modifying file response data. Multiple
434
- file transformers can be chained together.
435
-
436
- ```typescript
437
- await server.register(arocapi, {
438
- prisma,
439
- opensearch,
440
- accessTransformer: AllPublicAccessTransformer,
441
- fileAccessTransformer: AllPublicFileAccessTransformer,
442
- fileTransformers: [
443
- // Add computed fields
444
- (file) => ({
445
- ...file,
446
- displayFilename: file.filename.toUpperCase(),
447
- sizeInKB: Math.round(file.size / 1024),
448
- extension: file.filename.split('.').pop(),
449
- }),
450
- // Fetch parent entity information
451
- async (file) => {
452
- const parent = await prisma.entity.findFirst({
453
- where: { rocrateId: file.memberOf },
454
- });
455
-
456
- return {
457
- ...file,
458
- parentEntity: parent ? {
459
- id: parent.rocrateId,
460
- name: parent.name,
461
- type: parent.entityType,
462
- } : null,
463
- };
464
- },
465
- ],
466
- });
467
- ```
468
-
469
- ### File Transformation Pipeline
470
-
471
- Every file response flows through this three-stage pipeline:
472
-
473
- 1. **Base file transformer** - Converts database File records to standard format
474
- 2. **File access transformer** - Adds access control information (content only)
475
- 3. **File transformers** - Optional additional transformations
476
-
477
- ### Applied Routes
478
-
479
- File transformers are applied to the files listing route:
480
-
481
- - `GET /files` - File list (each file transformed)
482
-
483
- **Note**: The `/file/:id` endpoint (for downloading file content) uses the `fileHandler` system, not file transformers. See the File Handler System section below for details.
484
-
485
- ## File Handler System
486
-
487
- The API provides two separate handler systems for serving different types of content. These are distinct from the file transformer system described above.
488
-
489
- **Important Distinction:**
490
-
491
- - **File Transformers** (above) - Transform file metadata in the `/files` listing endpoint response
492
- - **File Handler** (this section) - Serves actual file content via the `/file/:id` download endpoint
493
- - **RO-Crate Handler** (this section) - Serves RO-Crate metadata via the `/entity/:id/rocrate` endpoint
494
-
495
- ### File Handler (Required)
496
-
497
- The `fileHandler` parameter is **required**. It serves actual file content for the `/file/:id` endpoint.
498
-
499
- This handler is responsible for:
500
-
501
- - Streaming file content from your storage backend
502
- - Generating redirect URLs to external storage (S3, CDN, etc.)
503
- - Implementing access control for file downloads
504
- - Supporting HTTP range requests for media streaming
505
-
506
- ### RO-Crate Handler (Required)
507
-
508
- The `roCrateHandler` parameter is **required**. It serves RO-Crate metadata as JSON-LD
509
- for any entity type (Collection, Object, or File) via the `/entity/:id/rocrate` endpoint.
510
-
511
- **File Handler Example** (S3 with redirect):
512
-
513
- ```typescript
514
- fileHandler: {
515
- get: async (file) => {
516
- const command = new GetObjectCommand({
517
- Bucket: file.meta.bucket,
518
- Key: file.meta.s3Key,
519
- });
520
- const url = await getSignedUrl(s3, command, { expiresIn: 3600 });
521
-
522
- return { type: 'redirect', url };
523
- },
524
- head: async (file) => {
525
- return {
526
- contentType: file.mediaType,
527
- contentLength: file.size,
528
- };
529
- },
530
- }
531
- ```
532
-
533
- **File Handler Example** (local filesystem):
534
-
535
- ```typescript
536
- fileHandler: {
537
- get: async (file) => {
538
- const filePath = `/data/files/${file.meta.storagePath}`;
539
- const stats = await stat(filePath);
540
-
541
- return {
542
- type: 'stream',
543
- stream: createReadStream(filePath),
544
- metadata: {
545
- contentType: file.mediaType,
546
- contentLength: stats.size,
547
- lastModified: stats.mtime,
548
- },
549
- };
550
- },
551
- head: async (file) => {
552
- const filePath = `/data/files/${file.meta.storagePath}`;
553
- const stats = await stat(filePath);
554
-
555
- return {
556
- contentType: file.mediaType,
557
- contentLength: stats.size,
558
- lastModified: stats.mtime,
559
- };
560
- },
561
- }
562
- ```
563
-
564
- **RO-Crate Handler Example** (stream from database):
565
-
566
- ```typescript
567
- roCrateHandler: {
568
- get: async (entity) => {
569
- const jsonString = JSON.stringify(entity.rocrate, null, 2);
570
-
571
- return {
572
- type: 'stream',
573
- stream: Readable.from([jsonString]),
574
- metadata: {
575
- contentType: 'application/ld+json',
576
- contentLength: Buffer.byteLength(jsonString),
577
- },
578
- };
579
- },
580
- head: async (entity) => {
581
- const jsonString = JSON.stringify(entity.rocrate);
582
- return {
583
- contentType: 'application/ld+json',
584
- contentLength: Buffer.byteLength(jsonString),
585
- };
586
- },
587
- }
588
19
  ```
589
20
 
590
- ### File Handler Response Types
591
-
592
- The file handler must return one of two response types:
21
+ v5 is in beta, and the documentation site describes it. Until it is released, install it with
22
+ `pnpm add arocapi@beta`. Changes merged to `main` are published under the `beta` tag.
593
23
 
594
- **Redirect Response** - Redirect to external file location:
24
+ The v4 documentation is the [README at the v4.0.1 tag](https://github.com/crate-works/arocapi/blob/v4.0.1/README.md).
595
25
 
596
- ```typescript
597
- { type: 'redirect', url: 'https://storage.example.com/file.wav' }
598
- ```
26
+ ## Development
599
27
 
600
- **Stream Response** - Serve file content directly:
28
+ You need Node.js 22.18 or later, pnpm, and Docker with Docker Compose.
601
29
 
602
- ```typescript
603
- {
604
- type: 'stream',
605
- stream: Readable, // Node.js readable stream
606
- metadata: {
607
- contentType: 'audio/wav',
608
- contentLength: 1024,
609
- etag?: '"abc123"', // Optional
610
- lastModified?: new Date(), // Optional
611
- },
612
- }
30
+ ```sh
31
+ pnpm install
32
+ docker compose up -d # Postgres, OpenSearch and the Oni UI
33
+ pnpm generate # generate the Prisma client
34
+ pnpm db:migrate # apply migrations
35
+ pnpm seed # load the sample data in example/data
36
+ pnpm dev # build and serve src/index.dev.ts on http://localhost:9000
613
37
  ```
614
38
 
615
- ### Query Parameters
616
-
617
- The `/file/:id` endpoint supports these query parameters:
618
-
619
- - `disposition` - 'inline' (default) or 'attachment' for download prompts
620
- - `filename` - Custom filename for Content-Disposition header (defaults to entity.name)
621
- - `noRedirect` - If true with redirect response, returns JSON `{location: url}` instead of HTTP 302
622
-
623
- ### HTTP Range Support
624
-
625
- Range request support for media streaming can be implemented in your file
626
- handler by parsing the `Range` header from the request context and returning
627
- a partial stream:
628
-
629
- ```typescript
630
- fileHandler: {
631
- get: async (file, { request }) => {
632
- const range = request.headers.range;
633
- // Parse range header and return partial content stream
634
- // Your storage backend (S3, filesystem, etc.) likely supports range reads natively
635
- },
636
- }
637
- ```
638
-
639
- ### Authentication and Authorisation
640
-
641
- The library delegates authentication and authorisation to your transformers.
642
- The access transformer receives the full Fastify request context, so you can
643
- verify credentials and set access flags accordingly:
644
-
645
- ```typescript
646
- accessTransformer: async (entity, { request }) => {
647
- const token = request.headers.authorization;
648
- const user = await verifyToken(token);
649
-
650
- if (!user) {
651
- // Unauthenticated — metadata visible but content restricted
652
- return {
653
- ...entity,
654
- access: {
655
- metadata: true,
656
- content: false,
657
- contentAuthorizationUrl: 'https://auth.example.com/login',
658
- },
659
- };
660
- }
661
-
662
- const canAccess = await checkUserLicense(user, entity.contentLicenseId);
663
-
664
- return {
665
- ...entity,
666
- access: {
667
- metadata: true,
668
- content: canAccess,
669
- contentAuthorizationUrl: canAccess
670
- ? undefined
671
- : 'https://rems.example.com/request-access',
672
- },
673
- };
674
- }
675
- ```
676
-
677
- The same pattern applies to `fileAccessTransformer` for controlling file
678
- content downloads.
679
-
680
- ### Entity Meta Field
681
-
682
- The `meta` JSON field in the Entity model stores implementation-specific
683
- metadata for your file handler:
684
-
685
- ```typescript
686
- await prisma.entity.create({
687
- data: {
688
- id: 'http://example.com/file/123',
689
- name: 'audio.wav',
690
- entityType: 'http://schema.org/MediaObject',
691
- // ... other required fields (memberOf, rootCollection, metadataLicenseId, contentLicenseId)
692
- meta: {
693
- bucket: 's3://my-bucket',
694
- storagePath: 'collections/col-01',
695
- checksum: 'sha256:abc123...',
696
- },
697
- },
698
- });
699
- ```
700
-
701
- Your file handler can use this metadata to locate files in your storage system.
702
-
703
- ## Using with Express
704
-
705
- arocapi doesn't ship an Express adapter, but you can register the plugin on a
706
- Fastify instance and forward requests to it. Install `express` in your app
707
- first.
708
-
709
- ```typescript
710
- // src/index.ts
711
- import arocapi, { type Options } from 'arocapi';
712
- import express from 'express';
713
- import Fastify from 'fastify';
714
-
715
- const options: Options = {
716
- // Same options as passed to fastify.register in Creating Your API
717
- };
718
-
719
- const fastify = Fastify({
720
- logger: true,
721
- });
722
- fastify.register(arocapi, options);
723
- await fastify.ready();
724
-
725
- const app = express();
726
- app.use('/api', (req, res) => {
727
- fastify.routing(req, res);
728
- });
729
-
730
- const port = process.env.PORT || 3000;
731
- app.listen(port, () => {
732
- console.log(`API server listening on port ${port}`);
733
- });
734
- ```
735
-
736
- ## Development Workflow
737
-
738
- ### Local Development Setup
739
-
740
- 1. **Start the services:**
741
-
742
- ```bash
743
- # Start MySQL and OpenSearch using docker-compose
744
- docker compose up -d
745
- ```
746
-
747
- 2. **Set up the database:**
748
-
749
- ```bash
750
- # Generate Prisma client
751
- npx prisma generate
752
-
753
- # Apply database migrations
754
- npx prisma migrate deploy
755
- ```
756
-
757
- 3. **Run your application:**
758
-
759
- ```bash
760
- # For development mode
761
- pnpm run dev
762
-
763
- # Or start directly
764
- node dist/app.js
765
- ```
766
-
767
- 4. **Access your API:**
768
- - API: <http://localhost:3000>
769
- - Database: localhost:3306 (MySQL)
770
- - OpenSearch: <http://localhost:9200>
771
-
772
- ### Available Scripts
773
-
774
- When using arocapi in your project, you can use these scripts:
775
-
776
- ```bash
777
- # Development
778
- pnpm run dev # Start in development mode with hot reload
779
- pnpm run start # Start in production mode
780
-
781
- # Database Management
782
- npx prisma generate # Generate Prisma client
783
- npx prisma migrate dev # Create and apply new migration
784
- npx prisma migrate deploy # Apply existing migrations
785
- npx prisma studio # Open Prisma Studio (database GUI)
786
-
787
- # Database Console
788
- docker compose exec db mysql -u root -p catalog # Connect to MySQL console
789
-
790
- # Testing
791
- pnpm run test # Run test cases (if configured)
792
- ```
793
-
794
- ### Project Structure
795
-
796
- When implementing arocapi in your project, follow this structure:
797
-
798
- ```text
799
- your-project/
800
- ├── src/
801
- │ ├── app.ts # Your main application file
802
- │ └── generated/
803
- │ └── prisma/ # Generated Prisma client
804
- ├── prisma/
805
- │ ├── schema.prisma # Prisma schema
806
- │ ├── models/ # Your custom models
807
- │ └── arocapi/ # Symlink to arocapi models
808
- ├── prisma.config.ts # Prisma configuration
809
- ├── .env # Environment variables
810
- └── package.json
811
- ```
39
+ `.env` holds the local connection URLs and is committed, so don't put secrets in it.
812
40
 
813
- ## Resources
41
+ | Command | What it does |
42
+ | ----------------------------- | ------------------------------------------------------------------ |
43
+ | `pnpm lint` | Biome, type checking, the docs examples type check and knip |
44
+ | `pnpm test` | Start the test containers, apply migrations and run every test |
45
+ | `pnpm test src/app.test.ts` | Run one test file |
46
+ | `pnpm test:watch` | Run the tests in watch mode |
47
+ | `pnpm test:coverage` | Run the tests with an HTML coverage report |
48
+ | `pnpm dbconsole` | Open a psql console on the development database |
49
+ | `pnpm --filter website dev` | Run the documentation site locally |
50
+ | `pnpm --filter website build` | Build the documentation site, checking its diagrams and every link |
814
51
 
815
- - [RO-Crate](https://www.researchobject.org/ro-crate/specification)
816
- - [OCFL](https://ocfl.io/)
817
- - [PILARS](https://w3id.org/ldac/pilars)
818
- - [REMS](https://trust.aaf.edu.au/rems/)
52
+ Commit messages follow [Conventional Commits](https://www.conventionalcommits.org/); they decide the next version.
53
+ See [TESTING.md](TESTING.md) for the test setup and [RELEASING.md](RELEASING.md) for releases.