arocapi 4.0.1 → 5.0.0-beta.10

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (242) hide show
  1. package/LICENSE.md +1 -1
  2. package/README.md +37 -781
  3. package/dist/app.d.ts +38 -21
  4. package/dist/app.js +63 -45
  5. package/dist/app.js.map +1 -1
  6. package/dist/routes/capabilities.d.ts +7 -0
  7. package/dist/routes/capabilities.js +13 -0
  8. package/dist/routes/capabilities.js.map +1 -0
  9. package/dist/routes/entities.d.ts +5 -2
  10. package/dist/routes/entities.js +38 -58
  11. package/dist/routes/entities.js.map +1 -1
  12. package/dist/routes/entity.d.ts +7 -2
  13. package/dist/routes/entity.js +14 -30
  14. package/dist/routes/entity.js.map +1 -1
  15. package/dist/routes/file.d.ts +8 -2
  16. package/dist/routes/file.js +38 -73
  17. package/dist/routes/file.js.map +1 -1
  18. package/dist/routes/files.d.ts +5 -2
  19. package/dist/routes/files.js +25 -40
  20. package/dist/routes/files.js.map +1 -1
  21. package/dist/routes/metadata.d.ts +15 -0
  22. package/dist/routes/metadata.js +38 -0
  23. package/dist/routes/metadata.js.map +1 -0
  24. package/dist/routes/search.d.ts +8 -7
  25. package/dist/routes/search.js +91 -120
  26. package/dist/routes/search.js.map +1 -1
  27. package/dist/search/advanced.d.ts +8 -0
  28. package/dist/search/advanced.js +198 -0
  29. package/dist/search/advanced.js.map +1 -0
  30. package/dist/search/backend.d.ts +95 -0
  31. package/dist/search/backend.js +5 -0
  32. package/dist/search/backend.js.map +1 -0
  33. package/dist/search/fields.d.ts +63 -0
  34. package/dist/search/fields.js +203 -0
  35. package/dist/search/fields.js.map +1 -0
  36. package/dist/search/opensearch/advanced.d.ts +4 -0
  37. package/dist/search/opensearch/advanced.js +28 -0
  38. package/dist/search/opensearch/advanced.js.map +1 -0
  39. package/dist/search/opensearch/opensearch.d.ts +16 -0
  40. package/dist/search/opensearch/opensearch.js +187 -0
  41. package/dist/search/opensearch/opensearch.js.map +1 -0
  42. package/dist/search/opensearch/visibility.d.ts +3 -0
  43. package/dist/search/opensearch/visibility.js +27 -0
  44. package/dist/search/opensearch/visibility.js.map +1 -0
  45. package/dist/search/postgres/advanced.d.ts +7 -0
  46. package/dist/search/postgres/advanced.js +54 -0
  47. package/dist/search/postgres/advanced.js.map +1 -0
  48. package/dist/search/postgres/facets.d.ts +2 -0
  49. package/dist/search/postgres/facets.js +27 -0
  50. package/dist/search/postgres/facets.js.map +1 -0
  51. package/dist/search/postgres/geo.d.ts +6 -0
  52. package/dist/search/postgres/geo.js +42 -0
  53. package/dist/search/postgres/geo.js.map +1 -0
  54. package/dist/search/postgres/postgres.d.ts +19 -0
  55. package/dist/search/postgres/postgres.js +157 -0
  56. package/dist/search/postgres/postgres.js.map +1 -0
  57. package/dist/search/postgres/values.d.ts +8 -0
  58. package/dist/search/postgres/values.js +20 -0
  59. package/dist/search/postgres/values.js.map +1 -0
  60. package/dist/search/postgres/visibility.d.ts +4 -0
  61. package/dist/search/postgres/visibility.js +38 -0
  62. package/dist/search/postgres/visibility.js.map +1 -0
  63. package/dist/search/requestFilters.d.ts +39 -0
  64. package/dist/search/requestFilters.js +125 -0
  65. package/dist/search/requestFilters.js.map +1 -0
  66. package/dist/search/visibility.d.ts +51 -0
  67. package/dist/search/visibility.js +100 -0
  68. package/dist/search/visibility.js.map +1 -0
  69. package/dist/storage/backend.d.ts +58 -0
  70. package/dist/storage/backend.js +2 -0
  71. package/dist/storage/backend.js.map +1 -0
  72. package/dist/storage/deliver.d.ts +12 -0
  73. package/dist/storage/deliver.js +34 -0
  74. package/dist/storage/deliver.js.map +1 -0
  75. package/dist/storage/directory/directory.d.ts +6 -0
  76. package/dist/storage/directory/directory.js +24 -0
  77. package/dist/storage/directory/directory.js.map +1 -0
  78. package/dist/storage/entity-meta/entity-meta.d.ts +5 -0
  79. package/dist/storage/entity-meta/entity-meta.js +14 -0
  80. package/dist/storage/entity-meta/entity-meta.js.map +1 -0
  81. package/dist/storage/location.d.ts +19 -0
  82. package/dist/storage/location.js +36 -0
  83. package/dist/storage/location.js.map +1 -0
  84. package/dist/storage/ocfl/ocfl.d.ts +6 -0
  85. package/dist/storage/ocfl/ocfl.js +45 -0
  86. package/dist/storage/ocfl/ocfl.js.map +1 -0
  87. package/dist/storage/results.d.ts +7 -0
  88. package/dist/storage/results.js +22 -0
  89. package/dist/storage/results.js.map +1 -0
  90. package/dist/storage/s3/s3.d.ts +10 -0
  91. package/dist/storage/s3/s3.js +47 -0
  92. package/dist/storage/s3/s3.js.map +1 -0
  93. package/dist/transformers/default.d.ts +11 -22
  94. package/dist/transformers/default.js +41 -15
  95. package/dist/transformers/default.js.map +1 -1
  96. package/dist/types/access.d.ts +9 -0
  97. package/dist/types/access.js +2 -0
  98. package/dist/types/access.js.map +1 -0
  99. package/dist/types/context.d.ts +6 -0
  100. package/dist/types/context.js +2 -0
  101. package/dist/types/context.js.map +1 -0
  102. package/dist/types/transformers.d.ts +4 -10
  103. package/dist/types/visibility.d.ts +6 -0
  104. package/dist/types/visibility.js +2 -0
  105. package/dist/types/visibility.js.map +1 -0
  106. package/dist/utils/access.d.ts +17 -0
  107. package/dist/utils/access.js +41 -0
  108. package/dist/utils/access.js.map +1 -0
  109. package/dist/utils/errors.d.ts +19 -10
  110. package/dist/utils/errors.js +44 -8
  111. package/dist/utils/errors.js.map +1 -1
  112. package/dist/utils/headers.d.ts +1 -1
  113. package/dist/utils/headers.js +10 -1
  114. package/dist/utils/headers.js.map +1 -1
  115. package/dist/utils/validation.d.ts +2 -0
  116. package/dist/utils/validation.js +8 -0
  117. package/dist/utils/validation.js.map +1 -0
  118. package/dist/utils/visibility.d.ts +18 -0
  119. package/dist/utils/visibility.js +16 -0
  120. package/dist/utils/visibility.js.map +1 -0
  121. package/package.json +84 -45
  122. package/prisma/models/entity.prisma +9 -8
  123. package/prisma/models/entitySearch.prisma +21 -0
  124. package/prisma/models/file.prisma +3 -3
  125. package/src/app.ts +169 -102
  126. package/src/routes/capabilities.ts +22 -0
  127. package/src/routes/entities.ts +49 -68
  128. package/src/routes/entity.ts +22 -36
  129. package/src/routes/file.ts +54 -104
  130. package/src/routes/files.ts +32 -48
  131. package/src/routes/metadata.ts +67 -0
  132. package/src/routes/search.ts +132 -152
  133. package/src/search/advanced.ts +258 -0
  134. package/src/search/backend.ts +91 -0
  135. package/src/search/fields.ts +320 -0
  136. package/src/search/opensearch/advanced.ts +36 -0
  137. package/src/search/opensearch/opensearch.ts +290 -0
  138. package/src/search/opensearch/visibility.ts +38 -0
  139. package/src/search/postgres/advanced.ts +76 -0
  140. package/src/search/postgres/facets.ts +34 -0
  141. package/src/search/postgres/geo.ts +58 -0
  142. package/src/search/postgres/postgres.ts +232 -0
  143. package/src/search/postgres/values.ts +33 -0
  144. package/src/search/postgres/visibility.ts +54 -0
  145. package/src/search/requestFilters.ts +178 -0
  146. package/src/search/visibility.ts +193 -0
  147. package/src/storage/backend.ts +75 -0
  148. package/src/storage/deliver.ts +65 -0
  149. package/src/storage/directory/directory.ts +39 -0
  150. package/src/storage/entity-meta/entity-meta.ts +25 -0
  151. package/src/storage/location.ts +78 -0
  152. package/src/storage/ocfl/ocfl.ts +71 -0
  153. package/src/storage/results.ts +32 -0
  154. package/src/storage/s3/s3.ts +79 -0
  155. package/src/transformers/default.ts +115 -85
  156. package/src/types/access.ts +25 -0
  157. package/src/types/context.ts +15 -0
  158. package/src/types/transformers.ts +8 -34
  159. package/src/types/visibility.ts +21 -0
  160. package/src/utils/access.ts +80 -0
  161. package/src/utils/errors.ts +68 -11
  162. package/src/utils/headers.ts +22 -2
  163. package/src/utils/validation.ts +11 -0
  164. package/src/utils/visibility.ts +56 -0
  165. package/dist/app.test.d.ts +0 -1
  166. package/dist/app.test.js +0 -89
  167. package/dist/app.test.js.map +0 -1
  168. package/dist/express.d.ts +0 -4
  169. package/dist/express.js +0 -14
  170. package/dist/express.js.map +0 -1
  171. package/dist/index.dev.d.ts +0 -1
  172. package/dist/index.dev.js +0 -156
  173. package/dist/index.dev.js.map +0 -1
  174. package/dist/routes/crate.d.ts +0 -9
  175. package/dist/routes/crate.js +0 -82
  176. package/dist/routes/crate.js.map +0 -1
  177. package/dist/routes/crate.test.d.ts +0 -1
  178. package/dist/routes/crate.test.js +0 -342
  179. package/dist/routes/crate.test.js.map +0 -1
  180. package/dist/routes/entities.test.d.ts +0 -1
  181. package/dist/routes/entities.test.js +0 -273
  182. package/dist/routes/entities.test.js.map +0 -1
  183. package/dist/routes/entity.test.d.ts +0 -1
  184. package/dist/routes/entity.test.js +0 -130
  185. package/dist/routes/entity.test.js.map +0 -1
  186. package/dist/routes/file.test.d.ts +0 -1
  187. package/dist/routes/file.test.js +0 -315
  188. package/dist/routes/file.test.js.map +0 -1
  189. package/dist/routes/files.test.d.ts +0 -1
  190. package/dist/routes/files.test.js +0 -222
  191. package/dist/routes/files.test.js.map +0 -1
  192. package/dist/routes/search.test.d.ts +0 -1
  193. package/dist/routes/search.test.js +0 -780
  194. package/dist/routes/search.test.js.map +0 -1
  195. package/dist/test/helpers/fastify.d.ts +0 -9
  196. package/dist/test/helpers/fastify.js +0 -18
  197. package/dist/test/helpers/fastify.js.map +0 -1
  198. package/dist/test/integration.setup.d.ts +0 -29
  199. package/dist/test/integration.setup.js +0 -188
  200. package/dist/test/integration.setup.js.map +0 -1
  201. package/dist/test/integration.test.d.ts +0 -1
  202. package/dist/test/integration.test.js +0 -396
  203. package/dist/test/integration.test.js.map +0 -1
  204. package/dist/transformers/default.test.d.ts +0 -1
  205. package/dist/transformers/default.test.js +0 -163
  206. package/dist/transformers/default.test.js.map +0 -1
  207. package/dist/transformers/transformer.test.d.ts +0 -1
  208. package/dist/transformers/transformer.test.js +0 -187
  209. package/dist/transformers/transformer.test.js.map +0 -1
  210. package/dist/types/fileHandlers.d.ts +0 -41
  211. package/dist/types/fileHandlers.js +0 -2
  212. package/dist/types/fileHandlers.js.map +0 -1
  213. package/dist/utils/errors.test.d.ts +0 -1
  214. package/dist/utils/errors.test.js +0 -66
  215. package/dist/utils/errors.test.js.map +0 -1
  216. package/dist/utils/queryBuilder.d.ts +0 -35
  217. package/dist/utils/queryBuilder.js +0 -98
  218. package/dist/utils/queryBuilder.js.map +0 -1
  219. package/prisma.config.ts +0 -9
  220. package/src/app.test.ts +0 -119
  221. package/src/express.ts +0 -21
  222. package/src/index.dev.ts +0 -200
  223. package/src/reset.d.ts +0 -1
  224. package/src/routes/__snapshots__/entities.test.ts.snap +0 -96
  225. package/src/routes/__snapshots__/entity.test.ts.snap +0 -74
  226. package/src/routes/__snapshots__/search.test.ts.snap +0 -165
  227. package/src/routes/crate.test.ts +0 -423
  228. package/src/routes/crate.ts +0 -123
  229. package/src/routes/entities.test.ts +0 -320
  230. package/src/routes/entity.test.ts +0 -160
  231. package/src/routes/file.test.ts +0 -395
  232. package/src/routes/files.test.ts +0 -278
  233. package/src/routes/search.test.ts +0 -903
  234. package/src/test/__snapshots__/integration.test.ts.snap +0 -18
  235. package/src/test/helpers/fastify.ts +0 -27
  236. package/src/test/integration.setup.ts +0 -210
  237. package/src/test/integration.test.ts +0 -482
  238. package/src/transformers/default.test.ts +0 -184
  239. package/src/transformers/transformer.test.ts +0 -252
  240. package/src/types/fileHandlers.ts +0 -65
  241. package/src/utils/errors.test.ts +0 -75
  242. package/src/utils/queryBuilder.ts +0 -132
package/README.md CHANGED
@@ -1,797 +1,53 @@
1
- # An RO-Crate API for Data Collections : Arocapi
1
+ # arocapi
2
2
 
3
- This project a prototype implementation of an RO-Crate based collections
4
- archive and data portal based on the Portland Common Data model of collections
5
- suitable use in long-term archival repository systems that aim to follow the
6
- Protocols for Long Term Archival Repository Services
7
- [PILARS](https://w3id.org/ldac/pilars).
3
+ arocapi implements the [RO-Crate API](https://ro-crate-api.crate-works.org) for data collections, packaged as a
4
+ Fastify plugin. You mount it into your own Fastify app and bring your storage, search index and access
5
+ rules; arocapi serves the routes for entities, files, search and RO-Crate metadata, following the
6
+ [PILARS](https://w3id.org/ldac/pilars) protocols for long-term archival repositories.
8
7
 
9
- The library can be used to mount the API endpoints into your own Fastify or
10
- Express app.
8
+ **Documentation: <https://arocapi.crate-works.org>**
11
9
 
12
- The aim is to provide an API-centred reference implementation of a repository
13
- with an access-controlled web portal for data dissemination and deposit:
10
+ - [What is arocapi](https://arocapi.crate-works.org/introduction/what-is-arocapi/)
11
+ - [Getting started](https://arocapi.crate-works.org/getting-started/install/)
12
+ - [Configuration](https://arocapi.crate-works.org/configuration/data-sources/)
13
+ - [Routes](https://arocapi.crate-works.org/reference/routes/)
14
14
 
15
- - Backend-agnostic RO-Crate storage using pluggable storage-layers
16
- - [OCFL] - the Oxford Common File Layout specification on disk or cloud-based
17
- object storage
18
- - Object Storage as used in [PARADISEC]
19
- - A simple RO-Crate file layout (TODO - likely a super-simple protocol for
20
- directory hierarchy with the presence on RO-Crate used to indicate that
21
- everything under that directory is a single RO-Crate)
22
- - Authorisation based on the concept of Access (and Deposit, TODO) Licenses
23
- using an arms-length process with an external authority such as a REMS
24
- instance or other license/group management system
25
- - Based on the principle that all data MUST have a implementation-neutral
26
- natural-language description of its access conditions stored with it
27
- - Licenses may be based on 'traditional' access control lists maintained in
28
- software such as PARADISEC or a license manager such as REMS, (TODO: an
29
- example implementation using GitHub groups to manage license access, a
30
- simple ACL manager?)).
15
+ ## Install
31
16
 
32
- ## Prerequisites
33
-
34
- - Node.js (version 22 or higher)
35
- - pnpm package manager
36
- - Database: MySQL or PostgreSQL
37
- - OpenSearch for search functionality
38
- - Docker and Docker Compose (for development)
39
-
40
- ## Quick Start
41
-
42
- ### 1. Installation
43
-
44
- ```bash
45
- # Install the package
17
+ ```sh
46
18
  pnpm add arocapi
47
-
48
- # Install required peer dependencies
49
- pnpm add @prisma/client @opensearch-project/opensearch
50
- pnpm add -D prisma
51
- ```
52
-
53
- ### 2. Environment Setup
54
-
55
- Create a `.env` file in your project root:
56
-
57
- ```env
58
- # Database Configuration (You can use an DB that Prisma supports)
59
- DATABASE_URL="mysql://root:password@localhost:3306/catalog"
60
-
61
- # OpenSearch Configuration
62
- OPENSEARCH_URL="http://localhost:9200"
63
- ```
64
-
65
- ## Creating Your API
66
-
67
- > [!TIP]
68
- > We recommend using the Fastify as it will be a native integration
69
-
70
- ### Option 1: Fastify Implementation
71
-
72
- Create your Fastify application with Typescript support:
73
-
74
- ```typescript
75
- // src/index.ts
76
- import { Client } from '@opensearch-project/opensearch';
77
- import arocapi, { AllPublicAccessTransformer, AllPublicFileAccessTransformer } from 'arocapi';
78
- import Fastify from 'fastify';
79
- import { PrismaClient } from './generated/prisma/client.js';
80
- import { Readable } from 'stream';
81
-
82
- const prisma = new PrismaClient();
83
-
84
- if (!process.env.OPENSEARCH_URL) {
85
- throw new Error('OPENSEARCH_URL environment variable is not set');
86
- }
87
- const opensearchUrl = process.env.OPENSEARCH_URL;
88
- const opensearch = new Client({ node: opensearchUrl });
89
-
90
- const fastify = Fastify({
91
- logger: true,
92
- });
93
-
94
- // For fully public datasets
95
- fastify.register(arocapi, {
96
- prisma,
97
- opensearch,
98
- accessTransformer: AllPublicAccessTransformer,
99
- fileAccessTransformer: AllPublicFileAccessTransformer,
100
- // Required: File handler for serving File entity content
101
- fileHandler: {
102
- get: async (file) => {
103
- const fileUrl = `https://storage.example.com/${file.meta.storagePath}`;
104
- return { type: 'redirect', url: fileUrl };
105
- },
106
- head: async (file) => ({
107
- contentType: file.mediaType,
108
- contentLength: file.size,
109
- }),
110
- },
111
- // Required: RO-Crate handler for serving RO-Crate metadata
112
- roCrateHandler: {
113
- get: async (entity) => {
114
- const jsonString = JSON.stringify(entity.rocrate, null, 2);
115
- return {
116
- type: 'stream',
117
- stream: Readable.from([jsonString]),
118
- metadata: {
119
- contentType: 'application/ld+json',
120
- contentLength: Buffer.byteLength(jsonString),
121
- },
122
- };
123
- },
124
- head: async (entity) => ({
125
- contentType: 'application/ld+json',
126
- contentLength: Buffer.byteLength(JSON.stringify(entity.rocrate)),
127
- }),
128
- },
129
- });
130
-
131
- try {
132
- await fastify.listen({ port: 3000 });
133
- } catch (err) {
134
- fastify.log.error(err);
135
- process.exit(1);
136
- }
137
- ```
138
-
139
- ### Option 2: Express Implementation
140
-
141
- Create your Express application:
142
-
143
- ```typescript
144
- // src/app.ts
145
- import { Client } from '@opensearch-project/opensearch';
146
- import Arocapi from 'arocapi/express';
147
- import express from 'express';
148
- import expressListRoutes from 'express-list-routes';
149
- import { PrismaClient } from './generated/prisma/client.js';
150
-
151
- const prisma = new PrismaClient();
152
-
153
- if (!process.env.OPENSEARCH_URL) {
154
- throw new Error('OPENSEARCH_URL environment variable is not set');
155
- }
156
- const opensearchUrl = process.env.OPENSEARCH_URL;
157
- const opensearch = new Client({ node: opensearchUrl });
158
-
159
- const app = express();
160
-
161
- const arocapi = await Arocapi({ opensearch, prisma });
162
- app.use('/api', arocapi);
163
-
164
- const port = process.env.PORT || 3000;
165
- app.listen(port, () => {
166
- console.log(`API server listening on port ${port}`);
167
- });
168
- ```
169
-
170
- ## Database Setup with Prisma
171
-
172
- ### 1. Initialize Prisma
173
-
174
- ```bash
175
- # Initialize prisma with your preferred database provider
176
- npx prisma init --datasource-provider mysql
177
- # or for PostgreSQL: npx prisma init --datasource-provider postgresql
178
- ```
179
-
180
- ### 2. Configure Prisma Multi-Model Setup
181
-
182
- Modify the `prisma.config.ts` file in your project root to support folder based schemas:
183
-
184
- ```typescript
185
- // prisma.config.ts
186
- import 'dotenv/config';
187
- import { defineConfig } from 'prisma/config';
188
-
189
- export default defineConfig({
190
- // Adds support for folder based schemas
191
- schema: 'prisma',
192
- });
193
- ```
194
-
195
- ### 3. Set Up Model Structure
196
-
197
- Create the directory structure and link to arocapi models:
198
-
199
- ```bash
200
- # Create prisma models directory
201
- mkdir -p prisma/models
202
-
203
- # Link to the upstream arocapi models
204
- ln -s ../../node_modules/arocapi/prisma/models prisma/arocapi
205
- ```
206
-
207
- ### 4. Generate Prisma Client and Run Migrations
208
-
209
- ```bash
210
- # Create database migrations for this version of arocapi
211
- npx prisma migrate dev --name="arocapi-$(pnpm list | grep arocapi | awk '{ print $2 }')"
212
-
213
- # Generate the Prisma client
214
- npx prisma generate
215
-
216
- ```
217
-
218
- ## Available API Routes
219
-
220
- The arocapi provides the following endpoints:
221
-
222
- - `GET /entities` - List all entities with pagination and filtering
223
- - `GET /entity/:id` - Get a specific entity by ID
224
- - `GET /entity/:id/rocrate` - Download RO-Crate metadata
225
- - `GET /files` - List files with pagination and filtering
226
- - `GET /file/:id` - Download or access file content
227
- - `POST /search` - Search entities using OpenSearch
228
-
229
- ## Customising OpenSearch query
230
-
231
- The OpenSearch query builder used in the `/search` endpoint can be customised to match any
232
- structure of the actual OpenSearch data. The default OpensearchQueryBuilder class is used to
233
- build all the queries for the search and an optional config can be passed on
234
- to customise the class behaviour.
235
-
236
- ### Query Builder (Optional)
237
- Use the `queryBuilderClass` optional parameter to specify a custom builder class.
238
- If not specified, the default `OpensearchQueryBuilder` is used.
239
- The custom class should extend the `OpensearchQueryBuilder` class and
240
- may override any or all of the methods as required:
241
-
242
- - `buildQuery`: For generating the base search query.
243
- - `buildAggregations`: For generating the facets or aggregations related query.
244
- - `buildSort`: For generating the ordering/sorting part of the query.
245
-
246
- ### Query Builder Options (Optional)
247
- The `queryBuilderOptions` optional parameter is passed as a constructor argument
248
- when instantiating the queryBuilderClass. The default `OpensearchQueryBuilder` can be
249
- configured with:
250
-
251
- - `aggregations`: This field controls the aggregation and will be passed verbatim as the `aggs` field in the generated search query.
252
-
253
- ## Customising Entity Responses
254
-
255
- The API provides a flexible transformer system for customising entity responses
256
- through two types of transformers:
257
-
258
- ### Access Transformer (Required)
259
-
260
- The `accessTransformer` parameter is **required** for security. You must explicitly
261
- choose how access control is handled for your repository.
262
-
263
- **For fully public datasets**, use `AllPublicAccessTransformer`:
264
-
265
- ```typescript
266
- import arocapi, { AllPublicAccessTransformer } from 'arocapi';
267
-
268
- await server.register(arocapi, {
269
- prisma,
270
- opensearch,
271
- accessTransformer: AllPublicAccessTransformer,
272
- });
273
- ```
274
-
275
- **For restricted content**, implement a custom access transformer:
276
-
277
- ```typescript
278
- const accessTransformer = async (entity, { request }) => {
279
- // Custom logic to determine access
280
- const user = await authenticateUser(request);
281
- const canAccessContent = await checkLicense(entity.contentLicenseId, user);
282
-
283
- return {
284
- ...entity,
285
- access: {
286
- metadata: true, // Metadata always visible
287
- content: canAccessContent,
288
- contentAuthorisationUrl: canAccessContent
289
- ? undefined
290
- : 'https://rems.example.com/request-access',
291
- },
292
- };
293
- };
294
- ```
295
-
296
- > [!WARNING]
297
- > The `accessTransformer` is required to prevent accidental exposure of
298
- > restricted content. You must make an explicit choice about access control for
299
- > your repository.
300
-
301
- ### Entity Transformers
302
-
303
- Optional transformations for enriching or modifying response data. Multiple
304
- transformers can be chained together.
305
-
306
- ```typescript
307
- await server.register(arocapi, {
308
- prisma,
309
- opensearch,
310
- accessTransformer: AllPublicAccessTransformer,
311
- entityTransformers: [
312
- // Add computed fields
313
- (entity) => ({
314
- ...entity,
315
- displayName: `${entity.name} [${entity.entityType.split('/').pop()}]`,
316
- }),
317
- // Add counts
318
- async (entity) => {
319
- const objectCount = entity.memberOf
320
- ? await prisma.entity.count({
321
- where: { memberOf: entity.id },
322
- })
323
- : 0;
324
-
325
- return {
326
- ...entity,
327
- counts: {
328
- objects: objectCount,
329
- }
330
- };
331
- },
332
- ],
333
- });
334
- ```
335
-
336
- ### Transformation Pipeline
337
-
338
- Every entity response flows through this three-stage pipeline:
339
-
340
- 1. **Base transformer** - Converts database entities to standard format
341
- 2. **Access transformer** - Adds access control information
342
- 3. **Entity transformers** - Optional additional transformations
343
-
344
- ### Transformer Context
345
-
346
- Access and entity transformers receive a `TransformerContext` as their second argument with the shape `{ request, fastify }`. The `request` is the Fastify request for per-call data (headers, auth, query). The `fastify` instance is provided as an extension point — arocapi does not decorate it with anything itself, but consumers who decorate `fastify` with their own helpers (e.g. via `fastify.decorate('userService', ...)`) can read those decorations through `context.fastify` from inside their transformers. For database access, prefer closing over your own client reference as shown in the examples below.
347
-
348
- ### Common Use Cases
349
-
350
- **Access control for restricted content:**
351
-
352
- ```typescript
353
- accessTransformer: async (entity, { request }) => {
354
- const hasAccess = await checkUserPermissions(request, entity.contentLicenseId);
355
- return {
356
- ...entity,
357
- access: {
358
- metadata: true,
359
- content: hasAccess,
360
- },
361
- };
362
- }
363
- ```
364
-
365
- **Adding computed or derived fields:**
366
-
367
- ```typescript
368
- entityTransformers: [
369
- (entity) => ({
370
- ...entity,
371
- shortId: entity.id.split('/').pop(),
372
- year: extractYear(entity.description),
373
- }),
374
- ]
375
- ```
376
-
377
- **Fetching related data asynchronously:**
378
-
379
- ```typescript
380
- entityTransformers: [
381
- async (entity) => ({
382
- ...entity,
383
- stats: await fetchEntityStats(entity.id, prisma),
384
- }),
385
- ]
386
- ```
387
-
388
- ## Customising File Responses
389
-
390
- The API provides a separate transformer system for file responses, similar to entity transformers but specifically designed for files.
391
-
392
- ### File Access Transformer (Required)
393
-
394
- The `fileAccessTransformer` parameter is **required** for security. You must explicitly choose how access control is handled for file content.
395
-
396
- **Key Difference from Entity Transformers**: File metadata (filename, size, mediaType, etc.) is always accessible. Only content access is controlled via the `access.content` field.
397
-
398
- **For fully public datasets**, use `AllPublicFileAccessTransformer`:
399
-
400
- ```typescript
401
- import arocapi, { AllPublicAccessTransformer, AllPublicFileAccessTransformer } from 'arocapi';
402
-
403
- await server.register(arocapi, {
404
- prisma,
405
- opensearch,
406
- accessTransformer: AllPublicAccessTransformer,
407
- fileAccessTransformer: AllPublicFileAccessTransformer,
408
- });
409
- ```
410
-
411
- **For restricted content**, implement a custom file access transformer:
412
-
413
- ```typescript
414
- const fileAccessTransformer = async (file, { request }) => {
415
- // Custom logic to determine file content access
416
- const user = await authenticateUser(request);
417
- const canAccessContent = await checkLicense(file.contentLicenseId, user);
418
-
419
- return {
420
- ...file,
421
- access: {
422
- content: canAccessContent,
423
- contentAuthorizationUrl: canAccessContent
424
- ? undefined
425
- : 'https://rems.example.com/request-access',
426
- },
427
- };
428
- };
429
-
430
- await server.register(arocapi, {
431
- prisma,
432
- opensearch,
433
- accessTransformer: AllPublicAccessTransformer,
434
- fileAccessTransformer,
435
- });
436
- ```
437
-
438
- > [!WARNING]
439
- > The `fileAccessTransformer` is required to prevent accidental exposure of
440
- > restricted file content. You must make an explicit choice about access control
441
- > for your files.
442
-
443
- ### File Transformers
444
-
445
- Optional transformations for enriching or modifying file response data. Multiple
446
- file transformers can be chained together.
447
-
448
- ```typescript
449
- await server.register(arocapi, {
450
- prisma,
451
- opensearch,
452
- accessTransformer: AllPublicAccessTransformer,
453
- fileAccessTransformer: AllPublicFileAccessTransformer,
454
- fileTransformers: [
455
- // Add computed fields
456
- (file) => ({
457
- ...file,
458
- displayFilename: file.filename.toUpperCase(),
459
- sizeInKB: Math.round(file.size / 1024),
460
- extension: file.filename.split('.').pop(),
461
- }),
462
- // Fetch parent entity information
463
- async (file) => {
464
- const parent = await prisma.entity.findFirst({
465
- where: { rocrateId: file.memberOf },
466
- });
467
-
468
- return {
469
- ...file,
470
- parentEntity: parent ? {
471
- id: parent.rocrateId,
472
- name: parent.name,
473
- type: parent.entityType,
474
- } : null,
475
- };
476
- },
477
- ],
478
- });
479
- ```
480
-
481
- ### File Transformation Pipeline
482
-
483
- Every file response flows through this three-stage pipeline:
484
-
485
- 1. **Base file transformer** - Converts database File records to standard format
486
- 2. **File access transformer** - Adds access control information (content only)
487
- 3. **File transformers** - Optional additional transformations
488
-
489
- ### Applied Routes
490
-
491
- File transformers are applied to the files listing route:
492
-
493
- - `GET /files` - File list (each file transformed)
494
-
495
- **Note**: The `/file/:id` endpoint (for downloading file content) uses the `fileHandler` system, not file transformers. See the File Handler System section below for details.
496
-
497
- ## File Handler System
498
-
499
- The API provides two separate handler systems for serving different types of content. These are distinct from the file transformer system described above.
500
-
501
- **Important Distinction:**
502
-
503
- - **File Transformers** (above) - Transform file metadata in the `/files` listing endpoint response
504
- - **File Handler** (this section) - Serves actual file content via the `/file/:id` download endpoint
505
- - **RO-Crate Handler** (this section) - Serves RO-Crate metadata via the `/entity/:id/rocrate` endpoint
506
-
507
- ### File Handler (Required)
508
-
509
- The `fileHandler` parameter is **required**. It serves actual file content for the `/file/:id` endpoint.
510
-
511
- This handler is responsible for:
512
-
513
- - Streaming file content from your storage backend
514
- - Generating redirect URLs to external storage (S3, CDN, etc.)
515
- - Implementing access control for file downloads
516
- - Supporting HTTP range requests for media streaming
517
-
518
- ### RO-Crate Handler (Required)
519
-
520
- The `roCrateHandler` parameter is **required**. It serves RO-Crate metadata as JSON-LD
521
- for any entity type (Collection, Object, or File) via the `/entity/:id/rocrate` endpoint.
522
-
523
- **File Handler Example** (S3 with redirect):
524
-
525
- ```typescript
526
- fileHandler: {
527
- get: async (file) => {
528
- const command = new GetObjectCommand({
529
- Bucket: file.meta.bucket,
530
- Key: file.meta.s3Key,
531
- });
532
- const url = await getSignedUrl(s3, command, { expiresIn: 3600 });
533
-
534
- return { type: 'redirect', url };
535
- },
536
- head: async (file) => {
537
- return {
538
- contentType: file.mediaType,
539
- contentLength: file.size,
540
- };
541
- },
542
- }
543
- ```
544
-
545
- **File Handler Example** (local filesystem):
546
-
547
- ```typescript
548
- fileHandler: {
549
- get: async (file) => {
550
- const filePath = `/data/files/${file.meta.storagePath}`;
551
- const stats = await stat(filePath);
552
-
553
- return {
554
- type: 'stream',
555
- stream: createReadStream(filePath),
556
- metadata: {
557
- contentType: file.mediaType,
558
- contentLength: stats.size,
559
- lastModified: stats.mtime,
560
- },
561
- };
562
- },
563
- head: async (file) => {
564
- const filePath = `/data/files/${file.meta.storagePath}`;
565
- const stats = await stat(filePath);
566
-
567
- return {
568
- contentType: file.mediaType,
569
- contentLength: stats.size,
570
- lastModified: stats.mtime,
571
- };
572
- },
573
- }
574
- ```
575
-
576
- **RO-Crate Handler Example** (stream from database):
577
-
578
- ```typescript
579
- roCrateHandler: {
580
- get: async (entity) => {
581
- const jsonString = JSON.stringify(entity.rocrate, null, 2);
582
-
583
- return {
584
- type: 'stream',
585
- stream: Readable.from([jsonString]),
586
- metadata: {
587
- contentType: 'application/ld+json',
588
- contentLength: Buffer.byteLength(jsonString),
589
- },
590
- };
591
- },
592
- head: async (entity) => {
593
- const jsonString = JSON.stringify(entity.rocrate);
594
- return {
595
- contentType: 'application/ld+json',
596
- contentLength: Buffer.byteLength(jsonString),
597
- };
598
- },
599
- }
600
- ```
601
-
602
- ### File Handler Response Types
603
-
604
- The file handler must return one of two response types:
605
-
606
- **Redirect Response** - Redirect to external file location:
607
-
608
- ```typescript
609
- { type: 'redirect', url: 'https://storage.example.com/file.wav' }
610
- ```
611
-
612
- **Stream Response** - Serve file content directly:
613
-
614
- ```typescript
615
- {
616
- type: 'stream',
617
- stream: Readable, // Node.js readable stream
618
- metadata: {
619
- contentType: 'audio/wav',
620
- contentLength: 1024,
621
- etag?: '"abc123"', // Optional
622
- lastModified?: new Date(), // Optional
623
- },
624
- }
625
- ```
626
-
627
- ### Query Parameters
628
-
629
- The `/file/:id` endpoint supports these query parameters:
630
-
631
- - `disposition` - 'inline' (default) or 'attachment' for download prompts
632
- - `filename` - Custom filename for Content-Disposition header (defaults to entity.name)
633
- - `noRedirect` - If true with redirect response, returns JSON `{location: url}` instead of HTTP 302
634
-
635
- ### HTTP Range Support
636
-
637
- Range request support for media streaming can be implemented in your file
638
- handler by parsing the `Range` header from the request context and returning
639
- a partial stream:
640
-
641
- ```typescript
642
- fileHandler: {
643
- get: async (file, { request }) => {
644
- const range = request.headers.range;
645
- // Parse range header and return partial content stream
646
- // Your storage backend (S3, filesystem, etc.) likely supports range reads natively
647
- },
648
- }
649
- ```
650
-
651
- ### Authentication and Authorisation
652
-
653
- The library delegates authentication and authorisation to your transformers.
654
- The access transformer receives the full Fastify request context, so you can
655
- verify credentials and set access flags accordingly:
656
-
657
- ```typescript
658
- accessTransformer: async (entity, { request }) => {
659
- const token = request.headers.authorization;
660
- const user = await verifyToken(token);
661
-
662
- if (!user) {
663
- // Unauthenticated — metadata visible but content restricted
664
- return {
665
- ...entity,
666
- access: {
667
- metadata: true,
668
- content: false,
669
- contentAuthorizationUrl: 'https://auth.example.com/login',
670
- },
671
- };
672
- }
673
-
674
- const canAccess = await checkUserLicense(user, entity.contentLicenseId);
675
-
676
- return {
677
- ...entity,
678
- access: {
679
- metadata: true,
680
- content: canAccess,
681
- contentAuthorizationUrl: canAccess
682
- ? undefined
683
- : 'https://rems.example.com/request-access',
684
- },
685
- };
686
- }
687
- ```
688
-
689
- The same pattern applies to `fileAccessTransformer` for controlling file
690
- content downloads.
691
-
692
- ### Entity Meta Field
693
-
694
- The `meta` JSON field in the Entity model stores implementation-specific
695
- metadata for your file handler:
696
-
697
- ```typescript
698
- await prisma.entity.create({
699
- data: {
700
- id: 'http://example.com/file/123',
701
- name: 'audio.wav',
702
- entityType: 'http://schema.org/MediaObject',
703
- // ... other required fields (memberOf, rootCollection, metadataLicenseId, contentLicenseId)
704
- meta: {
705
- bucket: 's3://my-bucket',
706
- storagePath: 'collections/col-01',
707
- checksum: 'sha256:abc123...',
708
- },
709
- },
710
- });
711
19
  ```
712
20
 
713
- Your file handler can use this metadata to locate files in your storage system.
714
-
715
- ## Development Workflow
716
-
717
- ### Local Development Setup
718
-
719
- 1. **Start the services:**
720
-
721
- ```bash
722
- # Start MySQL and OpenSearch using docker-compose
723
- docker compose up -d
724
- ```
725
-
726
- 2. **Set up the database:**
21
+ v5 is in beta, and the documentation site describes it. Until it is released, install it with
22
+ `pnpm add arocapi@beta`. Changes merged to `main` are published under the `beta` tag.
727
23
 
728
- ```bash
729
- # Generate Prisma client
730
- npx prisma generate
24
+ The v4 documentation is the [README at the v4.0.1 tag](https://github.com/crate-works/arocapi/blob/v4.0.1/README.md).
731
25
 
732
- # Apply database migrations
733
- npx prisma migrate deploy
734
- ```
26
+ ## Development
735
27
 
736
- 3. **Run your application:**
28
+ You need Node.js 22.18 or later, pnpm, and Docker with Docker Compose.
737
29
 
738
- ```bash
739
- # For development mode
740
- pnpm run dev
741
-
742
- # Or start directly
743
- node dist/app.js
744
- ```
745
-
746
- 4. **Access your API:**
747
- - API: <http://localhost:3000>
748
- - Database: localhost:3306 (MySQL)
749
- - OpenSearch: <http://localhost:9200>
750
-
751
- ### Available Scripts
752
-
753
- When using arocapi in your project, you can use these scripts:
754
-
755
- ```bash
756
- # Development
757
- pnpm run dev # Start in development mode with hot reload
758
- pnpm run start # Start in production mode
759
-
760
- # Database Management
761
- npx prisma generate # Generate Prisma client
762
- npx prisma migrate dev # Create and apply new migration
763
- npx prisma migrate deploy # Apply existing migrations
764
- npx prisma studio # Open Prisma Studio (database GUI)
765
-
766
- # Database Console
767
- docker compose exec db mysql -u root -p catalog # Connect to MySQL console
768
-
769
- # Testing
770
- pnpm run test # Run test cases (if configured)
30
+ ```sh
31
+ pnpm install
32
+ docker compose up -d # Postgres, OpenSearch and the Oni UI
33
+ pnpm generate # generate the Prisma client
34
+ pnpm db:migrate # apply migrations
35
+ pnpm seed # load the sample data in example/data
36
+ pnpm dev # build and serve src/index.dev.ts on http://localhost:9000
771
37
  ```
772
38
 
773
- ### Project Structure
774
-
775
- When implementing arocapi in your project, follow this structure:
776
-
777
- ```text
778
- your-project/
779
- ├── src/
780
- │ ├── app.ts # Your main application file
781
- │ └── generated/
782
- │ └── prisma/ # Generated Prisma client
783
- ├── prisma/
784
- │ ├── schema.prisma # Prisma schema
785
- │ ├── models/ # Your custom models
786
- │ └── arocapi/ # Symlink to arocapi models
787
- ├── prisma.config.ts # Prisma configuration
788
- ├── .env # Environment variables
789
- └── package.json
790
- ```
39
+ `.env` holds the local connection URLs and is committed, so don't put secrets in it.
791
40
 
792
- ## Resources
41
+ | Command | What it does |
42
+ | ----------------------------- | ------------------------------------------------------------------ |
43
+ | `pnpm lint` | Biome, type checking, the docs examples type check and knip |
44
+ | `pnpm test` | Start the test containers, apply migrations and run every test |
45
+ | `pnpm test src/app.test.ts` | Run one test file |
46
+ | `pnpm test:watch` | Run the tests in watch mode |
47
+ | `pnpm test:coverage` | Run the tests with an HTML coverage report |
48
+ | `pnpm dbconsole` | Open a psql console on the development database |
49
+ | `pnpm --filter website dev` | Run the documentation site locally |
50
+ | `pnpm --filter website build` | Build the documentation site, checking its diagrams and every link |
793
51
 
794
- - [RO-Crate](https://www.researchobject.org/ro-crate/specification)
795
- - [OCFL](https://ocfl.io/)
796
- - [PILARS](https://w3id.org/ldac/pilars)
797
- - [REMS](https://trust.aaf.edu.au/rems/)
52
+ Commit messages follow [Conventional Commits](https://www.conventionalcommits.org/); they decide the next version.
53
+ See [TESTING.md](TESTING.md) for the test setup and [RELEASING.md](RELEASING.md) for releases.