@usehenri/uploads 0.0.0 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +117 -0
- package/LICENSE +21 -0
- package/README.md +8 -1
- package/index.js +42 -0
- package/module.js +8 -0
- package/package.json +56 -10
- package/src/bytes.js +86 -0
- package/src/config.js +243 -0
- package/src/download.js +126 -0
- package/src/errors.js +92 -0
- package/src/file.js +204 -0
- package/src/module.js +694 -0
- package/src/multipart.js +696 -0
- package/src/names.js +191 -0
- package/src/signing.js +333 -0
- package/src/sniff.js +292 -0
- package/src/storage/index.js +125 -0
- package/src/storage/local.js +345 -0
- package/src/variants.js +392 -0
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,117 @@
|
|
|
1
|
+
# @usehenri/uploads
|
|
2
|
+
|
|
3
|
+
## 1.2.0
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- [#372](https://github.com/usehenri/henri/pull/372) [`1a86acb`](https://github.com/usehenri/henri/commit/1a86acbf15e4a43e5fb81277bb22e101c06e77a4) Thanks [@reel](https://github.com/reel)! - File uploads, in a new package: `@usehenri/uploads`.
|
|
8
|
+
|
|
9
|
+
An application installs it and gets `henri.uploads`, `req.files`,
|
|
10
|
+
`req.file(field)` and `req.permitFiles(...fields)` — `req.permit()` for files,
|
|
11
|
+
which removes what a controller did not ask for on the spot. `store()` moves a
|
|
12
|
+
file into the storage and answers the record to write to a model
|
|
13
|
+
(`{ key, name, type, size, checksum, storage, uploadedAt }`), and
|
|
14
|
+
`henri.uploads.send(res, record)` streams it back as a download.
|
|
15
|
+
|
|
16
|
+
The parser is busboy, and every bound is enforced as it reads rather than
|
|
17
|
+
checked afterwards: `maxTotalSize` (25mb, checked against `Content-Length`
|
|
18
|
+
first and counted again as the bytes arrive), `maxFileSize` (10mb), `maxFiles`
|
|
19
|
+
(10), `maxFields` (100), `maxFieldNameSize` (100 bytes) and `maxFieldSize`,
|
|
20
|
+
which defaults to `config.bodyLimit`. The type of a file is decided from its
|
|
21
|
+
first bytes, never from the `Content-Type` or the extension the client sent,
|
|
22
|
+
and `uploads.allow` matches that. The stored name is generated, so no name a
|
|
23
|
+
client sends ever reaches a path. Files land in `storage/uploads` (`0700`,
|
|
24
|
+
objects `0600`), outside everything the application serves, and nothing is kept
|
|
25
|
+
unless a controller calls `store()` — a request that is refused, times out or is
|
|
26
|
+
abandoned leaves no temporary file behind.
|
|
27
|
+
|
|
28
|
+
The local disk is one implementation of a documented `HenriStorage` contract;
|
|
29
|
+
an object store is another, named by module id in `config.uploads.storage` and
|
|
30
|
+
resolved from the application. henri ships no S3 client.
|
|
31
|
+
|
|
32
|
+
`@usehenri/core` gains the `uploads` configuration key (validated whether or
|
|
33
|
+
not the package is installed), the `req.files`/`req.file`/`req.permitFiles`
|
|
34
|
+
declarations, and `res.boom.payloadTooLarge()` and
|
|
35
|
+
`res.boom.unsupportedMediaType()`.
|
|
36
|
+
|
|
37
|
+
`henri audit` gains three checks: `uploads.limits-disabled` (V12.1.1),
|
|
38
|
+
`uploads.type-check-disabled` (V12.2.1) and `uploads.root-served` (V12.4.1).
|
|
39
|
+
|
|
40
|
+
- [#407](https://github.com/usehenri/henri/pull/407) [`de1c1e0`](https://github.com/usehenri/henri/commit/de1c1e02ed83d13dcfeb8e44012f309eb663f03e) Thanks [@reel](https://github.com/reel)! - Uploads that survive a second machine: an object store, signed urls and variants
|
|
41
|
+
|
|
42
|
+
`@usehenri/uploads` shipped one backend — the local disk — and said `url()`
|
|
43
|
+
was allowed to answer `null` forever. Two application processes on two
|
|
44
|
+
machines therefore did not share uploads at all, a link to a stored file
|
|
45
|
+
meant writing a controller and a route for every one of them, and a
|
|
46
|
+
thumbnail meant leaving the framework entirely. All three are closed here.
|
|
47
|
+
|
|
48
|
+
**`@usehenri/s3`, a new package.** `config.uploads.storage` takes an object
|
|
49
|
+
now (`{ "adapter": "s3", "bucket": "…", "region": "…" }`, the shape
|
|
50
|
+
`config.shared` and a store already have) and `s3` resolves the package from
|
|
51
|
+
the application. One backend speaks to S3, R2, Spaces, MinIO and GCS's
|
|
52
|
+
interoperability mode; what tells them apart is an endpoint and a region. It
|
|
53
|
+
carries no dependency but `debug` — AWS Signature Version 4 is two hundred
|
|
54
|
+
lines of `node:crypto`, checked against the vectors AWS publishes. Every
|
|
55
|
+
safety property survives the move: the key is generated and refused if it is
|
|
56
|
+
not, the type still comes from the bytes and becomes the object's
|
|
57
|
+
`Content-Type`, the original name is metadata, and a part still lands on a
|
|
58
|
+
private local file that only `store()` promotes.
|
|
59
|
+
|
|
60
|
+
**Signed urls.** `henri.uploads.url(record, { expiresIn, disposition,
|
|
61
|
+
filename, type })` is one call whatever the backend: the provider's own
|
|
62
|
+
signature on an object store, henri's own (an HMAC over the key, the expiry,
|
|
63
|
+
the disposition, the name and the type, verified by a route it mounts) on
|
|
64
|
+
the local disk. Neither can be edited to name another object, widened, or
|
|
65
|
+
replayed past its expiry. Until then a signed url **is** a bearer
|
|
66
|
+
capability, so `config.uploads.urls` is off by default and `url()` refuses
|
|
67
|
+
with `HENRI_UPLOAD_URLS_DISABLED` rather than answering `null`.
|
|
68
|
+
`uploads.urls.cdn` puts a cache in front of henri's own, whose signature
|
|
69
|
+
deliberately does not cover the host.
|
|
70
|
+
|
|
71
|
+
**Variants.** `config.uploads.variants` declares them by name and
|
|
72
|
+
`henri.uploads.variant(record, 'thumb')` answers a record like any other.
|
|
73
|
+
The key is the source's plus a digest of the variant's terms, so the work
|
|
74
|
+
happens once, on demand, and never in the request that uploaded. `sharp` is
|
|
75
|
+
an **optional peer dependency** resolved from the application: without it
|
|
76
|
+
`variant()` refuses with `HENRI_UPLOAD_NO_IMAGE_LIBRARY` and the install
|
|
77
|
+
line rather than quietly answering the original, and `henri doctor` reports
|
|
78
|
+
it. A name never comes from a request, an SVG is refused, a source is
|
|
79
|
+
bounded at fifty megapixels and one frame, no metadata is carried forward,
|
|
80
|
+
and what the resize produced is sniffed before it is stored.
|
|
81
|
+
|
|
82
|
+
New configuration: `uploads.storage` in its object form, `uploads.urls`
|
|
83
|
+
(`expiresIn`, `path`, `cdn`) and `uploads.variants`. New codes:
|
|
84
|
+
`HENRI_UPLOAD_NO_IMAGE_LIBRARY`, `HENRI_UPLOAD_STORAGE_FAILED`,
|
|
85
|
+
`HENRI_UPLOAD_STORAGE_MISCONFIGURED`, `HENRI_UPLOAD_URLS_DISABLED`,
|
|
86
|
+
`HENRI_UPLOAD_URL_EXPIRED`, `HENRI_UPLOAD_URL_INVALID`,
|
|
87
|
+
`HENRI_UPLOAD_VARIANT_FAILED`, `HENRI_UPLOAD_VARIANT_UNKNOWN`,
|
|
88
|
+
`HENRI_UPLOAD_VARIANT_UNSUPPORTED`. `henri doctor` asks for
|
|
89
|
+
`@usehenri/s3` and `sharp` when the configuration names them.
|
|
90
|
+
|
|
91
|
+
### Patch Changes
|
|
92
|
+
|
|
93
|
+
- [#384](https://github.com/usehenri/henri/pull/384) [`68fe3af`](https://github.com/usehenri/henri/commit/68fe3afa8bdaa6d9a95d0e60858fd4abe28028b2) Thanks [@reel](https://github.com/reel)! - Test factories: a valid record with the fields the test does not care about already filled in.
|
|
94
|
+
|
|
95
|
+
A factory lives in `test/factories/<name>.js` and is read the first time a test asks for one. It is a plain object -- `{ attributes, traits, model, after }` -- and `@usehenri/testing` now exports `create`, `build`, `createList`, `defineFactory` and `resetFactories`:
|
|
96
|
+
|
|
97
|
+
```js
|
|
98
|
+
// test/factories/proposal.js
|
|
99
|
+
module.exports = {
|
|
100
|
+
attributes: {
|
|
101
|
+
eventId: async ({ create }) => (await create('event')).id,
|
|
102
|
+
speakerId: async ({ create }) => (await create('user')).id,
|
|
103
|
+
title: ({ sequence }) => `A proposal ${sequence}`,
|
|
104
|
+
},
|
|
105
|
+
traits: { submitted: { state: 'submitted', submittedAt: () => new Date() } },
|
|
106
|
+
};
|
|
107
|
+
|
|
108
|
+
await create('proposal', 'submitted', { speakerId: me.id });
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
Three rules hold it together. What the caller gives is never made, so an override of an association creates no second record. A value is a literal or a function of the build context (`attrs`, `build`, `create`, `sequence`, `traits`, `uid`), so there is no separate vocabulary for associations, sequences or computed fields. Fields resolve on demand rather than in the order they are written, so `await attrs.eventId` from another field's function keeps two of them on one parent whatever order the keys sit in. A trait is an override object with a name, kept next to the model because a state is rarely one field.
|
|
112
|
+
|
|
113
|
+
A factory writes through the model, so the password is still hashed, the timestamps are still stamped and a `paranoid` model still soft-deletes; `after(record, context)` covers what the model refuses to mass assign. Failures carry the new `HENRI_FACTORY_*` codes.
|
|
114
|
+
|
|
115
|
+
`@usehenri/uploads`: the local storage sweeps only the parts older than an hour when it starts. It could not tell a part a dead process left behind from one another process is streaming into right now, so a second application process -- or a second test file -- booting mid-upload failed that upload.
|
|
116
|
+
- Updated dependencies [[`792a15a`](https://github.com/usehenri/henri/commit/792a15ade614cf8b920d9197586f9866700d458e), [`1e23664`](https://github.com/usehenri/henri/commit/1e23664829bd1a356de28f404cfb21c9ae211388), [`5b627ad`](https://github.com/usehenri/henri/commit/5b627adfa37e9f16bc75af96cc8ff5308a91f688), [`4dff51e`](https://github.com/usehenri/henri/commit/4dff51edc050e29398c793a5aedb48776b7b7119), [`60dbf33`](https://github.com/usehenri/henri/commit/60dbf33c2a4f14e328a0df1cb44be061e15431e5), [`1b316c6`](https://github.com/usehenri/henri/commit/1b316c6f3c5d5eb5752c70b534092d1052956cc6), [`d074e8b`](https://github.com/usehenri/henri/commit/d074e8b482582e25f80d8a14b4735e69a2b7821e), [`7fd13f6`](https://github.com/usehenri/henri/commit/7fd13f631b75f7aa152b73046b50c6902ae3ca93), [`b559fb7`](https://github.com/usehenri/henri/commit/b559fb72b391eeb21a3f6a0cda1515e01ecbfafc), [`1c0dfe8`](https://github.com/usehenri/henri/commit/1c0dfe84a98eff2122512256c4f42ec7ccde4212), [`93060a8`](https://github.com/usehenri/henri/commit/93060a86df795dbbd99bf1895beb0cf14c4b86de), [`e031900`](https://github.com/usehenri/henri/commit/e031900082f28aec72af4cda9cd959f932e2ebc7), [`9173000`](https://github.com/usehenri/henri/commit/91730005efa88f073bfaaf67078c3ec0e137b459), [`62fac46`](https://github.com/usehenri/henri/commit/62fac46fd6cae5581979b73daf99700fd246e0ea), [`b7f33e2`](https://github.com/usehenri/henri/commit/b7f33e28a5e4844391befd75d08e42c4cf6212ed), [`3d6f3fc`](https://github.com/usehenri/henri/commit/3d6f3fc048d05db41be86069608d342e437408cb), [`b278119`](https://github.com/usehenri/henri/commit/b2781190de436eb5838e866446c9a0c8210bb6ca), [`b161e1b`](https://github.com/usehenri/henri/commit/b161e1b8fad94af2d2afc351dc1bc07dabbb1379), [`7cb0b04`](https://github.com/usehenri/henri/commit/7cb0b04b29b61dedaa82fcd1972646fb3765acfc), [`1616e34`](https://github.com/usehenri/henri/commit/1616e343a612be2bffffcfa5b23bfa8ad191bbe3), [`c8f5367`](https://github.com/usehenri/henri/commit/c8f53678b33341d086b467f801e959314afc7860), [`bcf4ce2`](https://github.com/usehenri/henri/commit/bcf4ce22bcd294844504164fac1fa4aef1ffec41), [`ab5a8e4`](https://github.com/usehenri/henri/commit/ab5a8e4a88c80cca070a2b8ad398c80babdaff11), [`43d267f`](https://github.com/usehenri/henri/commit/43d267f0f9d192b2c01e89c3925b7daf5000041b), [`9f868f3`](https://github.com/usehenri/henri/commit/9f868f3d9162fa218e34304110210e6949f97d5c), [`d88bf7f`](https://github.com/usehenri/henri/commit/d88bf7fe038a6b58e7bed02ff4c90755f6c0e65e), [`49398a6`](https://github.com/usehenri/henri/commit/49398a6308f0760f01c6ff2ec98aaa35f484474d), [`89dda62`](https://github.com/usehenri/henri/commit/89dda62da456a0a55600e79cfb65ce89f11258e2), [`62fac46`](https://github.com/usehenri/henri/commit/62fac46fd6cae5581979b73daf99700fd246e0ea), [`a93d6cc`](https://github.com/usehenri/henri/commit/a93d6cc39b33b261089e91f3e757b54fefc9fe15), [`d9f3be4`](https://github.com/usehenri/henri/commit/d9f3be49c5929d929a220220bf6e72fdcb135595), [`c44f025`](https://github.com/usehenri/henri/commit/c44f025acec3d5bbbb57e2310d02184a1053a10d), [`67cfb20`](https://github.com/usehenri/henri/commit/67cfb200ea0e0b31bacf2af183db6467b0fa011d), [`e661f98`](https://github.com/usehenri/henri/commit/e661f98fe8f8acce15aa10ce2dc320c5a2cb006f), [`43a0e1a`](https://github.com/usehenri/henri/commit/43a0e1a6a320baa43298391e1c1e0334d6cd28d5), [`cee57b9`](https://github.com/usehenri/henri/commit/cee57b9d3521a4a70c715222eae1f18ff4a6c128), [`61bf75c`](https://github.com/usehenri/henri/commit/61bf75cbaccda1aecff34408a175b2d85447d7a8), [`2c8a826`](https://github.com/usehenri/henri/commit/2c8a8265262dbf6ea5c3e73e8e7892a230d4d0f0), [`46d5dbc`](https://github.com/usehenri/henri/commit/46d5dbcc983c03e96ae5a87d7288c5d8a5adbc24), [`ba97ea9`](https://github.com/usehenri/henri/commit/ba97ea968f0b34cd67b7a3e803ecd34543b8aaaf), [`5ccd537`](https://github.com/usehenri/henri/commit/5ccd537b3621b54d11e7f24ccca39643ae7d5cf7), [`d88bf7f`](https://github.com/usehenri/henri/commit/d88bf7fe038a6b58e7bed02ff4c90755f6c0e65e), [`9895cbf`](https://github.com/usehenri/henri/commit/9895cbf4be85b476e341a5be915e3049e5a027de), [`61bf75c`](https://github.com/usehenri/henri/commit/61bf75cbaccda1aecff34408a175b2d85447d7a8), [`5a150d5`](https://github.com/usehenri/henri/commit/5a150d576208571c32b9cd12827d035e31ed4313), [`3c1c5b8`](https://github.com/usehenri/henri/commit/3c1c5b83ea135b000ddd6ffbfd05457b000f2f7c), [`2625067`](https://github.com/usehenri/henri/commit/26250673d91ab70ad024739d02b647754f75267d), [`aa3f90b`](https://github.com/usehenri/henri/commit/aa3f90bd6f42bb05431c33f3e4bf2202cb6bb7c6), [`a1c6769`](https://github.com/usehenri/henri/commit/a1c6769099e3dc28b22b5338a2a57b13bdf69f7a), [`0a8bb41`](https://github.com/usehenri/henri/commit/0a8bb415d352cd75b12d07e591c8ec7c16774a99), [`ec1c8c4`](https://github.com/usehenri/henri/commit/ec1c8c419f4d9063a7617472b2970fdb8a929fa1), [`dd2731d`](https://github.com/usehenri/henri/commit/dd2731d6a20fd96aa1be1aeb5e6ec0155001326b), [`ab52e18`](https://github.com/usehenri/henri/commit/ab52e187c420dfe381f03ed51c5c141fda525acb), [`b7b56e1`](https://github.com/usehenri/henri/commit/b7b56e190ae774abc0096fe2aebaf91f823115af), [`b7038ce`](https://github.com/usehenri/henri/commit/b7038ceaa430f4a0b9eaf7e983fc2844421bf636), [`1ea0f85`](https://github.com/usehenri/henri/commit/1ea0f85066b86fba31f58937cc10abb6359e6a26), [`762062a`](https://github.com/usehenri/henri/commit/762062aadc450d49b1a2d15524f9d579ab4f60e7), [`01a561a`](https://github.com/usehenri/henri/commit/01a561aa58650ec15df1c2659795a5e4c5bbfd53), [`bd1b630`](https://github.com/usehenri/henri/commit/bd1b63083b3817b8c47b8a187de76027458a1b32), [`27b5513`](https://github.com/usehenri/henri/commit/27b5513d1cae8aba734fe27da0ace2a82423e2f4), [`e31a3f7`](https://github.com/usehenri/henri/commit/e31a3f73e7e8facf3cedf7460f115e57995f32c3), [`2689779`](https://github.com/usehenri/henri/commit/26897798b840fd28a4bc091c050a83457b36905d), [`72cd1d3`](https://github.com/usehenri/henri/commit/72cd1d35ffb99311bbca815c1f6ab41ee3682f64), [`a2e1ec2`](https://github.com/usehenri/henri/commit/a2e1ec29df52462f12ebaae9bfbc1ad4f427b27f), [`afead74`](https://github.com/usehenri/henri/commit/afead7489498ed42e1893a25123ea772cac2ca09), [`a4ecba5`](https://github.com/usehenri/henri/commit/a4ecba50c663f4d5c741adbb6cd9bc0eefe0e5cc), [`baec3fd`](https://github.com/usehenri/henri/commit/baec3fd22be92bf8ffbaeb251b0b6c2771f8347a), [`1103628`](https://github.com/usehenri/henri/commit/110362808f8ec6d73a75ff7fc89a77f3e943d773), [`e865d94`](https://github.com/usehenri/henri/commit/e865d945d65419ac676f5a2fe3ba3b6114a1e53d), [`4274567`](https://github.com/usehenri/henri/commit/4274567e20a980657f07df9ec7db25296c7d55f5), [`aea429c`](https://github.com/usehenri/henri/commit/aea429ca99338a62370ab3e3d94bdc6b8c227601), [`8a8e3b3`](https://github.com/usehenri/henri/commit/8a8e3b33d7967b81f66633aa25c3075318f01d60), [`18715f9`](https://github.com/usehenri/henri/commit/18715f90ea8958dc57da1bb029b8209c36b84cc6), [`0d2ebc3`](https://github.com/usehenri/henri/commit/0d2ebc344bfcd80533ef900638083e4c105407bb), [`49398a6`](https://github.com/usehenri/henri/commit/49398a6308f0760f01c6ff2ec98aaa35f484474d), [`ec64e44`](https://github.com/usehenri/henri/commit/ec64e44e7b79a02da5fc587a72a9a6c900836982), [`831aa5c`](https://github.com/usehenri/henri/commit/831aa5c011f3432630c68b8d26755d2582f82f74), [`16824e8`](https://github.com/usehenri/henri/commit/16824e8fe9ccc6a04dab5d9b2481c29ff4f6b64b), [`fda9366`](https://github.com/usehenri/henri/commit/fda9366e9ed2b072764a995c5aa60205ca7a4725), [`1a86acb`](https://github.com/usehenri/henri/commit/1a86acbf15e4a43e5fb81277bb22e101c06e77a4), [`808d824`](https://github.com/usehenri/henri/commit/808d82471d59e64ccc735f617bab293eb572c46b), [`0b32fbd`](https://github.com/usehenri/henri/commit/0b32fbde19c95da8fe07fab76933840a4242c71c), [`c0c16e8`](https://github.com/usehenri/henri/commit/c0c16e873ba440aee9832160553cbb12ab81bd2c), [`41470bf`](https://github.com/usehenri/henri/commit/41470bf378d83ca3d35d00e8c31796fea5eb15e0), [`8e44e7e`](https://github.com/usehenri/henri/commit/8e44e7e882dd8741b3ac632651b453389d76bf2c), [`de1c1e0`](https://github.com/usehenri/henri/commit/de1c1e02ed83d13dcfeb8e44012f309eb663f03e), [`4b4677d`](https://github.com/usehenri/henri/commit/4b4677d4a09d39fe50b1fa4af577600342578daf)]:
|
|
117
|
+
- @usehenri/core@1.2.0
|
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2016-present, Félix-Antoine Paradis
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,10 @@
|
|
|
1
|
+
<!-- generated by scripts/prepublish.js -->
|
|
2
|
+
|
|
1
3
|
# @usehenri/uploads
|
|
2
4
|
|
|
3
|
-
|
|
5
|
+
Henri file uploads: bounded multipart parsing, files typed by their bytes and a storage seam.
|
|
6
|
+
|
|
7
|
+
Part of [henri](https://usehenri.io), the Rails-like React framework for
|
|
8
|
+
Node.js: [documentation](https://usehenri.io),
|
|
9
|
+
[source and issues](https://github.com/usehenri/henri),
|
|
10
|
+
[changelog](https://github.com/usehenri/henri/blob/master/packages/uploads/CHANGELOG.md).
|
package/index.js
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* `@usehenri/uploads`: bounded multipart parsing, files typed by their bytes,
|
|
3
|
+
* and a storage seam.
|
|
4
|
+
*
|
|
5
|
+
* An application installs the package and gets the module (`henri.uploads`),
|
|
6
|
+
* `req.files`, `req.file()` and `req.permitFiles()`. What is exported here is
|
|
7
|
+
* for the two things an application does beyond that: writing a storage of
|
|
8
|
+
* its own against `HenriStorage`, and reusing the name and type helpers in a
|
|
9
|
+
* validation of its own.
|
|
10
|
+
*
|
|
11
|
+
* See https://usehenri.io/guides/uploads/
|
|
12
|
+
*/
|
|
13
|
+
const UploadsModule = require('./src/module');
|
|
14
|
+
const LocalStorage = require('./src/storage/local');
|
|
15
|
+
|
|
16
|
+
const { DEFAULTS, settings } = require('./src/config');
|
|
17
|
+
const { UploadError, coded } = require('./src/errors');
|
|
18
|
+
const { UploadedFile } = require('./src/file');
|
|
19
|
+
const { UrlSigner } = require('./src/signing');
|
|
20
|
+
const { bytes, format } = require('./src/bytes');
|
|
21
|
+
const { contentDisposition, isKey, keyFor, safeName } = require('./src/names');
|
|
22
|
+
const { createStorage } = require('./src/storage');
|
|
23
|
+
const { extensionFor, sniff } = require('./src/sniff');
|
|
24
|
+
|
|
25
|
+
module.exports = UploadsModule;
|
|
26
|
+
module.exports.DEFAULTS = DEFAULTS;
|
|
27
|
+
module.exports.LocalStorage = LocalStorage;
|
|
28
|
+
module.exports.UploadError = UploadError;
|
|
29
|
+
module.exports.UploadedFile = UploadedFile;
|
|
30
|
+
module.exports.UploadsModule = UploadsModule;
|
|
31
|
+
module.exports.UrlSigner = UrlSigner;
|
|
32
|
+
module.exports.bytes = bytes;
|
|
33
|
+
module.exports.coded = coded;
|
|
34
|
+
module.exports.contentDisposition = contentDisposition;
|
|
35
|
+
module.exports.createStorage = createStorage;
|
|
36
|
+
module.exports.extensionFor = extensionFor;
|
|
37
|
+
module.exports.format = format;
|
|
38
|
+
module.exports.isKey = isKey;
|
|
39
|
+
module.exports.keyFor = keyFor;
|
|
40
|
+
module.exports.safeName = safeName;
|
|
41
|
+
module.exports.settings = settings;
|
|
42
|
+
module.exports.sniff = sniff;
|
package/module.js
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The henri module this package ships.
|
|
3
|
+
*
|
|
4
|
+
* `package.json` points at this file with `"henri": { "module": "./module.js" }`,
|
|
5
|
+
* which is all core reads: an application depending on `@usehenri/uploads`
|
|
6
|
+
* has the module in its boot, as `henri.uploads`.
|
|
7
|
+
*/
|
|
8
|
+
module.exports = require('./src/module');
|
package/package.json
CHANGED
|
@@ -1,15 +1,61 @@
|
|
|
1
1
|
{
|
|
2
|
-
"description": "Placeholder creating @usehenri/uploads on npm; the first real version is published by the henri release workflow",
|
|
3
|
-
"homepage": "https://usehenri.io",
|
|
4
|
-
"license": "MIT",
|
|
5
2
|
"name": "@usehenri/uploads",
|
|
6
|
-
"
|
|
7
|
-
|
|
8
|
-
|
|
3
|
+
"version": "1.2.0",
|
|
4
|
+
"description": "henri file uploads: bounded multipart parsing, files typed by their bytes and a storage seam",
|
|
5
|
+
"license": "MIT",
|
|
6
|
+
"author": "Felix-Antoine Paradis",
|
|
7
|
+
"homepage": "https://usehenri.io",
|
|
9
8
|
"repository": {
|
|
10
|
-
"directory": "packages/uploads",
|
|
11
9
|
"type": "git",
|
|
12
|
-
"url": "git+https://github.com/usehenri/henri.git"
|
|
10
|
+
"url": "git+https://github.com/usehenri/henri.git",
|
|
11
|
+
"directory": "packages/uploads"
|
|
12
|
+
},
|
|
13
|
+
"bugs": {
|
|
14
|
+
"url": "https://github.com/usehenri/henri/issues"
|
|
15
|
+
},
|
|
16
|
+
"keywords": [
|
|
17
|
+
"henri",
|
|
18
|
+
"uploads",
|
|
19
|
+
"multipart",
|
|
20
|
+
"files",
|
|
21
|
+
"busboy",
|
|
22
|
+
"storage",
|
|
23
|
+
"variants"
|
|
24
|
+
],
|
|
25
|
+
"main": "index.js",
|
|
26
|
+
"henri": {
|
|
27
|
+
"module": "./module.js"
|
|
28
|
+
},
|
|
29
|
+
"files": [
|
|
30
|
+
"index.js",
|
|
31
|
+
"module.js",
|
|
32
|
+
"src",
|
|
33
|
+
"CHANGELOG.md"
|
|
34
|
+
],
|
|
35
|
+
"publishConfig": {
|
|
36
|
+
"access": "public",
|
|
37
|
+
"provenance": true
|
|
38
|
+
},
|
|
39
|
+
"engines": {
|
|
40
|
+
"node": ">=22"
|
|
41
|
+
},
|
|
42
|
+
"dependencies": {
|
|
43
|
+
"busboy": "^1.6.0",
|
|
44
|
+
"debug": "^4.4.3"
|
|
45
|
+
},
|
|
46
|
+
"peerDependencies": {
|
|
47
|
+
"@usehenri/core": "^1.2.0",
|
|
48
|
+
"sharp": ">=0.33"
|
|
49
|
+
},
|
|
50
|
+
"peerDependenciesMeta": {
|
|
51
|
+
"sharp": {
|
|
52
|
+
"optional": true
|
|
53
|
+
}
|
|
13
54
|
},
|
|
14
|
-
"
|
|
15
|
-
|
|
55
|
+
"devDependencies": {
|
|
56
|
+
"@usehenri/core": "^1.2.0",
|
|
57
|
+
"express": "^5.2.1",
|
|
58
|
+
"sharp": "^0.35.4",
|
|
59
|
+
"supertest": "^7.2.2"
|
|
60
|
+
}
|
|
61
|
+
}
|
package/src/bytes.js
ADDED
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Sizes, the way the configuration writes them.
|
|
3
|
+
*
|
|
4
|
+
* `config.bodyLimit` is `"1mb"` or a number of bytes, and the upload limits
|
|
5
|
+
* are read the same way so that the two are comparable at a glance. This is
|
|
6
|
+
* the parser: no dependency, no unit henri does not document, and `false`
|
|
7
|
+
* kept as `false` because a limit an application removed on purpose is not
|
|
8
|
+
* the same thing as a limit it wrote badly.
|
|
9
|
+
*/
|
|
10
|
+
|
|
11
|
+
/** Multipliers of the suffixes a size accepts (no suffix is bytes) */
|
|
12
|
+
const UNITS = { gb: 1024 * 1024 * 1024, kb: 1024, mb: 1024 * 1024 };
|
|
13
|
+
|
|
14
|
+
/** A size: a number, with an optional unit */
|
|
15
|
+
/**
|
|
16
|
+
* The written form: an amount, then a unit.
|
|
17
|
+
*
|
|
18
|
+
* The surrounding whitespace is trimmed before this runs rather than
|
|
19
|
+
* matched by it: `^\s*…\s*$` around an optional group is quadratic, and
|
|
20
|
+
* `'9' + ' '.repeat(100000) + '!'` took five seconds to refuse when the two
|
|
21
|
+
* star quantifiers had a run of spaces to share between them.
|
|
22
|
+
*/
|
|
23
|
+
const SIZE = /^(\d+(?:\.\d+)?)\s*(b|kb|mb|gb)?$/iu;
|
|
24
|
+
|
|
25
|
+
/**
|
|
26
|
+
* A size in bytes
|
|
27
|
+
*
|
|
28
|
+
* @param {*} value a number of bytes, a string (`'10mb'`), or `false`
|
|
29
|
+
* @param {(number|false|null)} [fallback=null] what an unreadable value becomes
|
|
30
|
+
* @returns {(number|false|null)} the size, `false` for no limit, or the fallback
|
|
31
|
+
*/
|
|
32
|
+
function bytes(value, fallback = null) {
|
|
33
|
+
if (value === false) {
|
|
34
|
+
return false;
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
if (typeof value === 'number') {
|
|
38
|
+
return Number.isFinite(value) && value > 0 ? Math.floor(value) : fallback;
|
|
39
|
+
}
|
|
40
|
+
|
|
41
|
+
if (typeof value !== 'string') {
|
|
42
|
+
return fallback;
|
|
43
|
+
}
|
|
44
|
+
|
|
45
|
+
const match = SIZE.exec(value.trim());
|
|
46
|
+
|
|
47
|
+
if (!match) {
|
|
48
|
+
return fallback;
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
const unit = String(match[2] || '').toLowerCase();
|
|
52
|
+
const size = Number(match[1]) * (UNITS[unit] || 1);
|
|
53
|
+
|
|
54
|
+
return size > 0 ? Math.floor(size) : fallback;
|
|
55
|
+
}
|
|
56
|
+
|
|
57
|
+
/**
|
|
58
|
+
* A size, printed the way the documentation writes it
|
|
59
|
+
*
|
|
60
|
+
* @param {(number|false)} value a size in bytes, or `false`
|
|
61
|
+
* @returns {string} `'10mb'`, `'512kb'`, `'900b'` or `'no limit'`
|
|
62
|
+
*/
|
|
63
|
+
function format(value) {
|
|
64
|
+
if (value === false || value === null) {
|
|
65
|
+
return 'no limit';
|
|
66
|
+
}
|
|
67
|
+
|
|
68
|
+
for (const unit of ['gb', 'mb', 'kb']) {
|
|
69
|
+
if (value >= UNITS[unit]) {
|
|
70
|
+
return `${Math.round((value / UNITS[unit]) * 10) / 10}${unit}`;
|
|
71
|
+
}
|
|
72
|
+
}
|
|
73
|
+
|
|
74
|
+
return `${value}b`;
|
|
75
|
+
}
|
|
76
|
+
|
|
77
|
+
/**
|
|
78
|
+
* A limit as busboy wants it: a number, or `Infinity` for no limit
|
|
79
|
+
*
|
|
80
|
+
* @param {(number|false|null)} value the limit
|
|
81
|
+
* @returns {number} the limit, or `Infinity`
|
|
82
|
+
*/
|
|
83
|
+
const orInfinity = (value) =>
|
|
84
|
+
typeof value === 'number' && value > 0 ? value : Infinity;
|
|
85
|
+
|
|
86
|
+
module.exports = { UNITS, bytes, format, orInfinity };
|
package/src/config.js
ADDED
|
@@ -0,0 +1,243 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* `config.uploads`, normalized.
|
|
3
|
+
*
|
|
4
|
+
* Every limit here is enforced by the parser as it reads, which is the only
|
|
5
|
+
* kind of limit worth writing: a check that runs once the file is on disk
|
|
6
|
+
* has already let the disk fill. `false` is accepted on the four that bound
|
|
7
|
+
* a request, because an application that means "no limit" should have to
|
|
8
|
+
* write it -- and `henri audit` reports it when it does.
|
|
9
|
+
*
|
|
10
|
+
* `maxFieldSize` defaults to `config.bodyLimit` rather than to a number of
|
|
11
|
+
* its own. The two bound the same thing from two directions: `bodyLimit`
|
|
12
|
+
* is what `express.json()` and `express.urlencoded()` accept for a whole
|
|
13
|
+
* body, and a multipart body is bounded by `maxTotalSize` instead, with
|
|
14
|
+
* `maxFieldSize` bounding one of its non-file parts. Giving the part the
|
|
15
|
+
* same ceiling as a whole urlencoded body keeps `"name"` costing the same
|
|
16
|
+
* whichever encoding a form was posted with.
|
|
17
|
+
*/
|
|
18
|
+
const { bytes } = require('./bytes');
|
|
19
|
+
const { EXPIRES_IN, MAX_EXPIRES, PATH } = require('./signing');
|
|
20
|
+
const { variantsOf } = require('./variants');
|
|
21
|
+
|
|
22
|
+
/** The defaults, and the table the documentation prints */
|
|
23
|
+
const DEFAULTS = {
|
|
24
|
+
allow: null,
|
|
25
|
+
maxFieldNameSize: 100,
|
|
26
|
+
maxFieldSize: null,
|
|
27
|
+
maxFields: 100,
|
|
28
|
+
maxFileSize: '10mb',
|
|
29
|
+
maxFilenameLength: 255,
|
|
30
|
+
maxFiles: 10,
|
|
31
|
+
maxTotalSize: '25mb',
|
|
32
|
+
paths: null,
|
|
33
|
+
root: 'storage/uploads',
|
|
34
|
+
sniff: true,
|
|
35
|
+
storage: 'local',
|
|
36
|
+
urls: false,
|
|
37
|
+
variants: null,
|
|
38
|
+
};
|
|
39
|
+
|
|
40
|
+
/** The methods a body is read from; anything else never carries an upload */
|
|
41
|
+
const METHODS = new Set(['PATCH', 'POST', 'PUT']);
|
|
42
|
+
|
|
43
|
+
/**
|
|
44
|
+
* The storage an application named, split into the backend and its settings.
|
|
45
|
+
*
|
|
46
|
+
* Two forms, the way `config.stores.<name>` and `config.shared` already
|
|
47
|
+
* work: a string is a backend with nothing to configure (`"local"`,
|
|
48
|
+
* `"./lib/storage"`), and an object is a backend with settings, whose
|
|
49
|
+
* `adapter` names it and whose other keys are the backend's own. henri
|
|
50
|
+
* reads none of the latter -- a bucket, a region and an endpoint are
|
|
51
|
+
* `@usehenri/s3`'s business, not the framework's.
|
|
52
|
+
*
|
|
53
|
+
* @param {*} value what the configuration holds under `storage`
|
|
54
|
+
* @returns {{name: string, options: object}} the backend and its settings
|
|
55
|
+
*/
|
|
56
|
+
const storageOf = (value) => {
|
|
57
|
+
if (typeof value === 'string' && value.trim().length > 0) {
|
|
58
|
+
return { name: value.trim(), options: {} };
|
|
59
|
+
}
|
|
60
|
+
|
|
61
|
+
if (value && typeof value === 'object' && !Array.isArray(value)) {
|
|
62
|
+
const { adapter, ...options } = value;
|
|
63
|
+
|
|
64
|
+
return {
|
|
65
|
+
name:
|
|
66
|
+
typeof adapter === 'string' && adapter.trim()
|
|
67
|
+
? adapter.trim()
|
|
68
|
+
: DEFAULTS.storage,
|
|
69
|
+
options,
|
|
70
|
+
};
|
|
71
|
+
}
|
|
72
|
+
|
|
73
|
+
return { name: DEFAULTS.storage, options: {} };
|
|
74
|
+
};
|
|
75
|
+
|
|
76
|
+
/**
|
|
77
|
+
* A whole number above zero, `false`, or the fallback
|
|
78
|
+
*
|
|
79
|
+
* @param {*} value the value
|
|
80
|
+
* @param {(number|false)} fallback what an unreadable value becomes
|
|
81
|
+
* @returns {(number|false)} the count
|
|
82
|
+
*/
|
|
83
|
+
const count = (value, fallback) => {
|
|
84
|
+
if (value === false) {
|
|
85
|
+
return false;
|
|
86
|
+
}
|
|
87
|
+
|
|
88
|
+
const number = Number(value);
|
|
89
|
+
|
|
90
|
+
return Number.isInteger(number) && number > 0 ? number : fallback;
|
|
91
|
+
};
|
|
92
|
+
|
|
93
|
+
/**
|
|
94
|
+
* A plain object, or an empty one
|
|
95
|
+
*
|
|
96
|
+
* @param {*} value the value
|
|
97
|
+
* @returns {object} a plain object
|
|
98
|
+
*/
|
|
99
|
+
const objectOf = (value) =>
|
|
100
|
+
value && typeof value === 'object' && !Array.isArray(value) ? value : {};
|
|
101
|
+
|
|
102
|
+
/**
|
|
103
|
+
* The list of media types an application accepts, or null for every type
|
|
104
|
+
*
|
|
105
|
+
* @param {*} value what the configuration holds
|
|
106
|
+
* @returns {?Array<string>} the list, or null
|
|
107
|
+
*/
|
|
108
|
+
const allowList = (value) => {
|
|
109
|
+
if (!Array.isArray(value)) {
|
|
110
|
+
return null;
|
|
111
|
+
}
|
|
112
|
+
|
|
113
|
+
const entries = value
|
|
114
|
+
.filter((entry) => typeof entry === 'string' && entry.trim().length > 0)
|
|
115
|
+
.map((entry) => entry.trim().toLowerCase());
|
|
116
|
+
|
|
117
|
+
return entries.length > 0 ? entries : null;
|
|
118
|
+
};
|
|
119
|
+
|
|
120
|
+
/**
|
|
121
|
+
* The path prefixes multipart is parsed under, or null for every path
|
|
122
|
+
*
|
|
123
|
+
* @param {*} value what the configuration holds
|
|
124
|
+
* @returns {?Array<string>} the prefixes, or null
|
|
125
|
+
*/
|
|
126
|
+
const pathList = (value) => {
|
|
127
|
+
const entries = (Array.isArray(value) ? value : [value])
|
|
128
|
+
.filter((entry) => typeof entry === 'string' && entry.startsWith('/'))
|
|
129
|
+
.map((entry) => (entry.length > 1 ? entry.replace(/\/+$/u, '') : entry));
|
|
130
|
+
|
|
131
|
+
return entries.length > 0 ? entries : null;
|
|
132
|
+
};
|
|
133
|
+
|
|
134
|
+
/**
|
|
135
|
+
* Signed urls: off, or the settings of the ones this application hands out.
|
|
136
|
+
*
|
|
137
|
+
* Off is the default, and it is fail-closed rather than shy. A signed url
|
|
138
|
+
* is a bearer capability -- whoever holds the link holds the file, until it
|
|
139
|
+
* expires -- and on the local disk it also mounts a route that serves bytes
|
|
140
|
+
* without asking the application anything. Neither is a thing to acquire by
|
|
141
|
+
* installing a package; both are one line of configuration.
|
|
142
|
+
*
|
|
143
|
+
* @param {*} value what the configuration holds under `urls`
|
|
144
|
+
* @returns {(false|object)} the settings, or false
|
|
145
|
+
*/
|
|
146
|
+
const urlsOf = (value) => {
|
|
147
|
+
if (!value || typeof value !== 'object' || Array.isArray(value)) {
|
|
148
|
+
return false;
|
|
149
|
+
}
|
|
150
|
+
|
|
151
|
+
const seconds = Number(value.expiresIn);
|
|
152
|
+
const declared = typeof value.path === 'string' && value.path.startsWith('/');
|
|
153
|
+
|
|
154
|
+
return {
|
|
155
|
+
cdn: typeof value.cdn === 'string' ? value.cdn.replace(/\/+$/u, '') : '',
|
|
156
|
+
expiresIn:
|
|
157
|
+
Number.isInteger(seconds) && seconds > 0 && seconds <= MAX_EXPIRES
|
|
158
|
+
? seconds
|
|
159
|
+
: EXPIRES_IN,
|
|
160
|
+
path: declared ? value.path.replace(/\/+$/u, '') || PATH : PATH,
|
|
161
|
+
};
|
|
162
|
+
};
|
|
163
|
+
|
|
164
|
+
/**
|
|
165
|
+
* The settings of the uploads module
|
|
166
|
+
*
|
|
167
|
+
* @param {object} config henri's config module (anything with get/has)
|
|
168
|
+
* @returns {object} the settings, or `{ enabled: false }` when the
|
|
169
|
+
* application wrote `"uploads": false`
|
|
170
|
+
*/
|
|
171
|
+
function settings(config) {
|
|
172
|
+
const has = (key) =>
|
|
173
|
+
Boolean(config) && typeof config.has === 'function' && config.has(key);
|
|
174
|
+
const get = (key, fallback) => (has(key) ? config.get(key) : fallback);
|
|
175
|
+
const declared = get('uploads', {});
|
|
176
|
+
|
|
177
|
+
if (declared === false) {
|
|
178
|
+
return { enabled: false };
|
|
179
|
+
}
|
|
180
|
+
|
|
181
|
+
const uploads = objectOf(declared);
|
|
182
|
+
const bodyLimit = bytes(get('bodyLimit', '1mb'), 1024 * 1024);
|
|
183
|
+
const storage = storageOf(uploads.storage);
|
|
184
|
+
|
|
185
|
+
return {
|
|
186
|
+
allow: allowList(uploads.allow),
|
|
187
|
+
enabled: true,
|
|
188
|
+
maxFieldNameSize: count(
|
|
189
|
+
uploads.maxFieldNameSize,
|
|
190
|
+
DEFAULTS.maxFieldNameSize
|
|
191
|
+
),
|
|
192
|
+
maxFieldSize: bytes(uploads.maxFieldSize, bodyLimit),
|
|
193
|
+
maxFields: count(uploads.maxFields, DEFAULTS.maxFields),
|
|
194
|
+
maxFileSize: bytes(uploads.maxFileSize, bytes(DEFAULTS.maxFileSize)),
|
|
195
|
+
maxFilenameLength: count(
|
|
196
|
+
uploads.maxFilenameLength,
|
|
197
|
+
DEFAULTS.maxFilenameLength
|
|
198
|
+
),
|
|
199
|
+
maxFiles: count(uploads.maxFiles, DEFAULTS.maxFiles),
|
|
200
|
+
maxTotalSize: bytes(uploads.maxTotalSize, bytes(DEFAULTS.maxTotalSize)),
|
|
201
|
+
paths: pathList(uploads.paths),
|
|
202
|
+
root: typeof uploads.root === 'string' ? uploads.root : DEFAULTS.root,
|
|
203
|
+
sniff: uploads.sniff !== false,
|
|
204
|
+
storage: storage.name,
|
|
205
|
+
storageOptions: storage.options,
|
|
206
|
+
urls: urlsOf(uploads.urls),
|
|
207
|
+
variants: variantsOf(uploads.variants),
|
|
208
|
+
};
|
|
209
|
+
}
|
|
210
|
+
|
|
211
|
+
/**
|
|
212
|
+
* Does this request take a body a multipart parser should read?
|
|
213
|
+
*
|
|
214
|
+
* @param {Express.Request} req the request
|
|
215
|
+
* @param {?Array<string>} paths the configured prefixes, or null
|
|
216
|
+
* @returns {boolean} true when the parser runs for it
|
|
217
|
+
*/
|
|
218
|
+
function covers(req, paths) {
|
|
219
|
+
if (!METHODS.has(req.method)) {
|
|
220
|
+
return false;
|
|
221
|
+
}
|
|
222
|
+
|
|
223
|
+
if (!Array.isArray(paths)) {
|
|
224
|
+
return true;
|
|
225
|
+
}
|
|
226
|
+
|
|
227
|
+
const url = (req.path || req.url || '/').split('?')[0];
|
|
228
|
+
|
|
229
|
+
return paths.some(
|
|
230
|
+
(prefix) => url === prefix || url.startsWith(`${prefix}/`) || prefix === '/'
|
|
231
|
+
);
|
|
232
|
+
}
|
|
233
|
+
|
|
234
|
+
module.exports = {
|
|
235
|
+
DEFAULTS,
|
|
236
|
+
METHODS,
|
|
237
|
+
allowList,
|
|
238
|
+
count,
|
|
239
|
+
covers,
|
|
240
|
+
settings,
|
|
241
|
+
storageOf,
|
|
242
|
+
urlsOf,
|
|
243
|
+
};
|