@helia/verified-fetch 0.0.0 → 1.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (81) hide show
  1. package/README.md +353 -56
  2. package/dist/index.min.js +7 -29
  3. package/dist/src/index.d.ts +384 -69
  4. package/dist/src/index.d.ts.map +1 -1
  5. package/dist/src/index.js +345 -77
  6. package/dist/src/index.js.map +1 -1
  7. package/dist/src/singleton.d.ts +3 -0
  8. package/dist/src/singleton.d.ts.map +1 -0
  9. package/dist/src/singleton.js +15 -0
  10. package/dist/src/singleton.js.map +1 -0
  11. package/dist/src/types.d.ts +2 -0
  12. package/dist/src/types.d.ts.map +1 -0
  13. package/dist/src/types.js +2 -0
  14. package/dist/src/types.js.map +1 -0
  15. package/dist/src/utils/dag-cbor-to-safe-json.d.ts +7 -0
  16. package/dist/src/utils/dag-cbor-to-safe-json.d.ts.map +1 -0
  17. package/dist/src/utils/dag-cbor-to-safe-json.js +25 -0
  18. package/dist/src/utils/dag-cbor-to-safe-json.js.map +1 -0
  19. package/dist/src/utils/get-content-disposition-filename.d.ts +6 -0
  20. package/dist/src/utils/get-content-disposition-filename.d.ts.map +1 -0
  21. package/dist/src/utils/get-content-disposition-filename.js +16 -0
  22. package/dist/src/utils/get-content-disposition-filename.js.map +1 -0
  23. package/dist/src/utils/get-e-tag.d.ts +28 -0
  24. package/dist/src/utils/get-e-tag.d.ts.map +1 -0
  25. package/dist/src/utils/get-e-tag.js +18 -0
  26. package/dist/src/utils/get-e-tag.js.map +1 -0
  27. package/dist/src/utils/get-stream-from-async-iterable.d.ts +10 -0
  28. package/dist/src/utils/get-stream-from-async-iterable.d.ts.map +1 -0
  29. package/dist/src/utils/{get-stream-and-content-type.js → get-stream-from-async-iterable.js} +11 -11
  30. package/dist/src/utils/get-stream-from-async-iterable.js.map +1 -0
  31. package/dist/src/utils/get-tar-stream.d.ts +4 -0
  32. package/dist/src/utils/get-tar-stream.d.ts.map +1 -0
  33. package/dist/src/utils/get-tar-stream.js +46 -0
  34. package/dist/src/utils/get-tar-stream.js.map +1 -0
  35. package/dist/src/utils/parse-resource.d.ts +6 -1
  36. package/dist/src/utils/parse-resource.d.ts.map +1 -1
  37. package/dist/src/utils/parse-resource.js +2 -2
  38. package/dist/src/utils/parse-resource.js.map +1 -1
  39. package/dist/src/utils/parse-url-string.d.ts +10 -3
  40. package/dist/src/utils/parse-url-string.d.ts.map +1 -1
  41. package/dist/src/utils/parse-url-string.js +8 -4
  42. package/dist/src/utils/parse-url-string.js.map +1 -1
  43. package/dist/src/utils/responses.d.ts +5 -0
  44. package/dist/src/utils/responses.d.ts.map +1 -0
  45. package/dist/src/utils/responses.js +27 -0
  46. package/dist/src/utils/responses.js.map +1 -0
  47. package/dist/src/utils/select-output-type.d.ts +12 -0
  48. package/dist/src/utils/select-output-type.d.ts.map +1 -0
  49. package/dist/src/utils/select-output-type.js +148 -0
  50. package/dist/src/utils/select-output-type.js.map +1 -0
  51. package/dist/src/utils/walk-path.d.ts +2 -1
  52. package/dist/src/utils/walk-path.d.ts.map +1 -1
  53. package/dist/src/utils/walk-path.js +1 -3
  54. package/dist/src/utils/walk-path.js.map +1 -1
  55. package/dist/src/verified-fetch.d.ts +24 -27
  56. package/dist/src/verified-fetch.d.ts.map +1 -1
  57. package/dist/src/verified-fetch.js +297 -150
  58. package/dist/src/verified-fetch.js.map +1 -1
  59. package/dist/typedoc-urls.json +25 -18
  60. package/package.json +58 -116
  61. package/src/index.ts +391 -72
  62. package/src/singleton.ts +20 -0
  63. package/src/types.ts +1 -0
  64. package/src/utils/dag-cbor-to-safe-json.ts +27 -0
  65. package/src/utils/get-content-disposition-filename.ts +18 -0
  66. package/src/utils/get-e-tag.ts +36 -0
  67. package/src/utils/{get-stream-and-content-type.ts → get-stream-from-async-iterable.ts} +10 -9
  68. package/src/utils/get-tar-stream.ts +68 -0
  69. package/src/utils/parse-url-string.ts +17 -3
  70. package/src/utils/responses.ts +29 -0
  71. package/src/utils/select-output-type.ts +167 -0
  72. package/src/utils/walk-path.ts +4 -5
  73. package/src/verified-fetch.ts +340 -153
  74. package/dist/src/utils/get-content-type.d.ts +0 -11
  75. package/dist/src/utils/get-content-type.d.ts.map +0 -1
  76. package/dist/src/utils/get-content-type.js +0 -43
  77. package/dist/src/utils/get-content-type.js.map +0 -1
  78. package/dist/src/utils/get-stream-and-content-type.d.ts +0 -9
  79. package/dist/src/utils/get-stream-and-content-type.d.ts.map +0 -1
  80. package/dist/src/utils/get-stream-and-content-type.js.map +0 -1
  81. package/src/utils/get-content-type.ts +0 -55
package/README.md CHANGED
@@ -4,37 +4,54 @@
4
4
  </a>
5
5
  </p>
6
6
 
7
+ # @helia/verified-fetch
8
+
7
9
  [![ipfs.tech](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](https://ipfs.tech)
8
10
  [![Discuss](https://img.shields.io/discourse/https/discuss.ipfs.tech/posts.svg?style=flat-square)](https://discuss.ipfs.tech)
9
- [![codecov](https://img.shields.io/codecov/c/github/ipfs/helia.svg?style=flat-square)](https://codecov.io/gh/ipfs/helia)
10
- [![CI](https://img.shields.io/github/actions/workflow/status/ipfs/helia/main.yml?branch=main\&style=flat-square)](https://github.com/ipfs/helia/actions/workflows/main.yml?query=branch%3Amain)
11
+ [![codecov](https://img.shields.io/codecov/c/github/ipfs/helia-verified-fetch.svg?style=flat-square)](https://codecov.io/gh/ipfs/helia-verified-fetch)
12
+ [![CI](https://img.shields.io/github/actions/workflow/status/ipfs/helia-verified-fetch/js-test-and-release.yml?branch=main\&style=flat-square)](https://github.com/ipfs/helia-verified-fetch/actions/workflows/js-test-and-release.yml?query=branch%3Amain)
11
13
 
12
- > A fetch-like API for obtaining verified & trustless IPFS content on the web.
14
+ > A fetch-like API for obtaining verified & trustless IPFS content on the web
13
15
 
14
16
  # About
15
17
 
16
- `@helia/verified-fetch` is a library that provides a fetch-like API for fetching trustless content from IPFS and verifying it.
18
+ <!--
19
+
20
+ !IMPORTANT!
21
+
22
+ Everything in this README between "# About" and "# Install" is automatically
23
+ generated and will be overwritten the next time the doc generator is run.
24
+
25
+ To make changes to this section, please update the @packageDocumentation section
26
+ of src/index.js or src/index.ts
27
+
28
+ To experiment with formatting, please run "npm run docs" from the root of this
29
+ repo and examine the changes made.
17
30
 
18
- This library should act as a replacement for the `fetch()` API for fetching content from IPFS, and will return a [`Response`](https://developer.mozilla.org/en-US/docs/Web/API/Response) object that can be used in a similar manner to the `fetch()` API. This means browser and HTTP caching inside browser main threads, web-workers, and service workers, as well as other features of the `fetch()` API should work in a way familiar to developers.
31
+ -->
19
32
 
20
- Exports a `createVerifiedFetch` function that returns a `fetch()` like API method Helia for fetching IPFS content.
33
+ `@helia/verified-fetch` provides a [fetch](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API)-like API for retrieving content from the [IPFS](https://ipfs.tech/) network.
34
+
35
+ All content is retrieved in a [trustless manner](https://www.techopedia.com/definition/trustless), and the integrity of all bytes are verified by comparing hashes of the data. By default, CIDs are retrieved over HTTP from [trustless gateways](https://specs.ipfs.tech/http-gateways/trustless-gateway/).
36
+
37
+ This is a marked improvement over `fetch` which offers no such protections and is vulnerable to all sorts of attacks like [Content Spoofing](https://owasp.org/www-community/attacks/Content_Spoofing), [DNS Hijacking](https://en.wikipedia.org/wiki/DNS_hijacking), etc.
38
+
39
+ A `verifiedFetch` function is exported to get up and running quickly, and a `createVerifiedFetch` function is also available that allows customizing the underlying [Helia](https://helia.io/) node for complete control over how content is retrieved.
40
+
41
+ Browser-cache-friendly [Response](https://developer.mozilla.org/en-US/docs/Web/API/Response) objects are returned which should be instantly familiar to web developers.
21
42
 
22
43
  You may use any supported resource argument to fetch content:
23
44
 
24
- - CID instance
45
+ - [CID](https://multiformats.github.io/js-multiformats/classes/cid.CID.html) instance
25
46
  - IPFS URL
26
47
  - IPNS URL
27
48
 
28
- ## Example
49
+ ## Example - Getting started
29
50
 
30
51
  ```typescript
31
- import { createVerifiedFetch } from '@helia/verified-fetch'
52
+ import { verifiedFetch } from '@helia/verified-fetch'
32
53
 
33
- const fetch = await createVerifiedFetch({
34
- gateways: ['https://mygateway.example.net', 'https://trustless-gateway.link']
35
- })
36
-
37
- const resp = await fetch('ipfs://bafy...')
54
+ const resp = await verifiedFetch('ipfs://bafy...')
38
55
 
39
56
  const json = await resp.json()
40
57
  ```
@@ -42,27 +59,20 @@ const json = await resp.json()
42
59
  ## Example - Using a CID instance to fetch JSON
43
60
 
44
61
  ```typescript
45
- import { createVerifiedFetch } from '@helia/verified-fetch'
62
+ import { verifiedFetch } from '@helia/verified-fetch'
46
63
  import { CID } from 'multiformats/cid'
47
64
 
48
- const fetch = await createVerifiedFetch({
49
- gateways: ['https://mygateway.example.net', 'https://trustless-gateway.link']
50
- })
51
-
52
- const cid = CID.parse('bafyFoo') // some image file
53
- const response = await fetch(cid)
65
+ const cid = CID.parse('bafyFoo') // some json file
66
+ const response = await verifiedFetch(cid)
54
67
  const json = await response.json()
55
68
  ```
56
69
 
57
70
  ## Example - Using IPFS protocol to fetch an image
58
71
 
59
72
  ```typescript
60
- import { createVerifiedFetch } from '@helia/verified-fetch'
73
+ import { verifiedFetch } from '@helia/verified-fetch'
61
74
 
62
- const fetch = await createVerifiedFetch({
63
- gateways: ['https://mygateway.example.net', 'https://trustless-gateway.link']
64
- })
65
- const response = await fetch('ipfs://bafyFoo') // CID for some image file
75
+ const response = await verifiedFetch('ipfs://bafyFoo') // CID for some image file
66
76
  const blob = await response.blob()
67
77
  const image = document.createElement('img')
68
78
  image.src = URL.createObjectURL(blob)
@@ -71,23 +81,43 @@ document.body.appendChild(image)
71
81
 
72
82
  ## Example - Using IPNS protocol to stream a big file
73
83
 
84
+ ```typescript
85
+ import { verifiedFetch } from '@helia/verified-fetch'
86
+
87
+ const response = await verifiedFetch('ipns://mydomain.com/path/to/very-long-file.log')
88
+ const bigFileStreamReader = await response.body?.getReader()
89
+ ```
90
+
91
+ ## Configuration
92
+
93
+ ### Custom HTTP gateways and routers
94
+
95
+ Out of the box `@helia/verified-fetch` uses a default set of [trustless gateways](https://specs.ipfs.tech/http-gateways/trustless-gateway/) for fetching blocks and [HTTP delegated routers](https://specs.ipfs.tech/routing/http-routing-v1/) for performing routing tasks - looking up peers, resolving/publishing [IPNS](https://docs.ipfs.tech/concepts/ipns/) names, etc.
96
+
97
+ It's possible to override these by passing `gateways` and `routers` keys to the `createVerifiedFetch` function:
98
+
99
+ ## Example - Configuring gateways and routers
100
+
74
101
  ```typescript
75
102
  import { createVerifiedFetch } from '@helia/verified-fetch'
76
103
 
77
104
  const fetch = await createVerifiedFetch({
78
- gateways: ['https://mygateway.example.net', 'https://trustless-gateway.link']
105
+ gateways: ['https://trustless-gateway.link'],
106
+ routers: ['http://delegated-ipfs.dev']
79
107
  })
80
- const response = await fetch('ipns://mydomain.com/path/to/very-long-file.log')
81
- const bigFileStreamReader = await response.body.getReader()
108
+
109
+ const resp = await fetch('ipfs://bafy...')
110
+
111
+ const json = await resp.json()
82
112
  ```
83
113
 
84
- ### Configuration
114
+ ### Usage with customized Helia
85
115
 
86
- #### Usage with customized Helia
116
+ For full control of how `@helia/verified-fetch` fetches content from the distributed web you can pass a preconfigured Helia node to `createVerifiedFetch`.
87
117
 
88
- You can see variations of Helia and js-libp2p configuration options at <https://helia.io/interfaces/helia.index.HeliaInit.html>.
118
+ The [helia](https://www.npmjs.com/package/helia) module is configured with a libp2p node that is suited for decentralized applications, alternatively [@helia/http](https://www.npmjs.com/package/@helia/http) is available which uses HTTP gateways for all network operations.
89
119
 
90
- The `@helia/http` module is currently in-progress, but the init options should be a subset of the `helia` module's init options. See <https://github.com/ipfs/helia/issues/289> for more information.
120
+ You can see variations of Helia and js-libp2p configuration options at <https://helia.io/interfaces/helia.index.HeliaInit.html>.
91
121
 
92
122
  ```typescript
93
123
  import { trustlessGateway } from '@helia/block-brokers'
@@ -97,13 +127,13 @@ import { createVerifiedFetch } from '@helia/verified-fetch'
97
127
 
98
128
  const fetch = await createVerifiedFetch(
99
129
  await createHeliaHTTP({
100
- blockBrokers: [
101
- trustlessGateway({
102
- gateways: ['https://mygateway.example.net', 'https://trustless-gateway.link']
103
- })
104
- ],
105
- routers: ['http://delegated-ipfs.dev'].map((routerUrl) => delegatedHTTPRouting(routerUrl))
106
- })
130
+ blockBrokers: [
131
+ trustlessGateway({
132
+ gateways: ['https://mygateway.example.net', 'https://trustless-gateway.link']
133
+ })
134
+ ],
135
+ routers: ['http://delegated-ipfs.dev'].map((routerUrl) => delegatedHTTPRouting(routerUrl))
136
+ })
107
137
  )
108
138
 
109
139
  const resp = await fetch('ipfs://bafy...')
@@ -111,28 +141,295 @@ const resp = await fetch('ipfs://bafy...')
111
141
  const json = await resp.json()
112
142
  ```
113
143
 
114
- ### Comparison to fetch
144
+ ### Custom content-type parsing
145
+
146
+ By default, if the response can be parsed as JSON, `@helia/verified-fetch` sets the `Content-Type` header as `application/json`, otherwise it sets it as `application/octet-stream` - this is because the `.json()`, `.text()`, `.blob()`, and `.arrayBuffer()` methods will usually work as expected without a detailed content type.
147
+
148
+ If you require an accurate content-type you can provide a `contentTypeParser` function as an option to `createVerifiedFetch` to handle parsing the content type.
149
+
150
+ The function you provide will be called with the first chunk of bytes from the file and should return a string or a promise of a string.
151
+
152
+ ## Example - Customizing content-type parsing
153
+
154
+ ```typescript
155
+ import { createVerifiedFetch } from '@helia/verified-fetch'
156
+ import { fileTypeFromBuffer } from '@sgtpooki/file-type'
157
+
158
+ const fetch = await createVerifiedFetch({
159
+ gateways: ['https://trustless-gateway.link'],
160
+ routers: ['http://delegated-ipfs.dev']
161
+ }, {
162
+ contentTypeParser: async (bytes) => {
163
+ // call to some magic-byte recognition library like magic-bytes, file-type, or your own custom byte recognition
164
+ const result = await fileTypeFromBuffer(bytes)
165
+ return result?.mime
166
+ }
167
+ })
168
+ ```
169
+
170
+ ### Custom DNS resolvers
171
+
172
+ If you don't want to leak DNS queries to the default resolvers, you can provide your own list of DNS resolvers to `createVerifiedFetch`.
173
+
174
+ Note that you do not need to provide both a DNS-over-HTTPS and a DNS-over-JSON resolver, and you should prefer `dnsJsonOverHttps` resolvers for usage in the browser for a smaller bundle size. See <https://github.com/ipfs/helia/tree/main/packages/ipns#example---using-dns-json-over-https> for more information.
175
+
176
+ ## Example - Customizing DNS resolvers
177
+
178
+ ```typescript
179
+ import { createVerifiedFetch } from '@helia/verified-fetch'
180
+ import { dnsJsonOverHttps, dnsOverHttps } from '@helia/ipns/dns-resolvers'
181
+
182
+ const fetch = await createVerifiedFetch({
183
+ gateways: ['https://trustless-gateway.link'],
184
+ routers: ['http://delegated-ipfs.dev'],
185
+ dnsResolvers: [
186
+ dnsJsonOverHttps('https://my-dns-resolver.example.com/dns-json'),
187
+ dnsOverHttps('https://my-dns-resolver.example.com/dns-query')
188
+ ]
189
+ })
190
+ ```
191
+
192
+ ### IPLD codec handling
193
+
194
+ IPFS supports several data formats (typically referred to as codecs) which are included in the CID. `@helia/verified-fetch` attempts to abstract away some of the details for easier consumption.
195
+
196
+ #### DAG-PB
197
+
198
+ [DAG-PB](https://ipld.io/docs/codecs/known/dag-pb/) is the codec we are most likely to encounter, it is what [UnixFS](https://github.com/ipfs/specs/blob/main/UNIXFS.md) uses under the hood.
199
+
200
+ ##### Using the DAG-PB codec as a Blob
201
+
202
+ ```typescript
203
+ import { verifiedFetch } from '@helia/verified-fetch'
204
+
205
+ const res = await verifiedFetch('ipfs://Qmfoo')
206
+ const blob = await res.blob()
207
+
208
+ console.info(blob) // Blob { size: x, type: 'application/octet-stream' }
209
+ ```
210
+
211
+ ##### Using the DAG-PB codec as an ArrayBuffer
212
+
213
+ ```typescript
214
+ import { verifiedFetch } from '@helia/verified-fetch'
215
+
216
+ const res = await verifiedFetch('ipfs://Qmfoo')
217
+ const buf = await res.arrayBuffer()
218
+
219
+ console.info(buf) // ArrayBuffer { [Uint8Contents]: < ... >, byteLength: x }
220
+ ```
221
+
222
+ ##### Using the DAG-PB codec as a stream
223
+
224
+ ```typescript
225
+ import { verifiedFetch } from '@helia/verified-fetch'
226
+
227
+ const res = await verifiedFetch('ipfs://Qmfoo')
228
+ const reader = res.body?.getReader()
229
+
230
+ if (reader == null) {
231
+ throw new Error('Could not create reader from response body')
232
+ }
233
+
234
+ while (true) {
235
+ const next = await reader.read()
236
+
237
+ if (next?.done === true) {
238
+ break
239
+ }
240
+
241
+ if (next?.value != null) {
242
+ console.info(next.value) // Uint8Array(x) [ ... ]
243
+ }
244
+ }
245
+ ```
246
+
247
+ ##### Content-Type
248
+
249
+ When fetching `DAG-PB` data, the content type will be set to `application/octet-stream` unless a custom content-type parser is configured.
250
+
251
+ #### JSON
252
+
253
+ The JSON codec is a very simple codec, a block parseable with this codec is a JSON string encoded into a `Uint8Array`.
254
+
255
+ ##### Using the JSON codec
256
+
257
+ ```typescript
258
+ import * as json from 'multiformats/codecs/json'
259
+
260
+ const block = new TextEncoder().encode('{ "hello": "world" }')
261
+ const obj = json.decode(block)
262
+
263
+ console.info(obj) // { hello: 'world' }
264
+ ```
265
+
266
+ ##### Content-Type
267
+
268
+ When the `JSON` codec is encountered, the `Content-Type` header of the response will be set to `application/json`.
269
+
270
+ ### DAG-JSON
271
+
272
+ [DAG-JSON](https://ipld.io/docs/codecs/known/dag-json/) expands on the `JSON` codec, adding the ability to contain [CID](https://docs.ipfs.tech/concepts/content-addressing/)s which act as links to other blocks, and byte arrays.
273
+
274
+ `CID`s and byte arrays are represented using special object structures with a single `"/"` property.
275
+
276
+ Using `DAG-JSON` has two important caveats:
277
+
278
+ 1. Your `JSON` structure cannot contain an object with only a `"/"` property, as it will be interpreted as a special type.
279
+ 2. Since `JSON` has no technical limit on number sizes, `DAG-JSON` also allows numbers larger than `Number.MAX_SAFE_INTEGER`. JavaScript requires use of `BigInt`s to represent numbers larger than this, and `JSON.parse` does not support them, so precision will be lost.
280
+
281
+ Otherwise this codec follows the same rules as the `JSON` codec.
282
+
283
+ ##### Using the DAG-JSON codec
284
+
285
+ ```typescript
286
+ import * as dagJson from '@ipld/dag-json'
287
+
288
+ const block = new TextEncoder().encode(`{
289
+ "hello": "world",
290
+ "cid": {
291
+ "/": "baeaaac3imvwgy3zao5xxe3de"
292
+ },
293
+ "buf": {
294
+ "/": {
295
+ "bytes": "AAECAwQ"
296
+ }
297
+ }
298
+ }`)
299
+
300
+ const obj = dagJson.decode(block)
301
+
302
+ console.info(obj)
303
+ // {
304
+ // hello: 'world',
305
+ // cid: CID(baeaaac3imvwgy3zao5xxe3de),
306
+ // buf: Uint8Array(5) [ 0, 1, 2, 3, 4 ]
307
+ // }
308
+ ```
309
+
310
+ ##### Content-Type
311
+
312
+ When the `DAG-JSON` codec is encountered in the requested CID, the `Content-Type` header of the response will be set to `application/json`.
313
+
314
+ `DAG-JSON` data can be parsed from the response by using the `.json()` function, which will return `CID`s/byte arrays as plain `{ "/": ... }` objects:
315
+
316
+ ```typescript
317
+ import { verifiedFetch } from '@helia/verified-fetch'
318
+ import * as dagJson from '@ipld/dag-json'
319
+
320
+ const res = await verifiedFetch('ipfs://bafyDAGJSON')
321
+
322
+ // either:
323
+ const obj = await res.json()
324
+ console.info(obj.cid) // { "/": "baeaaac3imvwgy3zao5xxe3de" }
325
+ console.info(obj.buf) // { "/": { "bytes": "AAECAwQ" } }
326
+ ```
327
+
328
+ Alternatively, it can be decoded using the `@ipld/dag-json` module and the `.arrayBuffer()` method, in which case you will get `CID` objects and `Uint8Array`s:
329
+
330
+ ```typescript
331
+ import { verifiedFetch } from '@helia/verified-fetch'
332
+ import * as dagJson from '@ipld/dag-json'
333
+
334
+ const res = await verifiedFetch('ipfs://bafyDAGJSON')
335
+
336
+ // or:
337
+ const obj = dagJson.decode<any>(await res.arrayBuffer())
338
+ console.info(obj.cid) // CID(baeaaac3imvwgy3zao5xxe3de)
339
+ console.info(obj.buf) // Uint8Array(5) [ 0, 1, 2, 3, 4 ]
340
+ ```
341
+
342
+ #### DAG-CBOR
343
+
344
+ [DAG-CBOR](https://ipld.io/docs/codecs/known/dag-cbor/) uses the [Concise Binary Object Representation](https://cbor.io/) format for serialization instead of JSON.
345
+
346
+ This supports more datatypes in a safer way than JSON and is smaller on the wire to boot so is usually preferable to JSON or DAG-JSON.
347
+
348
+ ##### Content-Type
349
+
350
+ Not all data types supported by `DAG-CBOR` can be successfully turned into JSON and back into the same binary form.
351
+
352
+ When a decoded block can be round-tripped to JSON, the `Content-Type` will be set to `application/json`. In this case the `.json()` method on the `Response` object can be used to obtain an object representation of the response.
353
+
354
+ When it cannot, the `Content-Type` will be `application/octet-stream` - in this case the `@ipld/dag-json` module must be used to deserialize the return value from `.arrayBuffer()`.
355
+
356
+ ##### Detecting JSON-safe DAG-CBOR
357
+
358
+ If the `Content-Type` header of the response is `application/json`, the `.json()` method may be used to access the response body in object form, otherwise the `.arrayBuffer()` method must be used to decode the raw bytes using the `@ipld/dag-cbor` module.
359
+
360
+ ```typescript
361
+ import { verifiedFetch } from '@helia/verified-fetch'
362
+ import * as dagCbor from '@ipld/dag-cbor'
363
+
364
+ const res = await verifiedFetch('ipfs://bafyDagCborCID')
365
+ let obj
366
+
367
+ if (res.headers.get('Content-Type') === 'application/json') {
368
+ // DAG-CBOR data can be safely decoded as JSON
369
+ obj = await res.json()
370
+ } else {
371
+ // response contains non-JSON friendly data types
372
+ obj = dagCbor.decode(await res.arrayBuffer())
373
+ }
374
+
375
+ console.info(obj) // ...
376
+ ```
377
+
378
+ ## The `Accept` header
379
+
380
+ The `Accept` header can be passed to override certain response processing, or to ensure that the final `Content-Type` of the response is the one that is expected.
381
+
382
+ If the final `Content-Type` does not match the `Accept` header, or if the content cannot be represented in the format dictated by the `Accept` header, or you have configured a custom content type parser, and that parser returns a value that isn't in the accept header, a [406: Not Acceptable](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/406) response will be returned:
383
+
384
+ ```typescript
385
+ import { verifiedFetch } from '@helia/verified-fetch'
386
+
387
+ const res = await verifiedFetch('ipfs://bafyJPEGImageCID', {
388
+ headers: {
389
+ accept: 'image/png'
390
+ }
391
+ })
392
+
393
+ console.info(res.status) // 406 - the image was a JPEG but we specified PNG as the accept header
394
+ ```
395
+
396
+ It can also be used to skip processing the data from some formats such as `DAG-CBOR` if you wish to handle decoding it yourself:
397
+
398
+ ```typescript
399
+ import { verifiedFetch } from '@helia/verified-fetch'
400
+
401
+ const res = await verifiedFetch('ipfs://bafyDAGCBORCID', {
402
+ headers: {
403
+ accept: 'application/octet-stream'
404
+ }
405
+ })
406
+
407
+ console.info(res.headers.get('accept')) // application/octet-stream
408
+ const buf = await res.arrayBuffer() // raw bytes, not processed as JSON
409
+ ```
410
+
411
+ ## Comparison to fetch
115
412
 
116
- First, this library will require instantiation in order to configure the gateways and delegated routers, or potentially a custom Helia instance. Secondly, once your verified-fetch method is created, it will act as similar to the `fetch()` API as possible.
413
+ This module attempts to act as similarly to the `fetch()` API as possible.
117
414
 
118
415
  [The `fetch()` API](https://developer.mozilla.org/en-US/docs/Web/API/fetch) takes two parameters:
119
416
 
120
417
  1. A [resource](https://developer.mozilla.org/en-US/docs/Web/API/fetch#resource)
121
418
  2. An [options object](https://developer.mozilla.org/en-US/docs/Web/API/fetch#options)
122
419
 
123
- #### Resource argument
420
+ ### Resource argument
124
421
 
125
- This library intends to support the following methods of fetching web3 content from IPFS:
422
+ This library supports the following methods of fetching web3 content from IPFS:
126
423
 
127
424
  1. IPFS protocol: `ipfs://<cidv0>` & `ipfs://<cidv0>`
128
425
  2. IPNS protocol: `ipns://<peerId>` & `ipns://<publicKey>` & `ipns://<hostUri_Supporting_DnsLink_TxtRecords>`
129
426
  3. CID instances: An actual CID instance `CID.parse('bafy...')`
130
427
 
131
- As well as support for pathing & params for item 1&2 above according to [IPFS - Path Gateway Specification](https://specs.ipfs.tech/http-gateways/path-gateway) & [IPFS - Trustless Gateway Specification](https://specs.ipfs.tech/http-gateways/trustless-gateway/). Further refinement of those specifications specifically for web-based scenarios can be found in the [Web Pathing Specification IPIP](https://github.com/ipfs/specs/pull/453).
428
+ As well as support for pathing & params for items 1 & 2 above according to [IPFS - Path Gateway Specification](https://specs.ipfs.tech/http-gateways/path-gateway) & [IPFS - Trustless Gateway Specification](https://specs.ipfs.tech/http-gateways/trustless-gateway/). Further refinement of those specifications specifically for web-based scenarios can be found in the [Web Pathing Specification IPIP](https://github.com/ipfs/specs/pull/453).
132
429
 
133
- If you pass a CID instance, we assume you want the content for that specific CID only, and do not support pathing or params for that CID.
430
+ If you pass a CID instance, it assumes you want the content for that specific CID only, and does not support pathing or params for that CID.
134
431
 
135
- #### Options argument
432
+ ### Options argument
136
433
 
137
434
  This library does not plan to support the exact Fetch API options object, as some of the arguments don't make sense. Instead, it will only support options necessary to meet [IPFS specs](https://specs.ipfs.tech/) related to specifying the resultant shape of desired content.
138
435
 
@@ -177,7 +474,7 @@ Some in-flight specs (IPIPs) that will affect the options object this library su
177
474
  4. [IPIP-0328: JSON and CBOR Response Formats on HTTP Gateways](https://specs.ipfs.tech/ipips/ipip-0328/)
178
475
  5. [IPIP-0288: TAR Response Format on HTTP Gateways](https://specs.ipfs.tech/ipips/ipip-0288/)
179
476
 
180
- #### Response types
477
+ ### Response types
181
478
 
182
479
  This library's purpose is to return reasonably representable content from IPFS. In other words, fetching content is intended for leaf-node content -- such as images/videos/audio & other assets, or other IPLD content (with link) -- that can be represented by <https://developer.mozilla.org/en-US/docs/Web/API/Response#instance_methods>. The content type you receive back will depend upon the CID you request as well as the `Accept` header value you provide.
183
480
 
@@ -185,7 +482,7 @@ All content we retrieve from the IPFS network is obtained via an AsyncIterable,
185
482
 
186
483
  If your content doesn't have a mime-type or an [IPFS spec](https://specs.ipfs.tech), this library will not support it, but you can use the [`helia`](https://github.com/ipfs/helia) library directly for those use cases. See [Unsupported response types](#unsupported-response-types) for more information.
187
484
 
188
- ##### Handling response types
485
+ #### Handling response types
189
486
 
190
487
  For handling responses we want to follow conventions/abstractions from Fetch API where possible:
191
488
 
@@ -194,12 +491,12 @@ For handling responses we want to follow conventions/abstractions from Fetch API
194
491
  - For plain text in utf-8, you would call `.text()`
195
492
  - For streaming response data, use something like `response.body.getReader()` to get a [`ReadableStream`](https://developer.mozilla.org/en-US/docs/Web/API/Streams_API/Using_readable_streams#consuming_a_fetch_as_a_stream).
196
493
 
197
- ##### Unsupported response types
494
+ #### Unsupported response types
198
495
 
199
496
  - Returning IPLD nodes or DAGs as JS objects is not supported, as there is no currently well-defined structure for representing this data in an [HTTP Response](https://developer.mozilla.org/en-US/docs/Web/API/Response). Instead, users should request `aplication/vnd.ipld.car` or use the [`helia`](https://github.com/ipfs/helia) library directly for this use case.
200
497
  - Others? Open an issue or PR!
201
498
 
202
- #### Response headers
499
+ ### Response headers
203
500
 
204
501
  This library will set the [HTTP Response](https://developer.mozilla.org/en-US/docs/Web/API/Response) headers to the appropriate values for the content type according to the appropriate [IPFS Specifications](https://specs.ipfs.tech/).
205
502
 
@@ -209,13 +506,13 @@ Some known header specifications:
209
506
  - <https://specs.ipfs.tech/http-gateways/trustless-gateway/#response-headers>
210
507
  - <https://specs.ipfs.tech/http-gateways/subdomain-gateway/#response-headers>
211
508
 
212
- #### Possible Scenarios that could cause confusion
509
+ ### Possible Scenarios that could cause confusion
213
510
 
214
- ##### Attempting to fetch the CID for content that does not make sense
511
+ #### Attempting to fetch the CID for content that does not make sense
215
512
 
216
513
  If you request `bafybeiaysi4s6lnjev27ln5icwm6tueaw2vdykrtjkwiphwekaywqhcjze`, which points to the root of the en.wikipedia.org mirror, a response object does not make sense.
217
514
 
218
- #### Errors
515
+ ### Errors
219
516
 
220
517
  Known Errors that can be thrown:
221
518
 
@@ -240,7 +537,7 @@ Loading this module through a script tag will make it's exports available as `He
240
537
 
241
538
  # API Docs
242
539
 
243
- - <https://ipfs.github.io/helia/modules/_helia_verified_fetch.html>
540
+ - <https://ipfs.github.io/helia-verified-fetch/modules/_helia_verified_fetch.html>
244
541
 
245
542
  # License
246
543
 
@@ -251,7 +548,7 @@ Licensed under either of
251
548
 
252
549
  # Contribute
253
550
 
254
- Contributions welcome! Please check out [the issues](https://github.com/ipfs/helia/issues).
551
+ Contributions welcome! Please check out [the issues](https://github.com/ipfs/helia-verified-fetch/issues).
255
552
 
256
553
  Also see our [contributing document](https://github.com/ipfs/community/blob/master/CONTRIBUTING_JS.md) for more information on how we work, and about contributing in general.
257
554