pmtiles-swarm 0.103.0 → 0.104.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (9) hide show
  1. package/CHANGELOG.md +24 -0
  2. package/README.md +56 -56
  3. package/docs/publishing.md +31 -4
  4. package/docs/serving-tiles.md +4 -0
  5. package/package.json +1 -1
  6. package/src/api.js +60 -6
  7. package/src/catalog.js +39 -2
  8. package/src/library.js +23 -3
  9. package/src/web/index.html +2 -1
package/CHANGELOG.md CHANGED
@@ -7,6 +7,30 @@
7
7
  ### 🐞 Bug fixes
8
8
  - _...Add new stuff here..._
9
9
 
10
+ ## 0.104.0
11
+ ### ✨ Features and improvements
12
+ - **An MBTiles archive is served under its own name.** `/archives/<infohash>/archive.mbtiles`,
13
+ and `/latest/<category>/archive.mbtiles`, rather than `.pmtiles` for everything. It is the same
14
+ endpoint and the same bytes — a web seed hands over the file on disk, which is all the piece
15
+ hashes care about — but a URL is read by people too, and `.pmtiles` on a SQLite file says it is
16
+ something it is not. Asking for the wrong extension answers 404 and names the right one.
17
+
18
+ An MBTiles archive still answers to `archive.pmtiles`, unadvertised. That is the URL every
19
+ torrent published before this distinction existed carries in its `url-list`, and nothing
20
+ rewrites a web seed once it is in one — withdrawing it would strand those swarms.
21
+
22
+ ### 🐞 Bug fixes
23
+ - **A hand-added web seed was checked for its scheme and nothing else.** The self-published seed
24
+ got the full reachability judgement; one added through the console or
25
+ `POST /api/torrents/<infohash>/webseeds` did not, so a loopback address went into the .torrent
26
+ unremarked — and nothing rewrites a web seed once it is in one. Both paths now run the same
27
+ check, before anything is written, so a bad URL in a batch does not leave half of it published.
28
+
29
+ The check also notices a URL ending in a slash. BEP 19 reads that as a directory and appends the
30
+ torrent name, which works only where the host serves the file that way — and webtorrent does
31
+ not implement the rule for single-file torrents, so it requests the directory and fails every
32
+ piece. Allowed, since it is legitimate against a host arranged for it, but reported as a warning.
33
+
10
34
  ## 0.103.0
11
35
  ### ✨ Features and improvements
12
36
  - **Elevation at a coordinate.** `GET /stacks/<id>/elevation?lon=&lat=&zoom=` answers the height
package/README.md CHANGED
@@ -750,62 +750,62 @@ which the endpoint answers 501.
750
750
 
751
751
  ## API
752
752
 
753
- | Method | Path | Purpose |
754
- | ---------------------- | ----------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
755
- | `GET` | `/api/status` | Engine health, counts, watched folders, save locations, free space, and whether the swarm can reach this node |
756
- | `GET` | `/api/torrents` | Catalog joined with live swarm state and what is left of each seeding limit |
757
- | `POST` | `/api/torrents` | Add via `{path}`, `{url}`, `{magnet}`, `{torrentUrl}`, or a raw `.torrent` body |
758
- | `DELETE` | `/api/torrents/:infoHash` | Remove (`?deleteData=true` to delete data too) |
759
- | `GET` | `/api/torrents/:infoHash` | One archive, with disk usage and how it is being read |
760
- | `GET` | `/api/torrents/:infoHash/file`, `/magnet` | Download the `.torrent`, or its magnet URI |
761
- | `GET` | `/api/torrents/:infoHash/peers`, `/trackers`, `/content` | Per-peer, per-tracker and per-file detail |
762
- | `GET` | `/api/torrents/:infoHash/pieces` | Which pieces are held, how rare each is, and what peers hold |
763
- | `PATCH` | `/api/torrents/:infoHash/mode` | Switch between mirror and cache |
764
- | `PATCH` | `/api/torrents/:infoHash/categories` | Set, add or remove categories |
765
- | `PATCH` | `/api/torrents/:infoHash/seeding` | Per-archive seeding limit, or "use the global one" |
766
- | `PATCH` `GET` | `/api/torrents/:infoHash/location` | Move the data; poll the move |
767
- | `POST` | `/api/torrents/:infoHash/recheck` | Hash what is on disk again and believe the result — for an archive whose progress and whose files disagree |
768
- | `POST` | `/api/torrents/:infoHash/pause`, `/resume` | Stop offering it, without forgetting it |
769
- | `POST` | `/api/torrents/:infoHash/webseeds` | Add web seeds — does not change the infohash |
770
- | `DELETE` | `/api/torrents/:infoHash/webseeds` | Drop web seeds — likewise |
771
- | `POST` | `/api/torrents/:infoHash/publish` | What this node offers of the archive itself: `serveArchive`, `selfWebSeed`, `publicDownload` |
772
- | `POST` | `/api/torrents/:infoHash/warm` | Pre-fetch a region (`GET` for progress, `DELETE` to cancel) |
773
- | `DELETE` | `/api/torrents/:infoHash/cache` | Reclaim cached pieces, keep the archive |
774
- | `POST` | `/api/torrents/:infoHash/check` | Has the source changed since the torrent was made? |
775
- | `POST` | `/api/torrents/:infoHash/rebuild` | Rebuild from the current source (mints a new infohash) |
776
- | `POST` | `/api/check-origins` | Check every archive with a watchable source |
777
- | `GET` `DELETE` | `/api/adds` | Adds still in flight — downloads, and local files being hashed — and cancelling a download by URL. A hash cannot be stopped part-way, so it is listed but not cancellable |
778
- | `GET` `POST` | `/api/speed` | Which speed limits are in force, and the manual switch between the two sets |
779
- | `GET` | `/api/categories` | Every category, with the endpoints resolving to its newest build |
780
- | `POST` | `/api/adopt`, `/api/adopt/candidates` | Take over what an engine or another node holds |
781
- | `POST` | `/api/sources/preview` | What a watched web location would take, without taking it |
782
- | `POST` | `/api/subscriptions/preview` | Whether a peer is reachable and what it offers |
783
- | `POST` | `/api/subscriptions/refresh` | Poll subscribed feeds now |
784
- | `POST` | `/api/sources/check` | Check scheduled sources now, rather than at the next due time |
785
- | `POST` | `/api/torrents/:infoHash/hooks/complete` | Run the completion hook again for one archive |
786
- | `GET` `POST` `DELETE` | `/api/tokens`, `/api/tokens/:id` | Mint, list and revoke access tokens |
787
- | `GET` `POST` | `/api/restart` | What a restart would do, and doing it |
788
- | `GET` `DELETE` | `/api/stats` | What this node has served — per-archive counters and the last N requests; `DELETE` clears them |
789
- | `GET` | `/api/storage` | What this node is holding that it could let go of — merged tiles, left-over temporary files, stopped exports, traffic history, tile counters |
790
- | `DELETE` | `/api/storage/:what` | Lets go of one of them. Everything listed is derived and can be rebuilt; the archives are not on the list |
791
- | `GET` | `/api/traffic` | Upload and download speed per archive over time, from `stats.db`; `infoHash`, `hours` and `buckets` narrow it, and `totals` ranks archives by bytes moved |
792
- | `GET` `PATCH` | `/api/config` | Read and change settings |
793
- | `POST` | `/api/login`, `/api/logout` | Console sign-in |
794
- | `GET` | `/api/session` | Who this request is, and whether a credential is needed at all |
795
- | `GET` | `/api/catalog` | The whole catalogue, for a peer keeping itself in step |
796
- | `GET` | `/archives/:infoHash/tiles.json` | TileJSON — **public** |
797
- | `GET` | `/archives/:infoHash/:z/:x/:y.:ext` | One tile — **public** |
798
- | `GET` | `/health` | 200 when this node can serve, 503 when its engine cannot — **public**, no credential, for a load balancer |
799
- | `GET` | `/archives/:infoHash/ready` | Whether this node can serve _this_ archive yet: 200 ready, 503 not yet, 415 never — **public** |
800
- | `GET` | `/archives/:infoHash/archive.torrent` | The `.torrent` a torrent-aware client joins with — **public** |
801
- | `GET` | `/archives/:infoHash/archive.pmtiles` | The archive itself, by byte range — what any PMTiles reader wants, and a valid web seed. Only where [`serveArchive`](docs/configuration.md#servearchive) is on — **public**. Complete archives, unless [`serveArchiveFromSwarm`](docs/configuration.md#servearchivefromswarm) is set, which answers a bounded range from the swarm instead |
802
- | `GET` | `/archives/:infoHash/preview` | Map preview for one archive — admin, not public |
803
- | `GET` | `/latest/` | Every category and the endpoints that resolve to its newest build — **public**, the index of everything below |
804
- | `GET` | `/latest/:category/tiles.json`, `/:name.torrent`, `/magnet` | The newest build in a category — **public**. The torrent name is yours to choose, so a link can read `planetiler-openmaptiles-latest.torrent`; it redirects to the immutable URL, which names the download after the build it actually is |
805
- | `GET` | `/latest/:category/archive.pmtiles` | The newest build in a category as a file, by byte range — **public**, and gated by [`serveArchive`](docs/configuration.md#servearchive). What to point a PMTiles reader at when you want "whichever is current" rather than one build |
806
- | `GET` | `/api/stacks` | Every stack, with what each source resolved to and why one cannot be served yet |
807
- | `GET` | `/api/torrents/:infoHash/stacks` | Which stacks would break if this archive were removed, and how |
808
- | `GET`, `PUT`, `DELETE` | `/api/stacks/:id/raw`, `/api/stacks/:id` | Read a recipe as written, save one, or remove it. `/raw` is the recipe; `/api/stacks` is what it resolved to |
753
+ | Method | Path | Purpose |
754
+ | ---------------------- | ----------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
755
+ | `GET` | `/api/status` | Engine health, counts, watched folders, save locations, free space, and whether the swarm can reach this node |
756
+ | `GET` | `/api/torrents` | Catalog joined with live swarm state and what is left of each seeding limit |
757
+ | `POST` | `/api/torrents` | Add via `{path}`, `{url}`, `{magnet}`, `{torrentUrl}`, or a raw `.torrent` body |
758
+ | `DELETE` | `/api/torrents/:infoHash` | Remove (`?deleteData=true` to delete data too) |
759
+ | `GET` | `/api/torrents/:infoHash` | One archive, with disk usage and how it is being read |
760
+ | `GET` | `/api/torrents/:infoHash/file`, `/magnet` | Download the `.torrent`, or its magnet URI |
761
+ | `GET` | `/api/torrents/:infoHash/peers`, `/trackers`, `/content` | Per-peer, per-tracker and per-file detail |
762
+ | `GET` | `/api/torrents/:infoHash/pieces` | Which pieces are held, how rare each is, and what peers hold |
763
+ | `PATCH` | `/api/torrents/:infoHash/mode` | Switch between mirror and cache |
764
+ | `PATCH` | `/api/torrents/:infoHash/categories` | Set, add or remove categories |
765
+ | `PATCH` | `/api/torrents/:infoHash/seeding` | Per-archive seeding limit, or "use the global one" |
766
+ | `PATCH` `GET` | `/api/torrents/:infoHash/location` | Move the data; poll the move |
767
+ | `POST` | `/api/torrents/:infoHash/recheck` | Hash what is on disk again and believe the result — for an archive whose progress and whose files disagree |
768
+ | `POST` | `/api/torrents/:infoHash/pause`, `/resume` | Stop offering it, without forgetting it |
769
+ | `POST` | `/api/torrents/:infoHash/webseeds` | Add web seeds — does not change the infohash |
770
+ | `DELETE` | `/api/torrents/:infoHash/webseeds` | Drop web seeds — likewise |
771
+ | `POST` | `/api/torrents/:infoHash/publish` | What this node offers of the archive itself: `serveArchive`, `selfWebSeed`, `publicDownload` |
772
+ | `POST` | `/api/torrents/:infoHash/warm` | Pre-fetch a region (`GET` for progress, `DELETE` to cancel) |
773
+ | `DELETE` | `/api/torrents/:infoHash/cache` | Reclaim cached pieces, keep the archive |
774
+ | `POST` | `/api/torrents/:infoHash/check` | Has the source changed since the torrent was made? |
775
+ | `POST` | `/api/torrents/:infoHash/rebuild` | Rebuild from the current source (mints a new infohash) |
776
+ | `POST` | `/api/check-origins` | Check every archive with a watchable source |
777
+ | `GET` `DELETE` | `/api/adds` | Adds still in flight — downloads, and local files being hashed — and cancelling a download by URL. A hash cannot be stopped part-way, so it is listed but not cancellable |
778
+ | `GET` `POST` | `/api/speed` | Which speed limits are in force, and the manual switch between the two sets |
779
+ | `GET` | `/api/categories` | Every category, with the endpoints resolving to its newest build |
780
+ | `POST` | `/api/adopt`, `/api/adopt/candidates` | Take over what an engine or another node holds |
781
+ | `POST` | `/api/sources/preview` | What a watched web location would take, without taking it |
782
+ | `POST` | `/api/subscriptions/preview` | Whether a peer is reachable and what it offers |
783
+ | `POST` | `/api/subscriptions/refresh` | Poll subscribed feeds now |
784
+ | `POST` | `/api/sources/check` | Check scheduled sources now, rather than at the next due time |
785
+ | `POST` | `/api/torrents/:infoHash/hooks/complete` | Run the completion hook again for one archive |
786
+ | `GET` `POST` `DELETE` | `/api/tokens`, `/api/tokens/:id` | Mint, list and revoke access tokens |
787
+ | `GET` `POST` | `/api/restart` | What a restart would do, and doing it |
788
+ | `GET` `DELETE` | `/api/stats` | What this node has served — per-archive counters and the last N requests; `DELETE` clears them |
789
+ | `GET` | `/api/storage` | What this node is holding that it could let go of — merged tiles, left-over temporary files, stopped exports, traffic history, tile counters |
790
+ | `DELETE` | `/api/storage/:what` | Lets go of one of them. Everything listed is derived and can be rebuilt; the archives are not on the list |
791
+ | `GET` | `/api/traffic` | Upload and download speed per archive over time, from `stats.db`; `infoHash`, `hours` and `buckets` narrow it, and `totals` ranks archives by bytes moved |
792
+ | `GET` `PATCH` | `/api/config` | Read and change settings |
793
+ | `POST` | `/api/login`, `/api/logout` | Console sign-in |
794
+ | `GET` | `/api/session` | Who this request is, and whether a credential is needed at all |
795
+ | `GET` | `/api/catalog` | The whole catalogue, for a peer keeping itself in step |
796
+ | `GET` | `/archives/:infoHash/tiles.json` | TileJSON — **public** |
797
+ | `GET` | `/archives/:infoHash/:z/:x/:y.:ext` | One tile — **public** |
798
+ | `GET` | `/health` | 200 when this node can serve, 503 when its engine cannot — **public**, no credential, for a load balancer |
799
+ | `GET` | `/archives/:infoHash/ready` | Whether this node can serve _this_ archive yet: 200 ready, 503 not yet, 415 never — **public** |
800
+ | `GET` | `/archives/:infoHash/archive.torrent` | The `.torrent` a torrent-aware client joins with — **public** |
801
+ | `GET` | `/archives/:infoHash/archive.pmtiles`, `/archive.mbtiles` | The archive itself, by byte range, under the extension its own file has — what any PMTiles reader wants, and a valid web seed. Only where [`serveArchive`](docs/configuration.md#servearchive) is on — **public**. Complete archives, unless [`serveArchiveFromSwarm`](docs/configuration.md#servearchivefromswarm) is set, which answers a bounded range from the swarm instead |
802
+ | `GET` | `/archives/:infoHash/preview` | Map preview for one archive — admin, not public |
803
+ | `GET` | `/latest/` | Every category and the endpoints that resolve to its newest build — **public**, the index of everything below |
804
+ | `GET` | `/latest/:category/tiles.json`, `/:name.torrent`, `/magnet` | The newest build in a category — **public**. The torrent name is yours to choose, so a link can read `planetiler-openmaptiles-latest.torrent`; it redirects to the immutable URL, which names the download after the build it actually is |
805
+ | `GET` | `/latest/:category/archive.pmtiles`, `/archive.mbtiles` | The newest build in a category as a file, by byte range, under the extension its own file has — **public**, and gated by [`serveArchive`](docs/configuration.md#servearchive). What to point a PMTiles reader at when you want "whichever is current" rather than one build |
806
+ | `GET` | `/api/stacks` | Every stack, with what each source resolved to and why one cannot be served yet |
807
+ | `GET` | `/api/torrents/:infoHash/stacks` | Which stacks would break if this archive were removed, and how |
808
+ | `GET`, `PUT`, `DELETE` | `/api/stacks/:id/raw`, `/api/stacks/:id` | Read a recipe as written, save one, or remove it. `/raw` is the recipe; `/api/stacks` is what it resolved to |
809
809
 
810
810
  | `POST` | `/api/stacks/:id/import` | Import a provider's list of PMTiles URLs as sources on this stack. An index with an `items` list, or a plain list of addresses; `dryRun` says what it would write without writing it |
811
811
  | `POST`, `DELETE` | `/api/stacks/:id/bake` | Run the stack over its sources and write the result as an archive, or stop one that is running. Answers as soon as the job starts; watch it on `/api/stacks` |
package/docs/publishing.md CHANGED
@@ -159,10 +159,37 @@ An archive already in circulation can be given a web seed at any time, from the
159
159
  `url-list` sits outside the info dictionary, so the infohash does not change and
160
160
  every magnet, tracker and peer already relying on it keeps working.
161
161
 
162
- The URL has to serve the whole archive, byte for byte, under the name the torrent
163
- gives it. A seed that answers with anything else is not a faster path to the
164
- data — peers will fetch from it, fail the piece hash, and fall back to the swarm
165
- having wasted the transfer.
162
+ The URL has to serve the whole archive, byte for byte, and honour `Range`. What
163
+ it does **not** have to do is match the torrent's file name: every archive here
164
+ is a single-file torrent, and BEP 19 says a `url-list` entry that does not end
165
+ in a slash is the full URL of the file. Clients fetch it exactly as written and
166
+ never compare it to anything.
167
+
168
+ The endpoint is still named for what the file is, because a URL is read by
169
+ people as well as by clients: an MBTiles archive is published as
170
+ `/archives/<infohash>/archive.mbtiles`, and a PMTiles one as `archive.pmtiles`.
171
+ Asking for the wrong one answers 404 and says which to use — except that an
172
+ MBTiles archive also still answers to `archive.pmtiles`, unadvertised, because
173
+ that is the URL torrents published before the distinction existed carry in their
174
+ `url-list`. Withdrawing it would strand those swarms.
175
+
176
+ The bytes are the whole contract, so nothing may be translated on the way out. A
177
+ seed that answers with anything else is not a faster path to the data — peers
178
+ will fetch from it, fail the piece hash, and fall back to the swarm having
179
+ wasted the transfer.
180
+
181
+ A URL **ending in a slash** is the one shape to avoid. BEP 19 reads that as a
182
+ directory and appends the torrent's file name, which works only where the host
183
+ serves the file under that name — and webtorrent does not implement the rule for
184
+ single-file torrents at all: it requests the URL as written, fetches whatever the
185
+ directory returns, and fails every piece. Adding one is allowed, since it is
186
+ legitimate against a host arranged that way, but it is reported back as a
187
+ warning. Give the full URL of the file.
188
+
189
+ Seeds are checked before they are written, and one that no peer could reach — a
190
+ loopback address, or anything that is not a URL — is refused outright rather than
191
+ published. If a batch contains one, none of the batch is written: nothing
192
+ rewrites a web seed once it is in a `.torrent`.
166
193
 
167
194
  Worth doing whenever an archive gets a public home after the fact: a swarm with
168
195
  one web seed starts cold in seconds rather than waiting for a peer to appear.
package/docs/serving-tiles.md CHANGED
@@ -121,6 +121,10 @@ GET /latest/{category}.xml a feed holding only the current build
121
121
  Point a style at `/latest/basemaps/tiles.json` and it keeps working across every
122
122
  rebuild, with no edit.
123
123
 
124
+ The extension follows the file: an MBTiles archive is served as
125
+ `archive.mbtiles`. It is the same endpoint and the same bytes either way — what
126
+ changes is only that the URL stops claiming a format the file is not.
127
+
124
128
  `archive.pmtiles` is the one of these that has to be asked for. It is off until
125
129
  [`serveArchive`](configuration.md#servearchive) is set, on the node or on the
126
130
  archive, because it is the only thing here that is neither small nor metered by
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pmtiles-swarm",
3
- "version": "0.103.0",
3
+ "version": "0.104.0",
4
4
  "description": "BitTorrent distribution for PMTiles map archives: create torrents, watch folders, publish and subscribe to RSS feeds, and seed through qBittorrent or an embedded client",
5
5
  "type": "module",
6
6
  "main": "src/index.js",
package/src/api.js CHANGED
@@ -15,6 +15,7 @@ import {
15
15
  isPublicSurface,
16
16
  } from './auth.js';
17
17
  import {
18
+ archiveExtension,
18
19
  normalizeCategories,
19
20
  publishingBase,
20
21
  publishingFor,
@@ -1158,7 +1159,7 @@ export function createApp({
1158
1159
  `${publishingBase({
1159
1160
  config,
1160
1161
  requestBase: baseUrl(req),
1161
- })}/archives/${entry.infoHash}/archive.pmtiles`,
1162
+ })}/archives/${entry.infoHash}/archive.${archiveExtension(entry)}`,
1162
1163
  // Worked out here: the console runs on the admin listener, whose port
1163
1164
  // no peer can reach.
1164
1165
  base: publishingBase({ config, requestBase: baseUrl(req) }),
@@ -2166,7 +2167,7 @@ export function createApp({
2166
2167
  // Absent rather than null where it is not on offer.
2167
2168
  ...(publishingFor(entry, config).publicDownload
2168
2169
  ? {
2169
- archive: `${baseUrl(req)}/archives/${entry.infoHash}/archive.pmtiles`,
2170
+ archive: `${baseUrl(req)}/archives/${entry.infoHash}/archive.${archiveExtension(entry)}`,
2170
2171
  }
2171
2172
  : {}),
2172
2173
  pmtiles: entry.pmtiles,
@@ -2638,10 +2639,15 @@ export function createApp({
2638
2639
  *
2639
2640
  * `If-Range` is honoured so a rebuild landing mid-read cannot splice two
2640
2641
  * builds. See docs/serving-tiles.md — "How a client knows the build moved".
2642
+ * @param {object} req - The request.
2643
+ * @param {object} res - The response.
2644
+ * @param {string} asked - The extension the URL used.
2645
+ * @returns {object|undefined} - The response, where it answers directly.
2641
2646
  */
2642
- app.get('/latest/:category/archive.pmtiles', (req, res) => {
2647
+ const serveLatestArchiveFile = (req, res, asked) => {
2643
2648
  const entry = newestIn(req.params.category, req);
2644
2649
  if (!entry) return res.status(404).json({ error: 'no such category' });
2650
+ if (!answersTo(entry, asked)) return wrongExtension(entry, asked, res);
2645
2651
 
2646
2652
  if (!publishingFor(entry, config).serveArchive) {
2647
2653
  return res.status(403).json({
@@ -2693,7 +2699,16 @@ export function createApp({
2693
2699
  });
2694
2700
  },
2695
2701
  );
2696
- });
2702
+ };
2703
+
2704
+ // Registered a name at a time rather than as `archive.:ext`, which would
2705
+ // also match `archive.torrent` and shadow the route below it.
2706
+ app.get('/latest/:category/archive.pmtiles', (req, res) =>
2707
+ serveLatestArchiveFile(req, res, 'pmtiles'),
2708
+ );
2709
+ app.get('/latest/:category/archive.mbtiles', (req, res) =>
2710
+ serveLatestArchiveFile(req, res, 'mbtiles'),
2711
+ );
2697
2712
 
2698
2713
  // The newest item on its own, for a subscriber that only ever wants the
2699
2714
  // current build and should not have to parse a backlog to find it.
@@ -3059,9 +3074,41 @@ export function createApp({
3059
3074
  * happened to be at that offset, which for a torrent's sparse allocation is
3060
3075
  * zeroes: worse than a refusal, because it looks like data.
3061
3076
  */
3062
- app.get('/archives/:infoHash/archive.pmtiles', (req, res) => {
3077
+ /**
3078
+ * Whether `archive.<ext>` is a name this archive answers to.
3079
+ *
3080
+ * Its own extension, always. And `.pmtiles` for an MBTiles archive as well,
3081
+ * because that is the URL every torrent published before this distinction
3082
+ * existed carries in its `url-list` -- nothing rewrites a web seed once it is
3083
+ * in one, so withdrawing it would strand those swarms. Nothing advertises it
3084
+ * any more: a door left open rather than an address given out.
3085
+ * @param {object} entry - The catalog entry.
3086
+ * @param {string} asked - The extension requested.
3087
+ * @returns {boolean} - Whether to answer.
3088
+ */
3089
+ const answersTo = (entry, asked) =>
3090
+ asked === archiveExtension(entry) ||
3091
+ (asked === 'pmtiles' && archiveExtension(entry) === 'mbtiles');
3092
+
3093
+ /**
3094
+ * Says which name to ask for, rather than a bare 404.
3095
+ * @param {object} entry - The catalog entry.
3096
+ * @param {string} asked - What was requested.
3097
+ * @param {object} res - The response.
3098
+ * @returns {object} - The response.
3099
+ */
3100
+ const wrongExtension = (entry, asked, res) =>
3101
+ res.status(404).json({
3102
+ error:
3103
+ `this archive is ${archiveExtension(entry)}, so it is not served as ` +
3104
+ `archive.${asked}`,
3105
+ hint: `ask for archive.${archiveExtension(entry)}`,
3106
+ });
3107
+
3108
+ const serveArchiveFile = (req, res, asked) => {
3063
3109
  const entry = catalog.get(req.params.infoHash);
3064
3110
  if (!entry) return res.status(404).json({ error: 'not found' });
3111
+ if (!answersTo(entry, asked)) return wrongExtension(entry, asked, res);
3065
3112
 
3066
3113
  if (!publishingFor(entry, config).serveArchive) {
3067
3114
  return res.status(403).json({
@@ -3116,7 +3163,14 @@ export function createApp({
3116
3163
  });
3117
3164
  },
3118
3165
  );
3119
- });
3166
+ };
3167
+
3168
+ app.get('/archives/:infoHash/archive.pmtiles', (req, res) =>
3169
+ serveArchiveFile(req, res, 'pmtiles'),
3170
+ );
3171
+ app.get('/archives/:infoHash/archive.mbtiles', (req, res) =>
3172
+ serveArchiveFile(req, res, 'mbtiles'),
3173
+ );
3120
3174
 
3121
3175
  app.get(
3122
3176
  '/archives/:infoHash/archive.torrent',
package/src/catalog.js CHANGED
@@ -250,6 +250,19 @@ export class Catalog {
250
250
  return this.#writing;
251
251
  }
252
252
  }
253
+ /**
254
+ * The extension an archive's own bytes are served under.
255
+ *
256
+ * The file on disk decides it, not the endpoint: an MBTiles archive is SQLite
257
+ * whatever it is asked for, and naming it `.pmtiles` tells a reader it is
258
+ * something it is not.
259
+ * @param {object} [entry] - The catalog entry.
260
+ * @returns {string} - `pmtiles` or `mbtiles`.
261
+ */
262
+ export function archiveExtension(entry) {
263
+ return entry?.kind === 'mbtiles' ? 'mbtiles' : 'pmtiles';
264
+ }
265
+
253
266
  /**
254
267
  * Whether a URL is one other peers could plausibly fetch.
255
268
  *
@@ -260,12 +273,13 @@ export class Catalog {
260
273
  * publish it, and what to say about it either way.
261
274
  */
262
275
  export function reachability(url) {
263
- let host;
276
+ let parsed;
264
277
  try {
265
- host = new URL(url).hostname.replace(/^\[|\]$/g, '').toLowerCase();
278
+ parsed = new URL(url);
266
279
  } catch {
267
280
  return { ok: false, why: `not a URL: ${url}` };
268
281
  }
282
+ const host = parsed.hostname.replace(/^\[|\]$/g, '').toLowerCase();
269
283
 
270
284
  if (host === 'localhost' || host === '::1' || /^127\./.test(host)) {
271
285
  return {
@@ -305,6 +319,29 @@ export function reachability(url) {
305
319
  };
306
320
  }
307
321
 
322
+ // A web seed addresses the file, not the directory it sits in. BEP 19 says
323
+ // a url-list entry ending in a slash is a directory, and the client appends
324
+ // the torrent's name to it -- so this only works where the host serves the
325
+ // file under that name, which is a different arrangement from the one a
326
+ // person usually means. Worse, webtorrent does not implement that rule for
327
+ // single-file torrents: it requests the URL exactly as written, so a
328
+ // directory URL fetches the directory and every piece fails to verify.
329
+ //
330
+ // A warning rather than a refusal, because pointing at a directory on a
331
+ // server that does serve the file by name is legitimate and someone may mean
332
+ // it. `pathname` rather than the raw string, so a bare origin -- which is a
333
+ // directory with the slash left off -- is caught too.
334
+ if (parsed.pathname.endsWith('/')) {
335
+ return {
336
+ ok: true,
337
+ warning:
338
+ `${url} names a directory rather than a file. A client following ` +
339
+ "BEP 19 will append the torrent's file name to it, and webtorrent " +
340
+ 'will request it exactly as written and fail every piece. Give the ' +
341
+ 'full URL of the file itself.',
342
+ };
343
+ }
344
+
308
345
  return { ok: true };
309
346
  }
310
347
 
package/src/library.js CHANGED
@@ -11,7 +11,12 @@ import {
11
11
  promote,
12
12
  suffixFor,
13
13
  } from './incomplete.js';
14
- import { publishingBase, publishingFor, reachability } from './catalog.js';
14
+ import {
15
+ archiveExtension,
16
+ publishingBase,
17
+ publishingFor,
18
+ reachability,
19
+ } from './catalog.js';
15
20
  import { linkLatest } from './latest-link.js';
16
21
  import { checkOrigin, fingerprintOrigin } from './origin.js';
17
22
  import { probeMbtiles } from './mbtiles.js';
@@ -3040,10 +3045,23 @@ export class Library {
3040
3045
  const wanted = (Array.isArray(urls) ? urls : [urls]).filter(Boolean);
3041
3046
  if (wanted.length === 0) throw new Error('no web seeds given');
3042
3047
 
3048
+ // The same judgement the self-published seed gets. Without this a hand-added
3049
+ // seed was checked for its scheme and nothing else, so a loopback address
3050
+ // went into the .torrent unremarked -- and nothing rewrites a web seed once
3051
+ // it is in one.
3052
+ const warnings = [];
3043
3053
  for (const url of wanted) {
3044
3054
  if (!/^https?:\/\//i.test(url)) {
3045
3055
  throw new Error(`web seed must be an http(s) URL: ${url}`);
3046
3056
  }
3057
+ const reach = reachability(url);
3058
+ // Refused before anything is written, so a bad one in a list of good
3059
+ // ones does not leave half of them published.
3060
+ if (!reach.ok) throw new Error(reach.why);
3061
+ if (reach.warning) {
3062
+ console.warn(`[web seed] ${reach.warning}`);
3063
+ warnings.push(reach.warning);
3064
+ }
3047
3065
  }
3048
3066
 
3049
3067
  const { webSeeds } = await this.#writeWebSeeds(
@@ -3064,7 +3082,7 @@ export class Library {
3064
3082
  live = results.every(Boolean);
3065
3083
  }
3066
3084
 
3067
- return { webSeeds, live };
3085
+ return { webSeeds, live, warnings };
3068
3086
  }
3069
3087
 
3070
3088
  /**
@@ -3159,7 +3177,9 @@ export class Library {
3159
3177
  'URL: set publishingUrl, or give one with the request',
3160
3178
  );
3161
3179
  }
3162
- webSeed = `${base}/archives/${infoHash}/archive.pmtiles`;
3180
+ // Named for what the file is. An MBTiles archive advertised under
3181
+ // `.pmtiles` tells every peer that fetches it that it is something else.
3182
+ webSeed = `${base}/archives/${infoHash}/archive.${archiveExtension(entry)}`;
3163
3183
  // Nothing rewrites a web seed once it is in a .torrent.
3164
3184
  const reach = reachability(webSeed);
3165
3185
  if (!reach.ok) throw new Error(reach.why);
package/src/web/index.html CHANGED
@@ -2373,7 +2373,8 @@
2373
2373
  // does not send one.
2374
2374
  const archiveUrl =
2375
2375
  on.url ??
2376
- `${location.origin}/archives/${entry.infoHash}/archive.pmtiles`;
2376
+ `${location.origin}/archives/${entry.infoHash}/archive.` +
2377
+ (entry.kind === 'mbtiles' ? 'mbtiles' : 'pmtiles');
2377
2378
 
2378
2379
  // Three switches rather than one, because they are three separate
2379
2380
  // exposures. Serving the file to a reader that was handed the URL,