hfs 3.0.2 → 3.0.3

package/README.md

@@ -0,0 +1,194 @@
+# HFS: HTTP File Server
+
+![logo and motto](hfs-logo-color-motto.svg)
+
+## Index
+
+- [Introduction](#introduction)
+- [How does it work](#how-does-it-work)
+- [Features](#features)
+- [Installation](#installation)
+  - [Other systems](#other-systems)
+- [Configuration](#configuration)
+  - [Where is it stored](#where-is-it-stored)
+- [Internationalization](#internationalization)
+- [Hidden features](#hidden-features)
+- [Contribute](#contribute)
+- [More](#more)
+
+## Introduction
+
+Access your files via web, directly from your disk.
+
+- Be your own server: share files **fresh from your disk** with **unlimited** space and bandwidth.
+- **Fast:** try zipping 100GB – download starts immediately.
+- **Smart:** HFS detects problems and suggests solutions.
+- Present things the way you want: share **even a single file**, even with a different name, with our *virtual file system*.
+- **Monitor** all activities in real-time.
+- **Bandwidth throttling**: decide how much to give.
+- **Direct transfers**: share large files with friends without having to upload them first.
+- **Expandable**: install plugins for additional features.
+
+Runs on: Windows, Linux, macOS, FreeBSD, Android
+
+## How does it work
+
+- run HFS on your computer; an administration webpage automatically shows up
+- select which files and folders you want to be accessible
+- access those files from a phone or another computer just using a browser
+- possibly create accounts and limit access to files
+
+## Features
+
+- https
+- easy certificate generation
+- virtual file system
+- mobile friendly
+- search
+- accounts
+- resumable downloads & uploads
+- download folders as zip archive
+- delete, move and rename files
+- plug-ins (anti-brute-force, thumbnails, ldap, themes, and more)
+- simple website serving
+- real-time monitoring of connections
+- [show some files](https://github.com/rejetto/hfs/discussions/270)
+- speed throttler
+- geographic firewall
+- admin web interface
+- multi-language front-end
+- virtual hosting
+- [reverse-proxy support](https://github.com/rejetto/hfs/wiki/Reverse-proxy)
+- comments in file descript.ion
+- integrated media player
+- [customizable with html, css, and javascript](https://github.com/rejetto/hfs/wiki/Customization)
+- dynamic-dns updater
+
+## Installation
+
+For Docker, see https://github.com/rejetto/hfs/wiki/Docker . 
+
+For service installation, see https://github.com/rejetto/hfs/wiki/Service-installation.
+
+The minimum Windows version required is 10 or Server 2019.
+
+1. Download the zip file for your operating system from https://github.com/rejetto/hfs/releases
+   - ⚠️ Antivirus problems on Windows? [READ THIS](https://github.com/rejetto/hfs/wiki/Antivirus)
+   - ⚠️ If you have Linux ARM or other unlisted/unsupported platforms, please see the [Other systems](#other-systems) section.
+2. Unzip and launch the `hfs` file.
+   - ⚠️ Mac: if you get *"cannot be opened because it is from an unidentified developer"*,
+     you can hold `control` key while clicking, then click `open`.
+3. The browser should automatically open at `localhost`, so you can configure the rest in the Admin-panel.
+
+Troubleshooting
+   - If a browser cannot be opened on the computer where you are installing HFS, 
+     you should enter this command in the HFS console: `create-admin <PASSWORD>`
+   - If you cannot access the console (like when you are running as a service), 
+       you can [edit the config file to add your admin account](config.md#accounts)
+   - If you don't want to use an editor, you can create the file with this command: 
+     
+     `echo "create-admin: PASSWORD" > config.yaml` 
+
+By default, HFS does not require a login when you access the *Admin-panel* from localhost.
+If you don't like this behavior, disable it in the Admin-panel or enter this console command `config localhost_admin false`.
+
+To uninstall, remove the files you unzipped and the configuration/data directory (see `config.md` for the location).
+
+### Other systems
+
+If you can't or don't want to run our binary versions, you can try this:
+
+ 1. [install node.js](https://nodejs.org) version 20 (or greater, but then compatibility is not guaranteed)
+2. run at the command line `npx hfs@latest`
+
+The `@latest` part is optional, and ensures that you are always up to date.
+
+If this procedure fails, it may be that you are missing one of [these requirements](https://github.com/nodejs/node-gyp#installation).
+
+Configuration and other files will be stored in `%HOME%/.vfs`
+
+## Configuration
+
+For configuration please see [config.md](config.md); it explains also where all configurations are stored.
+
+## Internationalization
+
+It is possible to show the Front-end in other languages.
+Translation for some languages is already provided. If you find an error, consider reporting it
+or [editing the source file](https://github.com/rejetto/hfs/tree/main/src/langs). 
+
+In the Languages section of the Admin-panel you can install additional language files.
+
+If your language is missing, please consider [translating yourself](https://github.com/rejetto/hfs/wiki/Translation). 
+
+## Hidden features
+
+- Append `#LOGIN` to the URL to open the login dialog
+- Append `?lang=CODE` to the URL to force a specific language
+- `Right-click` on toggle-all checkbox to *invert* the state of all checkboxes
+- Append `?login=USER:PASSWORD` to automatically log in to the browser
+- Append `?overwrite` when uploading to override the `dont_overwrite_uploading` configuration, provided you also have *delete* permission
+- Append `?search=PATTERN` to trigger a search on startup
+- Append `?onlyFiles` or `?onlyFolders` to limit the type of results
+- Append `?get=basic` to display a basic web interface, intended for older/simpler browsers 
+  - This is automatic if a basic browser is detected.
+- Append `?autoplay=shuffle` to trigger show & play; `?autoplay` will not shuffle, but also will not start until the list is complete 
+- `Right-click` on "check for updates" to enter a URL of a version to install
+- `Shift+click` on a file to *show* and play
+- `Ctrl+backspace` to navigate to the parent folder
+- Start typing a filename to focus it in the list
+- `--consoleFile PATH` to also output all stdout/stderr to a file
+- Set env.var. `DISABLE_UPDATE=1` (for containers)
+- Launch with `--debug` or env.var. `HFS_DEBUG=1` to generate additional console logs 
+- Launch with `--no-central` to skip fetching updated info from GitHub (uses built-in data only)
+
+## Contribute
+
+There are several ways to contribute
+
+- [Report bugs](https://github.com/rejetto/hfs/issues/new?labels=bug&template=bug_report.md)
+
+  It's very important to report bugs, and if you are not so sure about it, don't worry, we'll discuss it.
+  If you find important security problems, please [contact us privately](mailto:a@rejetto.com) so that we can publish a fix before
+  the problem is disclosed, for the safety of other users.  
+
+- Use beta versions, and give feedback. 
+
+  While betas have more problems, you'll get more features and give a huge help to the project. 
+
+- [Translate to your language](https://github.com/rejetto/hfs/wiki/Translation).
+
+- [Suggest ideas](https://github.com/rejetto/hfs/discussions)
+
+  While the project should not become too complex, yours may be an idea for a plugin.
+
+- Write guides or make videos for other users. [We got a wiki](https://github.com/rejetto/hfs/wiki)! 
+
+- Submit your code
+
+  If you'd like to make a change yourself in the code, please first open an "issue" or "discussion" about it,
+  so we'll try to cooperate and understand what's the best path for it.
+
+- [Make a plugin](dev-plugins.md)
+
+  A plugin can change the look (a theme), and/or introduce a new functionality.
+
+## Code Signing Policy
+
+Free code signing 🙏 provided by SignPath.io, certificate by [SignPath Foundation](https://signpath.org).
+
+Author/Reviewer/Approver: Massimo Melina.
+
+Privacy: Update checks are opt-out; other outbound connections are user-triggered.
+
+## More
+
+- [Additional information (Wiki)](https://github.com/rejetto/hfs/wiki)
+
+- [APIs](https://github.com/rejetto/hfs/wiki/APIs)
+
+- [Build yourself](dev.md)
+
+- [License](LICENSE.txt)
+
+- Flag images are public-domain, downloaded from https://flagpedia.net

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "hfs",
-  "version": "3.0.2",
+  "version": "3.0.3",
   "description": "HTTP File Server",
   "keywords": ["file server", "http server"],
   "homepage": "https://rejetto.com/hfs",
@@ -33,7 +33,7 @@
     "dist-bin-mac-arm": "cd dist && pkg . --public -C gzip -t node20-macos-arm64 && zip hfs-mac-arm64-$(jq -r .version ../package.json).zip hfs -r plugins && cd ..",
     "dist-bin-mac": "cd dist && pkg . --public -C gzip -t node20-macos-x64 && zip hfs-mac-x64-$(jq -r .version ../package.json).zip hfs -r plugins && cd ..",
     "dist-bin-linux": "cd dist && pkg . --public -C gzip -t node20-linux-x64 && zip hfs-linux-x64-$(jq -r .version ../package.json).zip hfs -r plugins && cd ..",
-    "dist-bin-linux-arm": "cd dist && pkg . --public -C gzip -t node20-linux-arm64 && zip hfs-linux-arm64-$(jq -r .version ../package.json).zip hfs -r plugins && cd ..",
+    "dist-bin-linux-arm": "cd dist && pkg . --public -C gzip -t node20-linux-arm64 ${GITHUB_ACTIONS:+--public-packages \"*\"} && zip hfs-linux-arm64-$(jq -r .version ../package.json).zip hfs -r plugins && cd ..",
     "dist-win": "npm run dist-modules && npm run dist-bin-win",
     "dist-mac-arm": "npm run dist-modules && npm run dist-bin-mac-arm",
     "dist-mac": "npm run dist-modules && npm run dist-bin-mac",
@@ -64,7 +64,10 @@
       "admin/**/*",
       "frontend/**/*",
       "**/node_modules/fswin/x64/*",
-      "**/node_modules/axios/dist/node/*"
+      "**/node_modules/axios/dist/node/*",
+      "**/node_modules/buffers/**",
+      "**/node_modules/binary/**",
+      "**/node_modules/unzip-stream/**"
     ],
     "targets": [
       "node20-win-x64",
@@ -119,7 +122,7 @@
     "@types/node": "^20.17.30",
     "@types/node-forge": "^1.3.14",
     "@types/picomatch": "^4.0.2",
-    "@yao-pkg/pkg": "^6.11.0",
+    "@yao-pkg/pkg": "6.13.1",
     "cross-env": "^10.0.0",
     "koa-better-http-proxy": "^0.2.10",
     "nm-prune": "^5.0.0",

package/src/api.auth.js

@@ -24,6 +24,7 @@ const login = async ({ username, password }, ctx) => {
         const account = await (0, auth_1.clearTextLogin)(ctx, username, password, 'api');
         if (!account)
             return new apiMiddleware_1.ApiError(const_1.HTTP_UNAUTHORIZED);
+        await (0, auth_1.setLoggedIn)(ctx, account.username);
     }
     catch (e) {
         return new apiMiddleware_1.ApiError(const_1.HTTP_UNAUTHORIZED, String(e));

package/src/auth.js

@@ -51,11 +51,7 @@ async function clearTextLogin(ctx, u, p, via) {
         return;
     const plugins = await events_1.default.emitAsync('clearTextLogin', { ctx, username: u, password: p, via }); // provide clear password to plugins
     const a = plugins?.some(x => x === true) ? (0, perm_1.getAccount)(u) : await srpCheck(u, p);
-    if (a) {
-        await setLoggedIn(ctx, a.username);
-        ctx.headers['x-username'] = a.username; // give an easier way to determine if the login was successful
-    }
-    else if (u)
+    if (!a && u)
         events_1.default.emit('failedLogin', { ctx, username: u, via });
     return a;
 }
@@ -64,8 +60,11 @@ async function setLoggedIn(ctx, username) {
     const s = ctx.session;
     if (!s)
         return ctx.throw(cross_const_1.HTTP_SERVER_ERROR, 'session');
+    delete ctx.state.usernames;
     if (username === false) {
-        events_1.default.emit('logout', ctx);
+        if (s.username)
+            events_1.default.emit('logout', ctx);
+        delete ctx.state.account;
         delete s.username;
         delete s.allowNet;
         return;

package/src/config.js

@@ -82,7 +82,7 @@ function defineConfig(k, defaultValue, compiler) {
                     return; // avoid infinite loop in case a subscriber changes the value
                 stack.push(cb);
                 try {
-                    cb(v, { k, was, version, defaultValue, object, onlyCompileChanged });
+                    return cb(v, { k, was, version, defaultValue, object, onlyCompileChanged });
                 }
                 finally {
                     stack.pop();
@@ -106,7 +106,7 @@ function defineConfig(k, defaultValue, compiler) {
     if (compiler)
         object.sub((v, more) => {
             if (!more.onlyCompileChanged)
-                compiled = compiler(v, more);
+                return compiled = compiler(v, more);
         });
     return object;
 }

package/src/consoleLog.js

@@ -10,7 +10,7 @@ const cross_1 = require("./cross");
 const fs_1 = require("fs");
 const argv_1 = require("./argv");
 exports.consoleLog = [];
-const f = argv_1.argv.consoleFile ? (0, fs_1.createWriteStream)(argv_1.argv.consoleFile, 'utf-8') : null;
+const f = argv_1.argv.consoleFile ? (0, fs_1.createWriteStream)(argv_1.argv.consoleFile, { flags: 'a', encoding: 'utf8' }) : null;
 for (const k of ['log', 'warn', 'error', 'debug']) {
     const original = console[k];
     console[k] = (...args) => {

package/src/log.js

@@ -102,6 +102,7 @@ const logSpam = (0, config_1.defineConfig)(misc_1.CFG.log_spam, false);
 const debounce = lodash_1.default.debounce(cb => cb(), 1000); // with this technique, i'll be able to debounce some code respecting the references in its closure
 const logMw = async (ctx, next) => {
     const now = new Date(); // request start
+    const userAtStart = (0, auth_1.getCurrentUsername)(ctx);
     // do it now so it's available for returning plugins
     ctx.state.completed = Promise.race([(0, events_1.once)(ctx.res, 'finish'), (0, events_1.once)(ctx.res, 'close')]);
     await next();
@@ -151,7 +152,7 @@ const logMw = async (ctx, next) => {
         const format = '%s - %s [%s] "%s %s HTTP/%s" %d %s %s\n'; // Apache's Common Log Format
         const a = now.toString().split(' '); // like nginx, our default log contains the time of log writing
         const date = a[2] + '/' + a[1] + '/' + a[3] + ':' + a[4] + ' ' + a[5]?.slice(3);
-        const user = (0, auth_1.getCurrentUsername)(ctx);
+        const user = (0, auth_1.getCurrentUsername)(ctx) || userAtStart;
         const length = ctx.state.length ?? ctx.length;
         const uri = ctx.originalUrl;
         const duration = (Date.now() - Number(now)) / 1000;

package/src/middlewares.js

@@ -82,21 +82,30 @@ const someSecurity = (ctx, next) => {
 exports.someSecurity = someSecurity;
 // limited to http proxies
 function getProxyDetected() {
-    if (proxyDetected?.state.whenProxyDetected < Date.now() - misc_1.DAY) // detection is reset after a day
+    if (Number(proxyDetected?.state.whenProxyDetected) < Date.now() - misc_1.DAY) // detection is reset after a day
         proxyDetected = undefined;
     return proxyDetected && { from: proxyDetected.socket.remoteAddress, for: proxyDetected.get('X-Forwarded-For') };
 }
 const prepareState = async (ctx, next) => {
-    if (ctx.session?.username) {
-        if (ctx.session.ts < auth_1.invalidateSessionBefore.get(ctx.session.username))
-            delete ctx.session.username;
-        ctx.session.maxAge = exports.sessionDuration.compiled();
+    const s = ctx.session;
+    if (s?.username) {
+        if (s.ts < auth_1.invalidateSessionBefore.get(s?.username))
+            delete s.username;
+        s.maxAge = exports.sessionDuration.compiled();
     }
     // calculate these once and for all
     ctx.state.connection = (0, connections_1.socket2connection)(ctx.socket);
-    const a = ctx.state.account = await urlLogin() || await getHttpAccount() || (0, perm_1.getAccount)(ctx.session?.username, false);
-    if (a && (!(0, perm_1.accountCanLogin)(a) || failAllowNet(ctx, a))) // enforce allow_net also after login
-        ctx.state.account = undefined;
+    let a = await urlLogin() || await getHttpAccount();
+    const loggedInNotBySession = a;
+    ctx.state.account = a ||= (0, perm_1.getAccount)(s?.username, false); // with least precedence, we consider session
+    if (a)
+        if (!(0, perm_1.accountCanLogin)(a) || failAllowNet(ctx, a)) // enforce allow_net also after login
+            await (0, auth_1.setLoggedIn)(ctx, false);
+        else if (loggedInNotBySession) {
+            if (a.username)
+                await (0, auth_1.setLoggedIn)(ctx, a.username);
+            ctx.headers['x-username'] = a.username; // give an easier way to determine if the login was successful
+        }
     ctx.state.revProxyPath = ctx.get('x-forwarded-prefix');
     (0, connections_1.updateConnectionForCtx)(ctx);
     await next();
@@ -114,7 +123,7 @@ const prepareState = async (ctx, next) => {
             return;
         try {
             const [u, p] = atob(b64).split(':');
-            if (!u || u === ctx.session?.username)
+            if (!u || u === s?.username)
                 return; // providing credentials, but not needed
             return (0, auth_1.clearTextLogin)(ctx, u, p || '', 'header');
         }

package/src/outboundProxy.js

@@ -1,14 +1,15 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 const config_1 = require("./config");
-const node_url_1 = require("node:url");
 const util_http_1 = require("./util-http");
 const util_os_1 = require("./util-os");
 const const_1 = require("./const");
 const cross_1 = require("./cross");
 const outboundProxy = (0, config_1.defineConfig)(cross_1.CFG.outbound_proxy, '', v => {
+    if (!v)
+        return;
     try {
-        (0, node_url_1.parse)(v); // just validate
+        (0, util_http_1.parseHttpUrl)(v); // just validate
         util_http_1.httpStream.defaultProxy = v;
         if (!v || process.env.HFS_SKIP_PROXY_TEST)
             return;

package/src/serveFile.js

@@ -101,12 +101,22 @@ async function serveFile(ctx, source, mime, content) {
             ctx.set('Cache-Control', `max-age=${cc}`);
         const { size } = stats;
         const range = applyRange(ctx, size);
-        ctx.body = (0, fs_1.createReadStream)(source, range);
+        if (ctx.status >= 400)
+            return; // applyRange may have set an error
+        ctx.body = (0, fs_1.createReadStream)(source, range || undefined);
         if (ctx.state.vfsNode)
             monitorAsDownload(ctx, size, range?.start);
     }
     catch (e) {
-        return ctx.status = const_1.HTTP_NOT_FOUND;
+        const status = {
+            ENOENT: const_1.HTTP_NOT_FOUND,
+            ENOTDIR: const_1.HTTP_NOT_FOUND,
+            EACCES: const_1.HTTP_FORBIDDEN,
+            EPERM: const_1.HTTP_FORBIDDEN,
+        }[String(e?.code)];
+        if (!status)
+            throw e;
+        ctx.status = status;
     }
 }
 function monitorAsDownload(ctx, size, offset) {
@@ -131,27 +141,32 @@ function applyRange(ctx, totalSize = ctx.response.length) {
     }
     const [unit, ranges] = range.split('=');
     if (unit !== 'bytes')
-        return ctx.throw(const_1.HTTP_BAD_REQUEST, 'bad range unit');
+        return badRequest('bad range unit');
     if (ranges?.includes(','))
-        return ctx.throw(const_1.HTTP_BAD_REQUEST, 'multi-range not supported');
-    let bytes = ranges?.split('-');
-    if (!bytes?.length)
-        return ctx.throw(const_1.HTTP_BAD_REQUEST, 'bad range');
+        return badRequest('multi-range not supported');
+    const bytes = ranges?.split('-');
+    if (bytes?.length !== 2)
+        return badRequest('bad range');
     const max = totalSize - 1;
-    const start = bytes[0] ? Number(bytes[0]) : Math.max(0, totalSize - Number(bytes[1])); // a negative start is relative to the end
-    const end = (bytes[0] && bytes[1]) ? Math.min(max, Number(bytes[1])) : max;
+    const [startTxt, endTxt] = bytes;
+    const start = startTxt ? Number(startTxt) : Math.max(0, totalSize - Number(endTxt)); // a negative start is relative to the end
+    const end = (startTxt && endTxt) ? Math.min(max, Number(endTxt)) : max;
+    if (isNaN(start) || startTxt && endTxt && isNaN(end))
+        return badRequest('bad range');
     // we don't support last-bytes without knowing max
-    if (isNaN(end) && isNaN(max) || end > max || start > max) {
-        ctx.status = const_1.HTTP_RANGE_NOT_SATISFIABLE;
+    if (isNaN(end) && isNaN(max) || end > max || start > max || start > end) {
         ctx.set('Content-Range', `bytes */${totalSize}`);
-        ctx.body = 'Requested Range Not Satisfiable';
-        return;
+        return badRequest('Requested Range Not Satisfiable', const_1.HTTP_RANGE_NOT_SATISFIABLE);
     }
     ctx.state.includesLastByte = end === max;
     ctx.status = const_1.HTTP_PARTIAL_CONTENT;
     ctx.set('Content-Range', `bytes ${start}-${isNaN(end) ? '' : end}/${isNaN(totalSize) ? '*' : totalSize}`);
     ctx.response.length = end - start + 1;
     return { start, end };
+    function badRequest(message, status = const_1.HTTP_BAD_REQUEST) {
+        ctx.status = status;
+        ctx.body = message;
+    }
 }
 function downloadLimiter(configMax, cbKey) {
     const map = new Map();

package/src/upload.js

@@ -189,7 +189,10 @@ function uploadWriter(base, baseUri, filename, ctx) {
                     while (fs_1.default.existsSync(dest));
                 }
                 try {
-                    await (0, promises_1.rename)(tempName, dest);
+                    const done = await (0, misc_1.waitFor)(// an antivirus may lock the temp file to scan it
+                    () => (0, promises_1.rename)(tempName, dest).then(() => true, e => e?.code !== 'EBUSY' && Promise.reject(e)), { timeout: 10_000 });
+                    if (!done)
+                        throw 'EBUSY';
                     if (mtime) // so we use it to touch the file
                         await (0, promises_1.utimes)(dest, Date.now() / 1000, mtime / 1000);
                     cancelDeletion(tempName); // not necessary, as deletion's failure is silent, but still

package/src/util-http.js

@@ -41,6 +41,7 @@ exports.stream2string = void 0;
 exports.httpString = httpString;
 exports.httpWithBody = httpWithBody;
 exports.httpStream = httpStream;
+exports.parseHttpUrl = parseHttpUrl;
 const node_url_1 = require("node:url");
 const node_https_1 = __importDefault(require("node:https"));
 const node_http_1 = __importDefault(require("node:http"));
@@ -49,6 +50,7 @@ const lodash_1 = __importDefault(require("lodash"));
 const consumers_1 = require("node:stream/consumers");
 Object.defineProperty(exports, "stream2string", { enumerable: true, get: function () { return consumers_1.text; } });
 const tls = __importStar(require("node:tls"));
+const cross_1 = require("./cross");
 async function httpString(url, options) {
     return await (0, consumers_1.text)(await httpStream(url, options));
 }
@@ -74,11 +76,13 @@ function httpStream(url, { body, proxy, jar, noRedirect, httpThrow = true, ...op
             if (!(body instanceof node_stream_1.Readable))
                 options.headers['content-length'] ??= Buffer.byteLength(body);
         }
-        if (jar)
-            options.headers.cookie = lodash_1.default.map(jar, (v, k) => `${k}=${v}; `).join('')
+        const { auth, ...parsed } = parseHttpUrl(url);
+        const hostJar = jar && (jar[parsed.hostname || ''] ||= {});
+        if (hostJar) {
+            options.headers.cookie = lodash_1.default.map(hostJar, (v, k) => `${k}=${v}; `).join('')
                 + (options.headers.cookie || ''); // preserve parameter
-        const { auth, ...parsed } = (0, node_url_1.parse)(url);
-        const proxyParsed = proxy ? (0, node_url_1.parse)(proxy) : null;
+        }
+        const proxyParsed = proxy ? parseHttpUrl(proxy) : null;
         Object.assign(options, lodash_1.default.pick(proxyParsed || parsed, ['hostname', 'port', 'path', 'protocol']));
         if (auth) {
             options.auth = auth;
@@ -87,7 +91,7 @@ function httpStream(url, { body, proxy, jar, noRedirect, httpThrow = true, ...op
         }
         if (proxy) {
             options.path = url; // full url as path
-            options.headers.host ??= (0, node_url_1.parse)(url).host || undefined; // keep original host header
+            options.headers.host ??= parsed.host || undefined; // keep original host header
         }
         // this needs the prefix "proxy-"
         const proxyAuth = proxyParsed?.auth ? { 'proxy-authorization': `Basic ${Buffer.from(proxyParsed.auth, 'utf8').toString('base64')}` } : undefined;
@@ -97,15 +101,15 @@ function httpStream(url, { body, proxy, jar, noRedirect, httpThrow = true, ...op
         const proto = options.protocol === 'https:' ? node_https_1.default : node_http_1.default;
         const req = proto.request(options, res => {
             console.debug("http responded", res.statusCode, "to", url);
-            if (jar)
+            if (hostJar)
                 for (const entry of res.headers['set-cookie'] || []) {
                     const [, k, v] = /(.+?)=([^;]+)/.exec(entry) || [];
                     if (!k)
                         continue;
                     if (v)
-                        jar[k] = v;
+                        hostJar[k] = v;
                     else
-                        delete jar[k];
+                        delete hostJar[k];
                 }
             if (!res.statusCode || httpThrow && res.statusCode >= 400)
                 return reject(new Error(String(res.statusCode), { cause: res }));
@@ -161,3 +165,15 @@ function httpStream(url, { body, proxy, jar, noRedirect, httpThrow = true, ...op
         abort() { controller.abort(); }
     });
 }
+// works the same way as the now deprecated url.parse()
+function parseHttpUrl(url) {
+    const parsed = new URL(url);
+    const options = (0, node_url_1.urlToHttpOptions)(parsed);
+    const withoutHash = url.split('#', 1)[0];
+    const authority = /^[a-z][a-z\d+.-]*:\/\/[^/?#]*/i.exec(withoutHash)?.[0];
+    return {
+        ...options,
+        host: parsed.host,
+        path: !authority ? '/' : (0, cross_1.enforceStarting)('/', withoutHash.slice(authority.length)), // unresolved paths are useful in our tests
+    };
+}