pixl-server-web 3.0.0 → 3.0.2

package/README.md

@@ -36,6 +36,7 @@ This module is a component for use in [pixl-server](https://www.github.com/jhuck
 	* [brotli_opts](#brotli_opts)
 	* [default_acl](#default_acl)
 	* [blacklist](#blacklist)
+	* [whitelist](#whitelist)
 	* [allow_hosts](#allow_hosts)
 	* [rewrites](#rewrites)
 	* [redirects](#redirects)
@@ -64,6 +65,7 @@ This module is a component for use in [pixl-server](https://www.github.com/jhuck
 	* [startup_message](#startup_message)
 	* [debug_ttl](#debug_ttl)
 	* [debug_bind_local](#debug_bind_local)
+	* [chaos](#chaos)
 	* [https](#https)
 	* [https_port](#https_port)
 	* [https_alt_ports](#https_alt_ports)
@@ -118,7 +120,23 @@ This module is a component for use in [pixl-server](https://www.github.com/jhuck
 	* [Determining HTTP or HTTPS](#determining-http-or-https)
 	* [Self-Referencing URLs](#self-referencing-urls)
 	* [Custom Method Handlers](#custom-method-handlers)
-	* [Let's Encrypt SSL Certificates](#lets-encrypt-ssl-certificates)
+	* [Let's Encrypt / ACME TLS Certificates](#lets-encrypt--acme-tls-certificates)
+		+ [ACME clients](#acme-clients)
+		+ [Point your domain at your server](#point-your-domain-at-your-server)
+		+ [Install Certbot](#install-certbot)
+			- [Ubuntu / Debian](#ubuntu--debian)
+			- [RHEL / CentOS / Fedora](#rhel--centos--fedora)
+		+ [Option A: HTTP-01 (webroot)](#option-a-http-01-webroot)
+			- [Ensure HTTP is working on port 80](#ensure-http-is-working-on-port-80)
+			- [Issue a certificate using webroot](#issue-a-certificate-using-webroot)
+		+ [Configure pixl-server-web for HTTPS](#configure-pixl-server-web-for-https)
+		+ [Automatic renewal](#automatic-renewal)
+			- [Check that renewal timers are installed](#check-that-renewal-timers-are-installed)
+		+ [Option B: DNS-01 with DNS API (wildcards, advanced)](#option-b-dns-01-with-dns-api-wildcards-advanced)
+			- [Using Certbot DNS plugins](#using-certbot-dns-plugins)
+			- [Using acme.sh](#using-acmesh)
+		+ [Where your certificates live](#where-your-certificates-live)
+		+ [Troubleshooting](#troubleshooting)
 	* [Request Max Dump](#request-max-dump)
 - [License](#license)
 
@@ -822,6 +840,29 @@ When set to `true` and running in debug mode (i.e. `--debug` CLI flag on startup
 
 This feature defaults to `false` (disabled).
 
+## chaos
+
+Use the `chaos` feature to introduce optional and random fault injection into your web requests.  Used for testing purposes, this feature can introduce a random delay on every request, and also hijack requests and inject random error responses based on probabilities you specify.  Here is how to use it:
+
+```json
+"chaos": { 
+	"enabled": true, 
+	"uri": ".+", 
+	"delay": { 
+		"min":0, 
+		"max":2000 
+	}, 
+	"errors": { 
+		"503 Service Unavailable": 0.1 
+	}, 
+	"headers": {
+		"Retry-After": 10
+	} 
+}
+```
+
+Set the `chaos.enabled` flag to `true` to enable fault injection.  By default, all URIs will be affected, unless you specify a `chaos.uri` (regular expression) to limit the requests.  Set `chaos.delay.min` and `chaos.delay.max` to the range you want to delay requests (in milliseconds).  Fill the `chaos.errors` object the HTTP repsonse codes (and status messages) you want to see, and how often.  The values are interpreted as probabilities from `0.0` (never) to `1.0` (always).  In the above example, the `HTTP 503` error code will be injected approximately 10% of the time.  When errors are injected, you can include additional response headers in the `chaos.headers` object.
+
 ## https
 
 This boolean allows you to enable HTTPS (SSL) support in the web server.  It defaults to `false`.  Note that you must also set `https_port`, and possibly `https_cert_file` and `https_key_file` for this to work.
@@ -1017,6 +1058,8 @@ callback(
 
 The content body can be a string, a [Buffer](https://nodejs.org/api/buffer.html) object, or a [readable stream](https://nodejs.org/api/stream.html#class-streamreadable).
 
+Note that you can omit the status text and just return a code, e.g. `"200"`, and the web server will fill in the text.
+
 ### Custom Response
 
 The second type of response is to send content directly to the underlying Node.js server by yourself, using `args.response` (see below).  If you do this, you can pass `true` to the callback function, indicating to the web server that you "handled" the response, and it shouldn't do anything else.  Example:
@@ -1895,78 +1938,246 @@ server.WebServer.addMethodHandler( "OPTIONS", "CORS Preflight", function(args, c
 } );
 ```
 
-## Let's Encrypt SSL Certificates
+## Let's Encrypt / ACME TLS Certificates
+
+These are instructions for using [Let's Encrypt](https://letsencrypt.org/) TLS certificates with **pixl-server-web**: how to get your certificate issued and how to keep it renewed automatically.
+
+The examples below use the domain `mydomain.com`. Replace this with your own real domain.
+
+### ACME clients
+
+Let's Encrypt issues certificates using the **ACME protocol**. To talk ACME, you need an ACME *client*.
 
-Here are instructions for using [Let's Encrypt](https://letsencrypt.org/) SSL certificates with pixl-server-web, specifically how to get your certificate issued and how to setup automatic renewal.
+For most people, Let's Encrypt recommends **Certbot** as the default ACME client:
 
-The first thing you should do is make sure your server has a public IP address, and point your domain name to it using a DNS "A" record.  For these examples we will be using the domain `mydomain.com`.
+https://letsencrypt.org/docs/client-options/
 
-Next, you will need to manually install [certbot](https://certbot.eff.org) on your server.  The easiest way to do this is to use the wrapper script [certbot-auto](https://certbot.eff.org/docs/install.html#certbot-auto), like this:
+Other popular clients include:
+
+* **acme.sh** – pure shell, great DNS-API support.
+* Various language-specific clients (Go, Rust, Node, etc.) – see the Let's Encrypt “ACME clients” list for a full catalog.
+
+These docs focus on **Certbot**, with a short note about acme.sh for DNS-based / wildcard setups.
+
+### Point your domain at your server
+
+Before you can get a certificate:
+
+1. Make sure your server has a **public IPv4 (and/or IPv6) address**.
+2. In your DNS provider’s control panel, create an **A record** for your domain:
+	- Name: `@` (or `mydomain.com`, provider-specific)
+	- Type: `A`
+	- Value: your server’s IPv4 address
+	- Optionally create an **AAAA record** for IPv6.
+3. Wait for DNS to propagate.
+
+You should be able to open http://mydomain.com/ in a browser and hit your server.
+
+### Install Certbot
+
+1. Go to: https://certbot.eff.org/
+2. Select:
+	- Your OS (e.g. “Ubuntu 24.04” or “Debian 12”)
+	- Web server: **“None of the above (or other)”**
+3. Follow the instructions shown there.
+
+Typical examples:
+
+#### Ubuntu / Debian
+
+Certbot’s maintainers (EFF) now publish current builds only through Snap:
 
 ```sh
-mkdir -p /usr/local/bin
-curl -s https://dl.eff.org/certbot-auto > /usr/local/bin/certbot-auto
-chmod a+x /usr/local/bin/certbot-auto
+sudo snap install --classic certbot
+sudo ln -s /snap/bin/certbot /usr/local/bin/certbot
 ```
 
-We'll be using the [Webroot](https://certbot.eff.org/docs/using.html#webroot) method for authorization.  Make sure you have a web server running on your server and listening on port 80 (only plain HTTP is required at this point).  Assuming your web server's document root path is `/var/www/html` issue this command:
+#### RHEL / CentOS / Fedora
+
+DNF is the correct command to use on RedHat and family:
 
 ```sh
-/usr/local/bin/certbot-auto certonly --webroot -w /var/www/html -d mydomain.com
+sudo dnf install certbot
 ```
 
-If you need certificates for multiple subdomains, you can repeat the `-d` flag, e.g. `-d mydomain.com -d www.mydomain.com`.
+Verify installation:
 
-Then follow the instructions on the console.  Certbot will ask you a number of questions including asking you for your e-mail address, accepting terms of service, etc.  When you are done, you should see a success message like this:
+```sh
+certbot --version
+```
+
+### Option A: HTTP-01 (webroot)
+
+HTTP-01 requires port **80** open and a web root directory that pixl-server-web (or another HTTP server) uses.
+
+#### Ensure HTTP is working on port 80
+
+Configure pixl-server-web (or any HTTP server) to:
+
+- Listen on **port 80**.
+- Serve static content from a directory, for example `/var/www/html`.
+
+Then visit: http://mydomain.com/
 
+#### Issue a certificate using webroot
+
+```sh
+sudo certbot certonly --webroot -w /var/www/html -d mydomain.com
 ```
-IMPORTANT NOTES:
- - Congratulations! Your certificate and chain have been saved at:
-   /etc/letsencrypt/live/mydomain.com/fullchain.pem
-   Your key file has been saved at:
-   /etc/letsencrypt/live/mydomain.com/privkey.pem
-   Your cert will expire on 2019-06-19. To obtain a new or tweaked
-   version of this certificate in the future, simply run certbot-auto
-   again. To non-interactively renew *all* of your certificates, run
-   "certbot-auto renew"
- - Your account credentials have been saved in your Certbot
-   configuration directory at /etc/letsencrypt. You should make a
-   secure backup of this folder now. This configuration directory will
-   also contain certificates and private keys obtained by Certbot so
-   making regular backups of this folder is ideal.
- - If you like Certbot, please consider supporting our work by:
 
-   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
-   Donating to EFF:                    https://eff.org/donate-le
+Add more hostnames if needed:
+
+```sh
+sudo certbot certonly --webroot -w /var/www/html -d mydomain.com -d www.mydomain.com
 ```
 
-Your SSL certificates are now ready to use in pixl-server-web.  Simply add the following properties to your `WebServer` configuration object, replacing `mydomain.com` with your own domain name:
+Certbot will create:
+
+```
+/etc/letsencrypt/live/mydomain.com/fullchain.pem
+/etc/letsencrypt/live/mydomain.com/privkey.pem
+/etc/letsencrypt/live/mydomain.com/cert.pem
+/etc/letsencrypt/live/mydomain.com/chain.pem
+```
+
+### Configure pixl-server-web for HTTPS
+
+In your pixl-server-web config:
 
 ```js
 "https": true,
 "https_port": 443,
-"https_cert_file": "/etc/letsencrypt/live/mydomain.com/cert.pem",
-"https_key_file": "/etc/letsencrypt/live/mydomain.com/privkey.pem",
-"https_ca_file": "/etc/letsencrypt/live/mydomain.com/chain.pem"
+"https_cert_file":  "/etc/letsencrypt/live/mydomain.com/cert.pem",
+"https_key_file":   "/etc/letsencrypt/live/mydomain.com/privkey.pem",
+"https_ca_file":    "/etc/letsencrypt/live/mydomain.com/chain.pem"
+```
+
+Then restart pixl-server-web so it can bind to port 443.
+
+Visit: https://mydomain.com/
+
+### Automatic renewal
+
+Let's Encrypt certificates are valid for **90 days**.
+
+Certbot automatically installs:
+
+- A **systemd timer**, or
+- A **cron job**
+
+that runs `certbot renew` twice a day.
+
+pixl-server-web will automatically reload certs when they change on disk.  There is no need to trigger a restart.
+
+#### Check that renewal timers are installed
+
+```sh
+systemctl list-timers | grep certbot
+```
+
+Test renewal:
+
+```sh
+sudo certbot renew --dry-run
+```
+
+### Option B: DNS-01 with DNS API (wildcards, advanced)
+
+Use DNS-01 if:
+
+* You want a **wildcard cert** (`*.mydomain.com`).
+* Port 80 is blocked or server is not exposed to the internet.
+* You want a central ACME box managing certs.
+
+DNS-01 works by creating a TXT record:
+
 ```
+_acme-challenge.mydomain.com
+```
+
+#### Using Certbot DNS plugins
 
-Then start your server as root and it should accept `https://` requests on port 443.
+Example: Cloudflare
 
-The final step is to make sure your certificates auto-renew before they expire (every 90 days).  The `certbot-auto` command takes care of this, but we have to take care of invoking it ourselves, i.e. from a [crontab](https://en.wikipedia.org/wiki/Cron).  It is recommended that you run the command every night, noting that it takes no action unless your certificates are about to expire (i.e. within 30 days).
+Install plugin (package varies by distro):
 
-If your certificates were renewed, you will also need to restart pixl-server-web.  The `certbot-auto` command can also do this for you, using a special `--post-hook` command-line argument.  Example:
+```
+sudo apt install python3-certbot-dns-cloudflare
+```
+
+Create credentials file:
+
+`~/.secrets/certbot/cloudflare.ini`
+
+```ini
+dns_cloudflare_api_token = YOUR_TOKEN
+```
+
+Lock permissions:
 
 ```sh
-/usr/local/bin/certbot-auto renew --post-hook "/opt/myapp/bin/control.sh restart"
+chmod 600 ~/.secrets/certbot/cloudflare.ini
 ```
 
-Toss that command into a shell script in `/etc/cron.daily/` and it'll run daily at 4 AM local server time.  Note that the command does produce output, even if your certs are not renewed, so you may want to silence it:
+Issue wildcard cert:
 
 ```sh
-/usr/local/bin/certbot-auto renew --post-hook "/opt/myapp/bin/control.sh restart" >/dev/null 2>&1
+sudo certbot certonly --dns-cloudflare --dns-cloudflare-credentials ~/.secrets/certbot/cloudflare.ini -d mydomain.com -d '*.mydomain.com'
 ```
 
-Certbot produces its own log file here: `/var/log/letsencrypt/letsencrypt.log`
+#### Using acme.sh
+
+Install:
+
+```sh
+curl https://get.acme.sh | sh
+```
+
+Issue DNS-based cert:
+
+```sh
+~/.acme.sh/acme.sh --issue --dns dns_cf -d mydomain.com -d '*.mydomain.com'
+```
+
+Install certs into your preferred paths:
+
+```sh
+~/.acme.sh/acme.sh --install-cert -d mydomain.com --key-file /etc/letsencrypt/live/mydomain.com/privkey.pem --fullchain-file /etc/letsencrypt/live/mydomain.com/fullchain.pem
+```
+
+### Where your certificates live
+
+Default Certbot paths:
+
+```
+/etc/letsencrypt/live/mydomain.com/privkey.pem
+/etc/letsencrypt/live/mydomain.com/fullchain.pem
+/etc/letsencrypt/live/mydomain.com/cert.pem
+/etc/letsencrypt/live/mydomain.com/chain.pem
+```
+
+### Troubleshooting
+
+Test renewal:
+
+```sh
+sudo certbot renew --dry-run
+```
+
+Logs:
+
+```
+/var/log/letsencrypt/letsencrypt.log
+```
+
+- DNS-01 issues:
+	- Verify TXT record exists using `dig` or `nslookup`.
+	- Ensure your API credentials have minimum DNS permissions.
+
+For more information:
+
+- https://letsencrypt.org/docs/
+- https://certbot.eff.org/docs/
 
 ## Request Max Dump
 

package/lib/http.js

@@ -209,7 +209,7 @@ module.exports = class HTTP {
 			if (self.config.get('log_socket_errors')) {
 				self.logError(err.code || 'socket', "Client error: " + socket._pixl_data.id + ": " + msg, err_args);
 			}
-			else {
+			else if (err.code != 'ECONNRESET') {
 				self.logDebug(5, "Client error: " + socket._pixl_data.id + ": " + msg, err_args);
 			}
 			

package/lib/https.js

@@ -300,7 +300,7 @@ module.exports = class HTTP2 {
 			if (self.config.get('log_socket_errors')) {
 				self.logError(err.code || 'socket', "Client error: " + socket._pixl_data.id + ": " + msg, err_args);
 			}
-			else {
+			else if (err.code != 'ECONNRESET') {
 				self.logDebug(5, "Client error: " + socket._pixl_data.id + ": " + msg, err_args);
 			}
 			

package/lib/response.js

@@ -72,10 +72,16 @@ module.exports = class Response {
 		// parse code and status
 		var http_code = 200;
 		var http_status = "OK";
+		
 		if (status.match(/^(\d+)\s+(.+)$/)) {
 			http_code = parseInt( RegExp.$1 );
 			http_status = RegExp.$2;
 		}
+		else if (this.responseCodes[status]) {
+			http_code = parseInt(status);
+			http_status = this.responseCodes[status];
+		}
+		
 		args.http_code = http_code;
 		args.http_status = http_status;
 		

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "pixl-server-web",
-	"version": "3.0.0",
+	"version": "3.0.2",
 	"description": "A web server component for the pixl-server framework.",
 	"author": "Joseph Huckaby <jhuckaby@gmail.com>",
 	"homepage": "https://github.com/jhuckaby/pixl-server-web",

package/web_server.js

@@ -85,7 +85,68 @@ module.exports = Class({
 	stats: null,
 	recent: null,
 	
-	badHeaderCharPattern: /([\x7F-\xFF\x00-\x1F\u00FF-\uFFFF])/g
+	badHeaderCharPattern: /([\x7F-\xFF\x00-\x1F\u00FF-\uFFFF])/g,
+	
+	responseCodes: {
+		"100": "Continue",
+		"101": "Switching Protocols",
+		"102": "Processing",
+		"103": "Early Hints",
+		"200": "OK",
+		"201": "Created",
+		"202": "Accepted",
+		"203": "Non Authoritative Information",
+		"204": "No Content",
+		"205": "Reset Content",
+		"206": "Partial Content",
+		"207": "Multi-Status",
+		"300": "Multiple Choices",
+		"301": "Moved Permanently",
+		"302": "Moved Temporarily",
+		"303": "See Other",
+		"304": "Not Modified",
+		"305": "Use Proxy",
+		"307": "Temporary Redirect",
+		"308": "Permanent Redirect",
+		"400": "Bad Request",
+		"401": "Unauthorized",
+		"402": "Payment Required",
+		"403": "Forbidden",
+		"404": "Not Found",
+		"405": "Method Not Allowed",
+		"406": "Not Acceptable",
+		"407": "Proxy Authentication Required",
+		"408": "Request Timeout",
+		"409": "Conflict",
+		"410": "Gone",
+		"411": "Length Required",
+		"412": "Precondition Failed",
+		"413": "Request Entity Too Large",
+		"414": "Request-URI Too Long",
+		"415": "Unsupported Media Type",
+		"416": "Requested Range Not Satisfiable",
+		"417": "Expectation Failed",
+		"418": "I'm a teapot",
+		"419": "Insufficient Space on Resource",
+		"420": "Method Failure",
+		"421": "Misdirected Request",
+		"422": "Unprocessable Entity",
+		"423": "Locked",
+		"424": "Failed Dependency",
+		"426": "Upgrade Required",
+		"428": "Precondition Required",
+		"429": "Too Many Requests",
+		"431": "Request Header Fields Too Large",
+		"451": "Unavailable For Legal Reasons",
+		"500": "Internal Server Error",
+		"501": "Not Implemented",
+		"502": "Bad Gateway",
+		"503": "Service Unavailable",
+		"504": "Gateway Timeout",
+		"505": "HTTP Version Not Supported",
+		"507": "Insufficient Storage",
+		"511": "Network Authentication Required"
+	}
 	
 },
 class WebServer extends Component {
@@ -122,10 +183,40 @@ class WebServer extends Component {
 			this.server.on('ready', function() { setTimeout( self.postStartupMessage.bind(self), 250 ); } );
 		}
 		
+		// optional chaos (fault injection)
+		if (this.config.getPath('chaos.enabled')) this.setupChaos();
+		
 		// start listeners
 		this.startAll(callback);
 	}
 	
+	setupChaos() {
+		// setup chaos system (random delays, errors, etc.)
+		// chaos: { enabled, uri?, delay?: { min:0, max:250 }, errors?: { "503 Service Unavailable": 0.1 }, headers? }
+		var self = this;
+		var chaos = this.config.get('chaos');
+		
+		this.addURIFilter( new RegExp(chaos.uri || '.+'), "Chaos", function(args, callback) {
+			var ms = chaos.delay ? Math.round( chaos.delay.min + (Math.random() * (chaos.delay.max - chaos.delay.min)) ) : 0;
+			if (ms) self.logDebug(9, `Chaos: Delaying request for ${ms}ms`);
+			
+			setTimeout( function() {
+				if (!chaos.errors) return callback(false); // passthru
+				var chosen = false;
+				
+				for (var status in chaos.errors) {
+					if (Math.random() <= chaos.errors[status]) { chosen = status; break; }
+				}
+				
+				if (chosen) {
+					self.logDebug(9, `Chaos: Injecting fault: $(chosen)`);
+					callback( chosen, chaos.headers || {}, "Simulated Error: " + chosen );
+				}
+				else callback(false);
+			}, ms); // setTimeout
+		}); // addURIFilter
+	}
+	
 	prepConfig() {
 		// prep config at startup, and when config is hot reloaded