ronin-web 0.3.0.rc1 → 1.0.0.beta1

data/man/ronin-web-reverse-proxy.1.md

@@ -0,0 +1,47 @@
+# ronin-web-proxy 1 "2022-01-01" Ronin "User Manuals"
+
+## SYNOPSIS
+
+`ronin-web proxy` [*options*] [`--host` *HOST*] [`--port` *PORT*]
+
+## DESCRIPTION
+
+Starts a HTTP proxy server.
+
+## OPTIONS
+
+`-H`, `--host` *HOST*
+  The host that the proxy server will listen on. Defaults to `localhost`.
+
+`-p`, `--port` *PORT*
+  The port that the proxy server will listen on. Default to `8080`.
+
+`-b`, `--show-body`
+  Controls whether to display the request/response bodies or not.
+
+`--rewrite-requests` {*STRING*:*REPLACE*|/*REGEXP*/:*REPLACE*}
+  Rewrites all request bodies by replacing the *STRING* or *REGEXP* with the
+  given *REPLACE* string.
+
+`--rewrite-responses` {*STRING*:*REPLACE*|/*REGEXP*/:*REPLACE*}
+  Rewrites all response bodies by replacing the *STRING* or *REGEXP* with the
+  given *REPLACE* string.
+
+`-h`, `--help`
+  prints help information.
+
+## EXAMPLES
+
+`sudo ronin-web proxy --host 0.0.0.0 --port 80`
+  Listen on the external interface on port 80.
+
+`ronin-web proxy --rewrite-responses https:http`
+  Replace every occurrence of `https` with `http` in the response bodies.
+
+## AUTHOR
+
+Postmodern <postmodern.mod3@gmail.com>
+
+## SEE ALSO
+
+ronin-web-server(1)

data/man/ronin-web-server.1

@@ -0,0 +1,59 @@
+.\" Generated by kramdown-man 0.1.8
+.\" https://github.com/postmodern/kramdown-man#readme
+.TH ronin-web-server 1 "2022-01-01" Ronin Web "User Manuals"
+.LP
+.SH SYNOPSIS
+.LP
+.HP
+\fBronin-web-server\fR \[lB]\fIoptions\fP\[rB]
+.LP
+.SH DESCRIPTION
+.LP
+.PP
+Starts a web server\.
+.LP
+.SH OPTIONS
+.LP
+.TP
+\fB-H\fR, \fB--host\fR \fIHOST\fP
+Host name or IP to bind to\. Defaults to \fBlocalhost\fR\.
+.LP
+.TP
+\fB-p\fR, \fB--port\fR \fIPORT\fP
+Port number to listen on\. Defaults to \fB8000\fR\.
+.LP
+.TP
+\fB-A\fR, \fB--basic-auth\fR \fIUSER\fP:\fIPASSWORD\fP
+Sets up Basic\-Authentication with the given \fIUSER\fP and \fIPASSWORD\fP\.
+.LP
+.TP
+\fB-d\fR, \fB--dir\fR \[sl]\fIPATH\fP:\fIDIR\fP
+Mounts a directory to the given \fIPATH\fP\.
+.LP
+.TP
+\fB-f\fR, \fB--file\fR \[sl]\fIPATH\fP:\fIFILE\fP
+Mounts a file to the given \fIPATH\fP\.
+.LP
+.TP
+\fB-r\fR, \fB--root\fR \fIDIR\fP
+Root directory to serve\.
+.LP
+.TP
+\fB-R\fR, \fB--redirect\fR \[sl]\fIPATH\fP:\fIURL\fP
+Registers a \fB302 Found\fR redirect at the given \fIPATH\fP
+.LP
+.TP
+\fB-h\fR, \fB--help\fR
+Print help information
+.LP
+.SH AUTHOR
+.LP
+.PP
+Postmodern 
+.MT postmodern\.mod3\[at]gmail\.com
+.ME
+.LP
+.SH SEE ALSO
+.LP
+.PP
+ronin\-web\-proxy(1)

data/man/ronin-web-server.1.md

@@ -0,0 +1,43 @@
+# ronin-web-server 1 "2022-01-01" Ronin Web "User Manuals"
+
+## SYNOPSIS
+
+`ronin-web-server` [*options*]
+
+## DESCRIPTION
+
+Starts a web server.
+
+## OPTIONS
+
+`-H`, `--host` *HOST*
+  Host name or IP to bind to. Defaults to `localhost`.
+
+`-p`, `--port` *PORT*
+  Port number to listen on. Defaults to `8000`.
+
+`-A`, `--basic-auth` *USER*:*PASSWORD*
+  Sets up Basic-Authentication with the given *USER* and *PASSWORD*.
+
+`-d`, `--dir` /*PATH*:*DIR*
+  Mounts a directory to the given *PATH*.
+
+`-f`, `--file` /*PATH*:*FILE*
+  Mounts a file to the given *PATH*.
+
+`-r`, `--root` *DIR*
+  Root directory to serve.
+
+`-R`, `--redirect` /*PATH*:*URL*
+  Registers a `302 Found` redirect at the given *PATH*
+
+`-h`, `--help`
+  Print help information
+
+## AUTHOR
+
+Postmodern <postmodern.mod3@gmail.com>
+
+## SEE ALSO
+
+ronin-web-proxy(1)

data/man/ronin-web-spider.1

@@ -0,0 +1,225 @@
+.\" Generated by kramdown-man 0.1.8
+.\" https://github.com/postmodern/kramdown-man#readme
+.TH ronin-web-spider 1 "2022-01-01" Ronin Web "User Manuals"
+.LP
+.SH SYNOPSIS
+.LP
+.HP
+\fBronin-web-spider\fR \[lB]\fIoptions\fP\[rB] \[lC]\fB--host\fR \fIHOST\fP \[or] \fB--domain\fR \fIDOMAIN\fP \[or] \fB--site\fR \fIURL\fP\[rC]
+.LP
+.SH DESCRIPTION
+.LP
+.PP
+Spiders a website\.
+.LP
+.SH OPTIONS
+.LP
+.TP
+\fB--open-timeout\fR \fISECS\fP
+Sets the connection open timeout\.
+.LP
+.TP
+\fB--read-timeout\fR \fISECS\fP
+Sets the read timeout\.
+.LP
+.TP
+\fB--ssl-timeout\fR \fISECS\fP
+Sets the SSL connection timeout\.
+.LP
+.TP
+\fB--continue-timeout\fR \fISECS\fP
+Sets the continue timeout\.
+.LP
+.TP
+\fB--keep-alive-timeout\fR \fISECS\fP
+Sets the connection keep alive timeout\.
+.LP
+.TP
+\fB-P\fR, \fB--proxy\fR \fIPROXY\fP
+Sets the proxy to use\.
+.LP
+.TP
+\fB-H\fR, \fB--header\fR \[lq]\fINAME\fP: \fIVALUE\fP\[rq]
+Sets a default header\.
+.LP
+.TP
+\fB--host-header\fR \fINAME\fP\[eq]\fIVALUE\fP
+Sets a default header\.
+.LP
+.HP
+\fB-u\fR, \fB--user-agent\fR chrome\[ru]linux\[or]chrome\[ru]macos\[or]chrome\[ru]windows\[or]chrome\[ru]iphone\[or]chrome\[ru]ipad\[or]chrome\[ru]android\[or]firefox\[ru]linux\[or]firefox\[ru]macos\[or]firefox\[ru]windows\[or]firefox\[ru]iphone\[or]firefox\[ru]ipad\[or]firefox\[ru]android\[or]safari\[ru]macos\[or]safari\[ru]iphone\[or]safari\[ru]ipad\[or]edge
+The \fBUser-Agent\fR to use\.
+.LP
+.TP
+\fB-U\fR, \fB--user-agent-string\fR \fISTRING\fP
+The raw \fBUser-Agent\fR string to use\.
+.LP
+.TP
+\fB-R\fR, \fB--referer\fR \fIURL\fP
+Sets the \fBReferer\fR URL\.
+.LP
+.TP
+\fB--delay\fR \fISECS\fP
+Sets the delay in seconds between each request\.
+.LP
+.TP
+\fB-l\fR, \fB--limit\fR \fICOUNT\fP
+Only spiders up to \fICOUNT\fP pages\.
+.LP
+.TP
+\fB-d\fR, \fB--max-depth\fR \fIDEPTH\fP
+Only spiders up to max depth\.
+.LP
+.TP
+\fB--enqueue\fR \fIURL\fP
+Adds the URL to the queue\.
+.LP
+.TP
+\fB--visited\fR \fIURL\fP
+Marks the URL as previously visited\.
+.LP
+.TP
+\fB--strip-fragments\fR
+Enables\[sl]disables stripping the fragment component of every URL\.
+.LP
+.TP
+\fB--strip-query\fR
+Enables\[sl]disables stripping the query component of every URL\.
+.LP
+.TP
+\fB--visit-host\fR \fIHOST\fP
+Visit URLs with the matching host name\.
+.LP
+.HP
+\fB--visit-hosts-like\fR \[sl]\fIREGEX\fP\[sl]
+Visit URLs with hostnames that match the \fIREGEX\fP\.
+.LP
+.TP
+\fB--ignore-host\fR \fIHOST\fP
+Ignore the host name\.
+.LP
+.HP
+\fB--ignore-hosts-like\fR \[sl]\fIREGEX\fP\[sl]
+Ignore the host names matching the \fIREGEX\fP\.
+.LP
+.TP
+\fB--visit-port\fR \fIPORT\fP
+Visit URLs with the matching port number\.
+.LP
+.HP
+\fB--visit-ports-like\fR \[sl]\fIREGEX\fP\[sl]
+Visit URLs with port numbers that match the \fIREGEX\fP\.
+.LP
+.TP
+\fB--ignore-port\fR \fIPORT\fP
+Ignore the port number\.
+.LP
+.HP
+\fB--ignore-ports-like\fR \[sl]\fIREGEX\fP\[sl]
+Ignore the port numbers matching the \fIREGEXP\fP\.
+.LP
+.TP
+\fB--visit-link\fR \fIURL\fP
+Visit the \fIURL\fP\.
+.LP
+.HP
+\fB--visit-links-like\fR \[sl]\fIREGEX\fP\[sl]
+Visit URLs that match the \fIREGEX\fP\.
+.LP
+.TP
+\fB--ignore-link\fR \fIURL\fP
+Ignore the \fIURL\fP\.
+.LP
+.HP
+\fB--ignore-links-like\fR \[sl]\fIREGEX\fP\[sl]
+Ignore URLs matching the \fIREGEX\fP\.
+.LP
+.TP
+\fB--visit-ext\fR \fIFILE\[ru]EXT\fP
+Visit URLs with the matching file ext\.
+.LP
+.HP
+\fB--visit-exts-like\fR \[sl]\fIREGEX\fP\[sl]
+Visit URLs with file exts that match the \fIREGEX\fP\.
+.LP
+.TP
+\fB--ignore-ext\fR \fIFILE\[ru]EXT\fP
+Ignore the URLs with the file ext\.
+.LP
+.HP
+\fB--ignore-exts-like\fR \[sl]\fIREGEX\fP\[sl]
+Ignore URLs with file exts matching the REGEX\.
+.LP
+.TP
+\fB-r\fR, \fB--robots\fR
+Specifies whether to honor \fBrobots.txt\fR\.
+.LP
+.TP
+\fB--host\fR \fIHOST\fP
+Spiders the specific \fIHOST\fP\.
+.LP
+.TP
+\fB--domain\fR \fIDOMAIN\fP
+Spiders the whole \fIDOMAIN\fP\.
+.LP
+.TP
+\fB--site\fR \fIURL\fP
+Spiders the website, starting at the \fIURL\fP\.
+.LP
+.TP
+\fB--print-status\fR
+Print the status codes for each URL\.
+.LP
+.TP
+\fB--print-headers\fR
+Print response headers for each URL\.
+.LP
+.TP
+\fB--print-header\fR \fINAME\fP
+Prints a specific header\.
+.LP
+.TP
+\fB--archive\fR \fIDIR\fP
+Archive every visited page to the \fIDIR\fP\.
+.LP
+.TP
+\fB--git-archive\fR \fIDIR\fP
+Archive every visited page to the git repository\.
+.LP
+.TP
+\fB-X\fR, \fB--xpath\fR \fIXPATH\fP
+Evaluates the XPath on each HTML page\.
+.LP
+.TP
+\fB-C\fR, \fB--css-path\fR \fIXPATH\fP
+Evaluates the CSS\-path on each HTML page\.
+.LP
+.TP
+\fB-v\fR, \fB--verbose\fR
+Enables verbose output\.
+.LP
+.TP
+\fB-h\fR, \fB--help\fR
+Print help information\.
+.LP
+.SH ENVIRONMENT
+.LP
+.TP
+\fIHTTP\[ru]PROXY\fP
+Sets the global HTTP proxy\.
+.LP
+.TP
+\fIRONIN\[ru]HTTP\[ru]PROXY\fP
+Sets the HTTP proxy for Ronin\.
+.LP
+.SH AUTHOR
+.LP
+.PP
+Postmodern 
+.MT postmodern\.mod3\[at]gmail\.com
+.ME
+.LP
+.SH SEE ALSO
+.LP
+.PP
+ronin\-web\-server(1) ronin\-web\-proxy(1) ronin\-web\-diff(1) ronin\-web\-new\-spider(1)

data/man/ronin-web-spider.1.md

@@ -0,0 +1,168 @@
+# ronin-web-spider 1 "2022-01-01" Ronin Web "User Manuals"
+
+## SYNOPSIS
+
+`ronin-web-spider` [*options*] {`--host` *HOST* \| `--domain` *DOMAIN* \| `--site` *URL*}
+
+## DESCRIPTION
+
+Spiders a website.
+
+## OPTIONS
+
+`--open-timeout` *SECS*
+  Sets the connection open timeout.
+
+`--read-timeout` *SECS*
+  Sets the read timeout.
+
+`--ssl-timeout` *SECS*
+  Sets the SSL connection timeout.
+
+`--continue-timeout` *SECS*
+  Sets the continue timeout.
+
+`--keep-alive-timeout` *SECS*
+  Sets the connection keep alive timeout.
+
+`-P`, `--proxy` *PROXY*
+  Sets the proxy to use.
+
+`-H`, `--header` "*NAME*: *VALUE*"
+  Sets a default header.
+
+`--host-header` *NAME*=*VALUE*
+  Sets a default header.
+
+`-u`, `--user-agent` chrome_linux|chrome_macos|chrome_windows|chrome_iphone|chrome_ipad|chrome_android|firefox_linux|firefox_macos|firefox_windows|firefox_iphone|firefox_ipad|firefox_android|safari_macos|safari_iphone|safari_ipad|edge
+  The `User-Agent` to use.
+
+`-U`, `--user-agent-string` *STRING*
+  The raw `User-Agent` string to use.
+
+`-R`, `--referer` *URL*
+  Sets the `Referer` URL.
+
+`--delay` *SECS*
+  Sets the delay in seconds between each request.
+
+`-l`, `--limit` *COUNT*
+  Only spiders up to *COUNT* pages.
+
+`-d`, `--max-depth` *DEPTH*
+  Only spiders up to max depth.
+
+`--enqueue` *URL*
+  Adds the URL to the queue.
+
+`--visited` *URL*
+  Marks the URL as previously visited.
+
+`--strip-fragments`
+  Enables/disables stripping the fragment component of every URL.
+
+`--strip-query`
+  Enables/disables stripping the query component of every URL.
+
+`--visit-host` *HOST*
+  Visit URLs with the matching host name.
+
+`--visit-hosts-like` /*REGEX*/
+  Visit URLs with hostnames that match the *REGEX*.
+
+`--ignore-host` *HOST*
+  Ignore the host name.
+
+`--ignore-hosts-like` /*REGEX*/
+  Ignore the host names matching the *REGEX*.
+
+`--visit-port` *PORT*
+  Visit URLs with the matching port number.
+
+`--visit-ports-like` /*REGEX*/
+  Visit URLs with port numbers that match the *REGEX*.
+
+`--ignore-port` *PORT*
+  Ignore the port number.
+
+`--ignore-ports-like` /*REGEX*/
+  Ignore the port numbers matching the *REGEXP*.
+
+`--visit-link` *URL*
+  Visit the *URL*.
+
+`--visit-links-like` /*REGEX*/
+  Visit URLs that match the *REGEX*.
+
+`--ignore-link` *URL*
+  Ignore the *URL*.
+
+`--ignore-links-like` /*REGEX*/
+  Ignore URLs matching the *REGEX*.
+
+`--visit-ext` *FILE_EXT*
+  Visit URLs with the matching file ext.
+
+`--visit-exts-like` /*REGEX*/
+  Visit URLs with file exts that match the *REGEX*.
+
+`--ignore-ext` *FILE_EXT*
+  Ignore the URLs with the file ext.
+
+`--ignore-exts-like` /*REGEX*/
+  Ignore URLs with file exts matching the REGEX.
+
+`-r`, `--robots`
+  Specifies whether to honor `robots.txt`.
+
+`--host` *HOST*
+  Spiders the specific *HOST*.
+
+`--domain` *DOMAIN*
+  Spiders the whole *DOMAIN*.
+
+`--site` *URL*
+  Spiders the website, starting at the *URL*.
+
+`--print-status`
+  Print the status codes for each URL.
+
+`--print-headers`
+  Print response headers for each URL.
+
+`--print-header` *NAME*
+  Prints a specific header.
+
+`--archive` *DIR*
+  Archive every visited page to the *DIR*.
+
+`--git-archive` *DIR*
+  Archive every visited page to the git repository.
+
+`-X`, `--xpath` *XPATH*
+  Evaluates the XPath on each HTML page.
+
+`-C`, `--css-path` *XPATH*
+  Evaluates the CSS-path on each HTML page.
+
+`-v`, `--verbose`
+  Enables verbose output.
+
+`-h`, `--help`
+  Print help information.
+
+## ENVIRONMENT
+
+*HTTP_PROXY*
+	Sets the global HTTP proxy.
+
+*RONIN_HTTP_PROXY*
+    Sets the HTTP proxy for Ronin.
+
+## AUTHOR
+
+Postmodern <postmodern.mod3@gmail.com>
+
+## SEE ALSO
+
+ronin-web-server(1) ronin-web-proxy(1) ronin-web-diff(1) ronin-web-new-spider(1)

data/man/ronin-web.1

@@ -0,0 +1,41 @@
+.\" Generated by kramdown-man 0.1.8
+.\" https://github.com/postmodern/kramdown-man#readme
+.TH ronin-web 1 "2022-01-01" Ronin Web "User Manuals"
+.LP
+.SH SYNOPSIS
+.LP
+.HP
+\fBronin-web\fR \[lB]\fIoptions\fP\[rB] \[lB]\fICOMMAND\fP \[lB]\fIARGS\fP\[rB]\[rB]
+.LP
+.SH DESCRIPTION
+.LP
+.PP
+A collection of useful web commands\.
+.LP
+.SH ARGUMENTS
+.LP
+.TP
+\fICOMMAND\fP
+The sub\-command to execute\.
+.LP
+.TP
+\fIARGS\fP
+Additional arguments for the sub\-command\.
+.LP
+.SH OPTIONS
+.LP
+.TP
+\fB-h\fR, \fB--help\fR
+Prints help information\.
+.LP
+.SH AUTHOR
+.LP
+.PP
+Postmodern 
+.MT postmodern\.mod3\[at]gmail\.com
+.ME
+.LP
+.SH SEE ALSO
+.LP
+.PP
+ronin\-web\-diff(1) ronin\-web\-html(1) ronin\-web\-server(1) ronin\-web\-spider(1) ronin\-web\-proxy(1)

data/man/ronin-web.1.md

@@ -0,0 +1,30 @@
+# ronin-web 1 "2022-01-01" Ronin Web "User Manuals"
+
+## SYNOPSIS
+
+`ronin-web` [*options*] [*COMMAND* [*ARGS*]]
+
+## DESCRIPTION
+
+A collection of useful web commands.
+
+## ARGUMENTS
+
+*COMMAND*
+  The sub-command to execute.
+
+*ARGS*
+  Additional arguments for the sub-command.
+
+## OPTIONS
+
+`-h`, `--help`
+  Prints help information.
+
+## AUTHOR
+
+Postmodern <postmodern.mod3@gmail.com>
+
+## SEE ALSO
+
+ronin-web-diff(1) ronin-web-html(1) ronin-web-server(1) ronin-web-spider(1) ronin-web-proxy(1)