single-file-cli 1.1.54 → 2.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/{.eslintrc.js → .eslintrc.json} +5 -6
- package/Dockerfile +2 -1
- package/README.MD +56 -60
- package/args.js +252 -334
- package/back-ends/chromium-back-end.js +223 -0
- package/back-ends/chromium-browser.js +89 -0
- package/back-ends/chromium-constants.js +157 -0
- package/back-ends/{common/scripts.js → single-file-script.js} +21 -49
- package/build-lib-dev.sh +84 -0
- package/build-lib.sh +80 -3
- package/compile.sh +13 -0
- package/deno-polyfill.js +223 -0
- package/deno.json +13 -0
- package/deno.lock +49 -0
- package/lib/single-file-bundle.js +1 -0
- package/package.json +9 -33
- package/single-file +39 -14
- package/single-file-cli-api.js +32 -44
- package/back-ends/extensions/bypass-csp/index.js +0 -36
- package/back-ends/extensions/bypass-csp/manifest.json +0 -20
- package/back-ends/extensions/disable-web-security/index.js +0 -54
- package/back-ends/extensions/disable-web-security/manifest.json +0 -20
- package/back-ends/extensions/network-idle/bg.js +0 -61
- package/back-ends/extensions/network-idle/content.js +0 -29
- package/back-ends/extensions/network-idle/manifest.json +0 -31
- package/back-ends/extensions/signed/bypass_csp-0.0.3-an+fx.xpi +0 -0
- package/back-ends/extensions/signed/disable_web_security-0.0.3-an+fx.xpi +0 -0
- package/back-ends/extensions/signed/network_idle-0.0.2-an+fx.xpi +0 -0
- package/back-ends/jsdom.js +0 -159
- package/back-ends/playwright-chromium.js +0 -126
- package/back-ends/playwright-firefox.js +0 -115
- package/back-ends/playwright-webkit.js +0 -126
- package/back-ends/puppeteer-firefox.js +0 -172
- package/back-ends/puppeteer.js +0 -201
- package/back-ends/webdriver-chromium.js +0 -172
- package/back-ends/webdriver-gecko.js +0 -190
- package/lib/single-file-bootstrap.js +0 -1
- package/lib/single-file-frames.js +0 -1
- package/lib/single-file-hooks-frames.js +0 -1
- package/lib/single-file-zip.min.js +0 -1
- package/lib/single-file.js +0 -1
- package/rollup.config.dev.js +0 -57
- package/rollup.config.js +0 -58
|
@@ -1,6 +1,4 @@
|
|
|
1
|
-
|
|
2
|
-
|
|
3
|
-
module.exports = {
|
|
1
|
+
{
|
|
4
2
|
"env": {
|
|
5
3
|
"es6": true,
|
|
6
4
|
"node": false,
|
|
@@ -11,7 +9,7 @@ module.exports = {
|
|
|
11
9
|
},
|
|
12
10
|
"extends": "eslint:recommended",
|
|
13
11
|
"parserOptions": {
|
|
14
|
-
"ecmaVersion":
|
|
12
|
+
"ecmaVersion": 2022,
|
|
15
13
|
"sourceType": "module"
|
|
16
14
|
},
|
|
17
15
|
"ignorePatterns": [
|
|
@@ -20,7 +18,8 @@ module.exports = {
|
|
|
20
18
|
"rules": {
|
|
21
19
|
"indent": [
|
|
22
20
|
"error",
|
|
23
|
-
"tab",
|
|
21
|
+
"tab",
|
|
22
|
+
{
|
|
24
23
|
"SwitchCase": 1
|
|
25
24
|
}
|
|
26
25
|
],
|
|
@@ -40,4 +39,4 @@ module.exports = {
|
|
|
40
39
|
"warn"
|
|
41
40
|
]
|
|
42
41
|
}
|
|
43
|
-
}
|
|
42
|
+
}
|
package/Dockerfile
CHANGED
|
@@ -5,8 +5,9 @@ RUN npm install --production single-file-cli
|
|
|
5
5
|
WORKDIR /usr/src/app/node_modules/single-file-cli
|
|
6
6
|
|
|
7
7
|
ENTRYPOINT [ \
|
|
8
|
+
"node", \
|
|
8
9
|
"./single-file", \
|
|
9
10
|
"--browser-executable-path", "/usr/bin/chromium-browser", \
|
|
10
11
|
"--output-directory", "./../../out/", \
|
|
11
12
|
"--browser-args", "[\"--no-sandbox\"]", \
|
|
12
|
-
"--dump-content" ]
|
|
13
|
+
"--dump-content" ]
|
package/README.MD
CHANGED
|
@@ -2,13 +2,11 @@
|
|
|
2
2
|
|
|
3
3
|
## Introduction
|
|
4
4
|
|
|
5
|
-
SingleFile can be launched from the command line by running it into a (headless)
|
|
6
|
-
|
|
7
|
-
page instead of being embedded into a WebExtension.
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
Alternatively, it can also emulate a browser with JavaScript disabled by using
|
|
11
|
-
[jsdom](https://github.com/jsdom/jsdom).
|
|
5
|
+
SingleFile can be launched from the command line by running it into a (headless) browser.
|
|
6
|
+
|
|
7
|
+
It runs through Deno as a standalone script injected into the web page via the Chrome DevTools Protocol instead of being embedded into a WebExtension.
|
|
8
|
+
|
|
9
|
+
|
|
12
10
|
|
|
13
11
|
## Installation with Docker
|
|
14
12
|
|
|
@@ -59,105 +57,103 @@ Alternatively, it can also emulate a browser with JavaScript disabled by using
|
|
|
59
57
|
|
|
60
58
|
## Manual installation
|
|
61
59
|
|
|
62
|
-
- Make sure Chrome or
|
|
63
|
-
through the `PATH` environment variable. Otherwise you will need to set the
|
|
64
|
-
`--browser-executable-path` option to help SingleFile locating it. As an
|
|
65
|
-
alternative to Chrome and Firefox, you can use jsdom by setting the
|
|
66
|
-
`--back-end` option to `jsdom`.
|
|
67
|
-
|
|
68
|
-
- Install [Node.js](https://nodejs.org)
|
|
69
|
-
|
|
70
|
-
- There are 3 ways to download the code of SingleFile, choose the one you prefer
|
|
71
|
-
(`npm` is installed with Node.js):
|
|
60
|
+
- Make sure Chrome or a Chromium-based browser is installed. Otherwise you might need to set the `--browser-executable-path` option to help SingleFile locating the path of the executable file.
|
|
72
61
|
|
|
73
|
-
|
|
62
|
+
- Install [Deno](https://deno.com/)
|
|
74
63
|
|
|
75
|
-
|
|
64
|
+
- There are 2 ways to download the code of SingleFile, choose the one you prefer:
|
|
76
65
|
|
|
77
66
|
- Download and unzip manually the
|
|
78
67
|
[master archive](https://github.com/gildas-lormeau/single-file-cli/archive/master.zip)
|
|
79
68
|
provided by Github
|
|
80
69
|
|
|
81
|
-
|
|
70
|
+
```sh
|
|
71
|
+
unzip master.zip .
|
|
72
|
+
```
|
|
82
73
|
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
74
|
+
```sh
|
|
75
|
+
cd single-file-cli-master
|
|
76
|
+
```
|
|
86
77
|
|
|
87
78
|
- Download with `git`
|
|
88
79
|
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
80
|
+
```sh
|
|
81
|
+
git clone --depth 1 --recursive https://github.com/gildas-lormeau/single-file-cli.git
|
|
82
|
+
```
|
|
92
83
|
|
|
93
|
-
|
|
84
|
+
```sh
|
|
85
|
+
cd single-file-cli
|
|
86
|
+
```
|
|
94
87
|
|
|
95
|
-
- Make `single-file` executable (Linux/Unix/BSD etc.)
|
|
96
|
-
installed globally.
|
|
88
|
+
- Make `single-file` executable (Linux/Unix/BSD etc.).
|
|
97
89
|
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
[Selenium WebDriver](https://www.npmjs.com/package/selenium-webdriver)
|
|
102
|
-
component (i.e. `geckodriver` for Firefox). Make sure it can be found through
|
|
103
|
-
the `PATH` environment variable or the `cli` folder. Otherwise you will need
|
|
104
|
-
to set the `--web-driver-executable-path` option to help WebDriver locating
|
|
105
|
-
the executable.
|
|
90
|
+
```sh
|
|
91
|
+
chmod +x single-file
|
|
92
|
+
```
|
|
106
93
|
|
|
107
94
|
## Run
|
|
108
95
|
|
|
109
96
|
- Syntax
|
|
110
97
|
|
|
111
|
-
|
|
98
|
+
```sh
|
|
99
|
+
single-file <url> [output] [options ...]
|
|
100
|
+
```
|
|
112
101
|
|
|
113
102
|
- Display help
|
|
114
103
|
|
|
115
|
-
|
|
104
|
+
```sh
|
|
105
|
+
single-file --help
|
|
106
|
+
```
|
|
116
107
|
|
|
117
108
|
- Examples
|
|
118
109
|
|
|
119
|
-
- Dump the
|
|
110
|
+
- Dump the HTML content of https://www.wikipedia.org into the console
|
|
120
111
|
|
|
121
|
-
|
|
112
|
+
```sh
|
|
113
|
+
single-file https://www.wikipedia.org --dump-content
|
|
114
|
+
```
|
|
122
115
|
|
|
123
116
|
- Save https://www.wikipedia.org into `wikipedia.html` in the current folder
|
|
124
117
|
|
|
125
|
-
|
|
118
|
+
```sh
|
|
119
|
+
single-file https://www.wikipedia.org wikipedia.html
|
|
120
|
+
```
|
|
126
121
|
|
|
127
122
|
- Save https://www.wikipedia.org into `wikipedia.html` in the current folder
|
|
128
123
|
with Firefox instead of Chrome
|
|
129
124
|
|
|
130
|
-
|
|
125
|
+
```sh
|
|
126
|
+
single-file https://www.wikipedia.org wikipedia.html --back-end=webdriver-gecko
|
|
127
|
+
```
|
|
131
128
|
|
|
132
129
|
- Save a list of URLs stored into `list-urls.txt` in the current folder
|
|
133
130
|
|
|
134
|
-
|
|
131
|
+
```sh
|
|
132
|
+
single-file --urls-file=list-urls.txt
|
|
133
|
+
```
|
|
135
134
|
|
|
136
135
|
- Save https://www.wikipedia.org and crawl its internal links with the query
|
|
137
136
|
parameters removed from the URL
|
|
138
137
|
|
|
139
|
-
|
|
138
|
+
```sh
|
|
139
|
+
single-file https://www.wikipedia.org --crawl-links=true --crawl-inner-links-only=true --crawl-max-depth=1 --crawl-rewrite-rule="^(.*)\\?.*$ $1"
|
|
140
|
+
```
|
|
140
141
|
|
|
141
142
|
- Save https://www.wikipedia.org and external links only
|
|
142
143
|
|
|
143
|
-
|
|
144
|
+
```sh
|
|
145
|
+
single-file https://www.wikipedia.org --crawl-links=true --crawl-inner-links-only=false --crawl-external-links-max-depth=1 --crawl-rewrite-rule="^.*wikipedia.*$"
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
## Compile executables
|
|
144
149
|
|
|
145
|
-
|
|
150
|
+
- Compile executables into `/dist`
|
|
146
151
|
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
executable of the browser. Using the option `--browser-executable-path` to
|
|
151
|
-
pass to `single-file` the complete path of the executable fixes this issue.
|
|
152
|
+
```sh
|
|
153
|
+
./compile.sh
|
|
154
|
+
```
|
|
152
155
|
|
|
153
|
-
- If saving a page takes an unusually long time, this may be due to a timeout
|
|
154
|
-
error that was automatically recovered. Setting `--browser-wait-until` to a
|
|
155
|
-
lower value (e.g. `networkidle0` or `load` instead of `networkidle2`) fixes
|
|
156
|
-
this issue.
|
|
157
156
|
|
|
158
157
|
## License
|
|
159
158
|
|
|
160
|
-
SingleFile
|
|
161
|
-
licensed under MIT. Please contact me at gildas.lormeau <at> gmail.com if
|
|
162
|
-
you are interested in licensing the SingleFile code for a commercial service or
|
|
163
|
-
product.
|
|
159
|
+
SingleFile and SingleFile CLI are licensed under AGPL. Code derived from third-party projects is licensed under MIT. Please contact me at gildas.lormeau <at> gmail.com if you are interested in licensing the SingleFile code for a commercial service or product.
|