single-file-cli 1.1.54 → 2.0.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (43) hide show
  1. package/{.eslintrc.js → .eslintrc.json} +5 -6
  2. package/Dockerfile +2 -1
  3. package/README.MD +56 -60
  4. package/args.js +252 -334
  5. package/back-ends/chromium-back-end.js +223 -0
  6. package/back-ends/chromium-browser.js +89 -0
  7. package/back-ends/chromium-constants.js +157 -0
  8. package/back-ends/{common/scripts.js → single-file-script.js} +21 -49
  9. package/build-lib-dev.sh +84 -0
  10. package/build-lib.sh +80 -3
  11. package/compile.sh +13 -0
  12. package/deno-polyfill.js +223 -0
  13. package/deno.json +13 -0
  14. package/deno.lock +49 -0
  15. package/lib/single-file-bundle.js +1 -0
  16. package/package.json +9 -33
  17. package/single-file +39 -14
  18. package/single-file-cli-api.js +32 -44
  19. package/back-ends/extensions/bypass-csp/index.js +0 -36
  20. package/back-ends/extensions/bypass-csp/manifest.json +0 -20
  21. package/back-ends/extensions/disable-web-security/index.js +0 -54
  22. package/back-ends/extensions/disable-web-security/manifest.json +0 -20
  23. package/back-ends/extensions/network-idle/bg.js +0 -61
  24. package/back-ends/extensions/network-idle/content.js +0 -29
  25. package/back-ends/extensions/network-idle/manifest.json +0 -31
  26. package/back-ends/extensions/signed/bypass_csp-0.0.3-an+fx.xpi +0 -0
  27. package/back-ends/extensions/signed/disable_web_security-0.0.3-an+fx.xpi +0 -0
  28. package/back-ends/extensions/signed/network_idle-0.0.2-an+fx.xpi +0 -0
  29. package/back-ends/jsdom.js +0 -159
  30. package/back-ends/playwright-chromium.js +0 -126
  31. package/back-ends/playwright-firefox.js +0 -115
  32. package/back-ends/playwright-webkit.js +0 -126
  33. package/back-ends/puppeteer-firefox.js +0 -172
  34. package/back-ends/puppeteer.js +0 -201
  35. package/back-ends/webdriver-chromium.js +0 -172
  36. package/back-ends/webdriver-gecko.js +0 -190
  37. package/lib/single-file-bootstrap.js +0 -1
  38. package/lib/single-file-frames.js +0 -1
  39. package/lib/single-file-hooks-frames.js +0 -1
  40. package/lib/single-file-zip.min.js +0 -1
  41. package/lib/single-file.js +0 -1
  42. package/rollup.config.dev.js +0 -57
  43. package/rollup.config.js +0 -58
@@ -1,6 +1,4 @@
1
- /* global module */
2
-
3
- module.exports = {
1
+ {
4
2
  "env": {
5
3
  "es6": true,
6
4
  "node": false,
@@ -11,7 +9,7 @@ module.exports = {
11
9
  },
12
10
  "extends": "eslint:recommended",
13
11
  "parserOptions": {
14
- "ecmaVersion": 2017,
12
+ "ecmaVersion": 2022,
15
13
  "sourceType": "module"
16
14
  },
17
15
  "ignorePatterns": [
@@ -20,7 +18,8 @@ module.exports = {
20
18
  "rules": {
21
19
  "indent": [
22
20
  "error",
23
- "tab", {
21
+ "tab",
22
+ {
24
23
  "SwitchCase": 1
25
24
  }
26
25
  ],
@@ -40,4 +39,4 @@ module.exports = {
40
39
  "warn"
41
40
  ]
42
41
  }
43
- };
42
+ }
package/Dockerfile CHANGED
@@ -5,8 +5,9 @@ RUN npm install --production single-file-cli
5
5
  WORKDIR /usr/src/app/node_modules/single-file-cli
6
6
 
7
7
  ENTRYPOINT [ \
8
+ "node", \
8
9
  "./single-file", \
9
10
  "--browser-executable-path", "/usr/bin/chromium-browser", \
10
11
  "--output-directory", "./../../out/", \
11
12
  "--browser-args", "[\"--no-sandbox\"]", \
12
- "--dump-content" ]
13
+ "--dump-content" ]
package/README.MD CHANGED
@@ -2,13 +2,11 @@
2
2
 
3
3
  ## Introduction
4
4
 
5
- SingleFile can be launched from the command line by running it into a (headless)
6
- browser. It runs through Node.js as a standalone script injected into the web
7
- page instead of being embedded into a WebExtension. To connect to the browser,
8
- it can use [Puppeteer](https://github.com/GoogleChrome/puppeteer) or
9
- [Selenium WebDriver](https://www.npmjs.com/package/selenium-webdriver).
10
- Alternatively, it can also emulate a browser with JavaScript disabled by using
11
- [jsdom](https://github.com/jsdom/jsdom).
5
+ SingleFile can be launched from the command line by running it into a (headless) browser.
6
+
7
+ It runs through Deno as a standalone script injected into the web page via the Chrome DevTools Protocol instead of being embedded into a WebExtension.
8
+
9
+
12
10
 
13
11
  ## Installation with Docker
14
12
 
@@ -59,105 +57,103 @@ Alternatively, it can also emulate a browser with JavaScript disabled by using
59
57
 
60
58
  ## Manual installation
61
59
 
62
- - Make sure Chrome or Firefox is installed and the executable can be found
63
- through the `PATH` environment variable. Otherwise you will need to set the
64
- `--browser-executable-path` option to help SingleFile locating it. As an
65
- alternative to Chrome and Firefox, you can use jsdom by setting the
66
- `--back-end` option to `jsdom`.
67
-
68
- - Install [Node.js](https://nodejs.org)
69
-
70
- - There are 3 ways to download the code of SingleFile, choose the one you prefer
71
- (`npm` is installed with Node.js):
60
+ - Make sure Chrome or a Chromium-based browser is installed. Otherwise you might need to set the `--browser-executable-path` option to help SingleFile locating the path of the executable file.
72
61
 
73
- - Download and install globally with `npm`
62
+ - Install [Deno](https://deno.com/)
74
63
 
75
- `npm install -g "single-file-cli"`
64
+ - There are 2 ways to download the code of SingleFile, choose the one you prefer:
76
65
 
77
66
  - Download and unzip manually the
78
67
  [master archive](https://github.com/gildas-lormeau/single-file-cli/archive/master.zip)
79
68
  provided by Github
80
69
 
81
- `unzip master.zip .`
70
+ ```sh
71
+ unzip master.zip .
72
+ ```
82
73
 
83
- `cd single-file-cli-master`
84
-
85
- `npm install`
74
+ ```sh
75
+ cd single-file-cli-master
76
+ ```
86
77
 
87
78
  - Download with `git`
88
79
 
89
- `git clone --depth 1 --recursive https://github.com/gildas-lormeau/single-file-cli.git`
90
-
91
- `cd single-file-cli`
80
+ ```sh
81
+ git clone --depth 1 --recursive https://github.com/gildas-lormeau/single-file-cli.git
82
+ ```
92
83
 
93
- `npm install`
84
+ ```sh
85
+ cd single-file-cli
86
+ ```
94
87
 
95
- - Make `single-file` executable (Linux/Unix/BSD etc.) if SingleFile is not
96
- installed globally.
88
+ - Make `single-file` executable (Linux/Unix/BSD etc.).
97
89
 
98
- `chmod +x single-file`
99
-
100
- - To use Firefox instead of Chrome, you must download the
101
- [Selenium WebDriver](https://www.npmjs.com/package/selenium-webdriver)
102
- component (i.e. `geckodriver` for Firefox). Make sure it can be found through
103
- the `PATH` environment variable or the `cli` folder. Otherwise you will need
104
- to set the `--web-driver-executable-path` option to help WebDriver locating
105
- the executable.
90
+ ```sh
91
+ chmod +x single-file
92
+ ```
106
93
 
107
94
  ## Run
108
95
 
109
96
  - Syntax
110
97
 
111
- `single-file <url> [output] [options ...]`
98
+ ```sh
99
+ single-file <url> [output] [options ...]
100
+ ```
112
101
 
113
102
  - Display help
114
103
 
115
- `single-file --help`
104
+ ```sh
105
+ single-file --help
106
+ ```
116
107
 
117
108
  - Examples
118
109
 
119
- - Dump the processed content of https://www.wikipedia.org into the console
110
+ - Dump the HTML content of https://www.wikipedia.org into the console
120
111
 
121
- `single-file https://www.wikipedia.org --dump-content`
112
+ ```sh
113
+ single-file https://www.wikipedia.org --dump-content
114
+ ```
122
115
 
123
116
  - Save https://www.wikipedia.org into `wikipedia.html` in the current folder
124
117
 
125
- `single-file https://www.wikipedia.org wikipedia.html`
118
+ ```sh
119
+ single-file https://www.wikipedia.org wikipedia.html
120
+ ```
126
121
 
127
122
  - Save https://www.wikipedia.org into `wikipedia.html` in the current folder
128
123
  with Firefox instead of Chrome
129
124
 
130
- `single-file https://www.wikipedia.org wikipedia.html --back-end=webdriver-gecko`
125
+ ```sh
126
+ single-file https://www.wikipedia.org wikipedia.html --back-end=webdriver-gecko
127
+ ```
131
128
 
132
129
  - Save a list of URLs stored into `list-urls.txt` in the current folder
133
130
 
134
- `single-file --urls-file=list-urls.txt`
131
+ ```sh
132
+ single-file --urls-file=list-urls.txt
133
+ ```
135
134
 
136
135
  - Save https://www.wikipedia.org and crawl its internal links with the query
137
136
  parameters removed from the URL
138
137
 
139
- `single-file https://www.wikipedia.org --crawl-links=true --crawl-inner-links-only=true --crawl-max-depth=1 --crawl-rewrite-rule="^(.*)\\?.*$ $1"`
138
+ ```sh
139
+ single-file https://www.wikipedia.org --crawl-links=true --crawl-inner-links-only=true --crawl-max-depth=1 --crawl-rewrite-rule="^(.*)\\?.*$ $1"
140
+ ```
140
141
 
141
142
  - Save https://www.wikipedia.org and external links only
142
143
 
143
- `single-file https://www.wikipedia.org --crawl-links=true --crawl-inner-links-only=false --crawl-external-links-max-depth=1 --crawl-rewrite-rule="^.*wikipedia.*$"`
144
+ ```sh
145
+ single-file https://www.wikipedia.org --crawl-links=true --crawl-inner-links-only=false --crawl-external-links-max-depth=1 --crawl-rewrite-rule="^.*wikipedia.*$"
146
+ ```
147
+
148
+ ## Compile executables
144
149
 
145
- ## Troubleshooting
150
+ - Compile executables into `/dist`
146
151
 
147
- - If the error message
148
- `UnhandledPromiseRejectionWarning: Error: Browser is not downloaded. Run "npm install" or "yarn install" at ChromeLauncher.launch`
149
- is displayed, it probably means that `single-file` was not able to find the
150
- executable of the browser. Using the option `--browser-executable-path` to
151
- pass to `single-file` the complete path of the executable fixes this issue.
152
+ ```sh
153
+ ./compile.sh
154
+ ```
152
155
 
153
- - If saving a page takes an unusually long time, this may be due to a timeout
154
- error that was automatically recovered. Setting `--browser-wait-until` to a
155
- lower value (e.g. `networkidle0` or `load` instead of `networkidle2`) fixes
156
- this issue.
157
156
 
158
157
  ## License
159
158
 
160
- SingleFile is licensed under AGPL. Code derived from third-party projects is
161
- licensed under MIT. Please contact me at gildas.lormeau &lt;at&gt; gmail.com if
162
- you are interested in licensing the SingleFile code for a commercial service or
163
- product.
159
+ SingleFile and SingleFile CLI are licensed under AGPL. Code derived from third-party projects is licensed under MIT. Please contact me at gildas.lormeau &lt;at&gt; gmail.com if you are interested in licensing the SingleFile code for a commercial service or product.