webforai 1.4.1 → 1.5.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (50) hide show
  1. package/LICENSE +1 -1
  2. package/README.md +61 -3
  3. package/dist/bin.js +1049 -0
  4. package/dist/cjs/cli/bin.js +34 -0
  5. package/dist/cjs/cli/commands/handlers/index.js +84 -0
  6. package/dist/cjs/cli/commands/handlers/loadHtml.js +127 -0
  7. package/dist/cjs/cli/constants.js +34 -0
  8. package/dist/cjs/cli/helpers/assertContinue.js +34 -0
  9. package/dist/cjs/cli/helpers/inputOutputPath.js +89 -0
  10. package/dist/cjs/cli/helpers/inputSourcePath.js +64 -0
  11. package/dist/cjs/cli/helpers/selectExtractMode.js +39 -0
  12. package/dist/cjs/cli/helpers/selectLoader.js +44 -0
  13. package/dist/cjs/cli/utils.js +122 -0
  14. package/dist/cjs/html-to-markdown.test.js +6 -34
  15. package/dist/cjs/index.js +5 -0
  16. package/dist/cli/bin.js +12 -0
  17. package/dist/cli/commands/handlers/index.js +51 -0
  18. package/dist/cli/commands/handlers/loadHtml.js +94 -0
  19. package/dist/cli/constants.js +9 -0
  20. package/dist/cli/helpers/assertContinue.js +11 -0
  21. package/dist/cli/helpers/inputOutputPath.js +56 -0
  22. package/dist/cli/helpers/inputSourcePath.js +31 -0
  23. package/dist/cli/helpers/selectExtractMode.js +16 -0
  24. package/dist/cli/helpers/selectLoader.js +21 -0
  25. package/dist/cli/utils.js +85 -0
  26. package/dist/html-to-markdown.test.js +6 -12
  27. package/dist/index.js +3 -0
  28. package/dist/types/cli/bin.d.ts +1 -0
  29. package/dist/types/cli/bin.js.map +1 -0
  30. package/dist/types/cli/commands/handlers/index.d.ts +13 -0
  31. package/dist/types/cli/commands/handlers/index.js.map +1 -0
  32. package/dist/types/cli/commands/handlers/loadHtml.d.ts +2 -0
  33. package/dist/types/cli/commands/handlers/loadHtml.js.map +1 -0
  34. package/dist/types/cli/constants.d.ts +5 -0
  35. package/dist/types/cli/constants.js.map +1 -0
  36. package/dist/types/cli/helpers/assertContinue.d.ts +1 -0
  37. package/dist/types/cli/helpers/assertContinue.js.map +1 -0
  38. package/dist/types/cli/helpers/inputOutputPath.d.ts +1 -0
  39. package/dist/types/cli/helpers/inputOutputPath.js.map +1 -0
  40. package/dist/types/cli/helpers/inputSourcePath.d.ts +1 -0
  41. package/dist/types/cli/helpers/inputSourcePath.js.map +1 -0
  42. package/dist/types/cli/helpers/selectExtractMode.d.ts +1 -0
  43. package/dist/types/cli/helpers/selectExtractMode.js.map +1 -0
  44. package/dist/types/cli/helpers/selectLoader.d.ts +1 -0
  45. package/dist/types/cli/helpers/selectLoader.js.map +1 -0
  46. package/dist/types/cli/utils.d.ts +5 -0
  47. package/dist/types/cli/utils.js.map +1 -0
  48. package/dist/types/index.d.ts +1 -0
  49. package/dist/types/index.js.map +1 -1
  50. package/package.json +11 -21
package/LICENSE CHANGED
@@ -1,6 +1,6 @@
1
1
  MIT License
2
2
 
3
- Copyright (c) 2023 inaridiy
3
+ Copyright (c) 2023 - present, inaridiy and webforai contributors
4
4
 
5
5
  Permission is hereby granted, free of charge, to any person obtaining a copy
6
6
  of this software and associated documentation files (the "Software"), to deal
package/README.md CHANGED
@@ -10,6 +10,7 @@ A library that provides a web interface for AI
10
10
  - Ultra-precise Markdown segmentation based on AST
11
11
  - Ultra-precise HTML retrieval functions using headless browsers
12
12
  - The core functionality is edge-native (runs on Cloudflare Workers!!!)
13
+ - Command-line interface for easy use without coding
13
14
 
14
15
  ## Demo
15
16
 
@@ -18,14 +19,25 @@ There is a demo API for Html2Markdown deployed on CloudflareWorker. Please acces
18
19
  - [NPM Package page](https://webforai.inaridiy.workers.dev/?url=https://www.npmjs.com/package/webforai)
19
20
  - [Wikipedia of Cloudflare (AI Mode)](https://webforai.inaridiy.workers.dev/?url=https://en.wikipedia.org/wiki/Cloudflare&mode=ai)
20
21
 
21
- ## Quick Start
22
-
23
- Just install and execute scripts
22
+ ## Installation
24
23
 
24
+ ### As a library
25
+ To use WebforAI as a library in your project, install it along with playwright:
25
26
  ```bash
26
27
  pnpm i webforai playwright
27
28
  ```
28
29
 
30
+ ### As a CLI tool
31
+ To use WebforAI as a command-line tool, install it globally:
32
+ ```bash
33
+ npm install -g webforai
34
+ ```
35
+
36
+ ## Quick Start
37
+
38
+ ### Using as a library
39
+ Just install and execute scripts
40
+
29
41
  ```js
30
42
  import { promises as fs } from "fs";
31
43
  import { htmlToMarkdown, htmlToMdast } from "webforai";
@@ -41,6 +53,18 @@ await fs.writeFile("output.md", markdown);
41
53
 
42
54
  other examples are in [examples](./examples/simple/src/index.ts)
43
55
 
56
+ ### Using as a CLI tool
57
+ After installing globally, you can use the webforai command:
58
+ ```bash
59
+ webforai https://www.npmjs.com/package/webforai output.md
60
+ ```
61
+ This will fetch the HTML from the specified URL, convert it to Markdown, and save it to output.md.
62
+
63
+ For more CLI options, run:
64
+ ```bash
65
+ webforai --help
66
+ ```
67
+
44
68
  ## Examples
45
69
 
46
70
  - [Simple Example](https://github.com/inaridiy/webforai/tree/main/examples/simple/src/index.ts)
@@ -68,3 +92,37 @@ Convert Mdast to Markdown. If `solveLinks` is specified, the relative links in t
68
92
 
69
93
  **`loadHtml(url: string, options?: LoadHtmlOptions): Promise<string>`**
70
94
  Load HTML from the specified URL. This function uses Playwright internally.
95
+
96
+ ### CLI Commands
97
+
98
+ **`webforai <source?> <outputPath?> [options]`**
99
+ Converts the HTML at the specified URL or path to Markdown and saves it to an output file.
100
+ Arguments and options can be specified in the interactive interface even if they are not specified.
101
+ - **`source`**
102
+ The URL or path to the HTML file to convert.
103
+ - **`outputPath`**
104
+ The path to save the output Markdown file.
105
+
106
+ #### Options
107
+ - **`--mode <mode>`**
108
+ Specify the mode to use for conversion. Options are `default` and `ai`. Default is `default`.
109
+ - **`--loader <loader>`**
110
+ Specify the loader to use for fetching HTML. Options are `fetch`, `playwright` and `puppeteer`. Default is `fetch`.
111
+ - **`--baseUrl <baseUrl>`**
112
+ Specify the base URL to use for relative links in the output Markdown.
113
+ - **`-o --stdout`**
114
+ Output the converted Markdown to the console instead of saving it to a file.
115
+ - **`-d --debug`**
116
+ Enable debug mode. This will output additional information to the console.
117
+
118
+ **`webforai --help`**
119
+ Displays help information for the CLI tool.
120
+
121
+ ## Contributing
122
+ 1. Fork it!
123
+ 2. Create your feature branch: `git checkout -b my-new-feature`
124
+ 3. Add your changes: `git add .`
125
+ 4. Commit your changes: `git commit -am 'Add some feature'`
126
+ 5. Add a changelog: `pnpm changeset`
127
+ 6. Push to the branch: `git push origin my-new-feature`
128
+ 7. Submit a pull request :sunglasses: