clean-web-scraper 3.2.5 → 3.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -10
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -98,7 +98,7 @@ Your AI-ready content is saved in a clean, structured format:
|
|
|
98
98
|
- 📁 Base folder: `./folderPath/example.com/`
|
|
99
99
|
- 📑 Files preserve original URL paths
|
|
100
100
|
- 🤖 No HTML, no noise - just clean, structured text (`.txt` files)
|
|
101
|
-
- 📊 `JSONL` and `CSV` outputs, ready for AI consumption
|
|
101
|
+
- 📊 `JSONL` and `CSV` outputs, ready for AI consumption, model training and fine-tuning
|
|
102
102
|
|
|
103
103
|
```bash
|
|
104
104
|
example.com/
|
|
@@ -197,15 +197,6 @@ text,title,author,description
|
|
|
197
197
|
"Another article","Second Page","Jane Smith","Another description"
|
|
198
198
|
```
|
|
199
199
|
|
|
200
|
-
## 🤖 AI/LLM Training Ready
|
|
201
|
-
|
|
202
|
-
The output is specifically formatted for AI training and fine-tuning purposes:
|
|
203
|
-
|
|
204
|
-
- Clean, processed text without HTML markup
|
|
205
|
-
- Multiple formats (JSONL, CSV, text files)
|
|
206
|
-
- Structured content perfect for fine-tuning LLMs
|
|
207
|
-
- Ready to use in your ML pipelines
|
|
208
|
-
|
|
209
200
|
## Standing with Palestine 🇵🇸
|
|
210
201
|
|
|
211
202
|
This project supports Palestinian rights and stands in solidarity with Palestine. We believe in the importance of documenting and preserving Palestinian narratives, history, and struggles for justice and liberation.
|