@abraca/wiki 2.27.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +81 -0
- package/dist/abracadabra-wiki.cjs +1418 -0
- package/dist/abracadabra-wiki.cjs.map +1 -0
- package/dist/abracadabra-wiki.esm.js +1387 -0
- package/dist/abracadabra-wiki.esm.js.map +1 -0
- package/dist/index.d.ts +27 -0
- package/package.json +44 -0
- package/src/connect.ts +69 -0
- package/src/crypto.ts +70 -0
- package/src/index.ts +508 -0
- package/src/parser.ts +62 -0
- package/src/render.ts +91 -0
- package/src/snapshot.ts +210 -0
- package/src/types.ts +45 -0
- package/src/wikipedia.ts +154 -0
package/README.md
ADDED
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
# @abraca/wiki
|
|
2
|
+
|
|
3
|
+
Streaming **Wikipedia → Abracadabra** importer. Fetches Wikipedia (or any
|
|
4
|
+
MediaWiki) articles and seeds them into a space as a live graph of CRDT docs:
|
|
5
|
+
every discovered title becomes a shell doc immediately, then bodies stream in
|
|
6
|
+
one fetch at a time — so the tree skeleton appears in the dashboard before the
|
|
7
|
+
first body is written.
|
|
8
|
+
|
|
9
|
+
> Extracted from `@abraca/cli` (where it used to be the `abracadabra wiki`
|
|
10
|
+
> command) so the heavyweight `wtf_wikipedia` dependency no longer ships with
|
|
11
|
+
> the core CLI.
|
|
12
|
+
|
|
13
|
+
## Install
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
pnpm add -g @abraca/wiki # or: pnpm dlx @abraca/wiki ...
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
Requires `@abraca/dabra` (peer). Authenticates with the same Ed25519 key file
|
|
20
|
+
as the `abracadabra` CLI (`~/.abracadabra/cli.key`), so both tools share one
|
|
21
|
+
account.
|
|
22
|
+
|
|
23
|
+
## Usage
|
|
24
|
+
|
|
25
|
+
```bash
|
|
26
|
+
abracadabra-wiki "<Article Title>" user-agent="you (you@example.com)" [options]
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
| Option | Default | Meaning |
|
|
30
|
+
| :-- | :-- | :-- |
|
|
31
|
+
| `mode=single\|split` | `split` | one doc per article, or split into sections + infobox |
|
|
32
|
+
| `depth=<n>` | `1` | follow internal links to depth N |
|
|
33
|
+
| `category-depth=<n>` | `1` | recurse into sub-categories |
|
|
34
|
+
| `lang=<code>` | `en` | wiki language |
|
|
35
|
+
| `domain=<host>` | — | 3rd-party MediaWiki host (overrides `lang`) |
|
|
36
|
+
| `parent=<docId>` | active space root | parent doc for the new graph |
|
|
37
|
+
| `user-agent=<str>` | — | **required** `Api-User-Agent` (Wikimedia etiquette) |
|
|
38
|
+
| `rate=<rps>` | `3` | max Wikipedia requests per second |
|
|
39
|
+
| `--include-categories` | off | expand each article's categories into nested graphs |
|
|
40
|
+
| `--dry-run` | off | fetch only the entry article, print its outline, write nothing |
|
|
41
|
+
|
|
42
|
+
### Environment
|
|
43
|
+
|
|
44
|
+
| Var | Purpose |
|
|
45
|
+
| :-- | :-- |
|
|
46
|
+
| `ABRA_URL` | server URL (required unless `--dry-run`) |
|
|
47
|
+
| `ABRA_KEY_FILE` | Ed25519 key seed (default `~/.abracadabra/cli.key`) |
|
|
48
|
+
| `ABRA_NAME` / `ABRA_COLOR` | presence identity |
|
|
49
|
+
| `ABRA_INVITE_CODE` | invite code for first-run registration |
|
|
50
|
+
| `ABRA_WIKI_USER_AGENT` | default `Api-User-Agent` (or pass `user-agent=`) |
|
|
51
|
+
|
|
52
|
+
### Examples
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
# Preview the structure without touching a server
|
|
56
|
+
abracadabra-wiki "Toronto Raptors" user-agent="me (me@example.com)" --dry-run
|
|
57
|
+
|
|
58
|
+
# Import two link-hops deep, split into section docs, with categories
|
|
59
|
+
export ABRA_URL=https://your-server.example.com
|
|
60
|
+
abracadabra-wiki "Jazz" depth=2 mode=split --include-categories \
|
|
61
|
+
user-agent="me (me@example.com)"
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
## Programmatic use
|
|
65
|
+
|
|
66
|
+
```ts
|
|
67
|
+
import { runWiki } from "@abraca/wiki";
|
|
68
|
+
|
|
69
|
+
const summary = await runWiki({
|
|
70
|
+
positional: ["Toronto Raptors"],
|
|
71
|
+
params: { "user-agent": "me (me@example.com)" },
|
|
72
|
+
flags: new Set(["dry-run"]),
|
|
73
|
+
});
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
`runWiki(args)` runs an import for already-parsed args and returns a summary
|
|
77
|
+
string. (The package is primarily a CLI; `abracadabra-wiki` is the bin.)
|
|
78
|
+
|
|
79
|
+
## License
|
|
80
|
+
|
|
81
|
+
MIT
|