shamela 1.0.6 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +247 -38
- package/dist/index.d.ts +303 -63
- package/dist/index.js +44 -0
- package/dist/index.js.map +1 -0
- package/package.json +29 -29
- package/dist/main.js +0 -629
- package/dist/main.js.map +0 -1
package/README.md
CHANGED
|
@@ -1,18 +1,18 @@
|
|
|
1
|
-
#
|
|
1
|
+
# shamela
|
|
2
2
|
|
|
3
3
|
[](https://wakatime.com/badge/user/a0b906ce-b8e7-4463-8bce-383238df6d4b/project/faef70ab-efdb-448b-ab83-0fc66c95888e)
|
|
4
4
|
[](https://github.com/ragaeeb/shamela/actions/workflows/e2e.yml)
|
|
5
5
|
[](https://github.com/ragaeeb/shamela/actions/workflows/build.yml) 
|
|
6
6
|

|
|
7
7
|
[](https://codecov.io/gh/ragaeeb/shamela)
|
|
8
|
-
[](https://bundlejs.com/?q=shamela%40latest)
|
|
9
9
|

|
|
10
10
|

|
|
11
11
|

|
|
12
12
|

|
|
13
13
|

|
|
14
14
|
|
|
15
|
-
A `
|
|
15
|
+
A `Node.js` library for accessing and downloading Maktabah Shamela v4 APIs. This library provides easy-to-use functions to interact with the Shamela API, download master and book databases, and retrieve book data programmatically.
|
|
16
16
|
|
|
17
17
|
## Table of Contents
|
|
18
18
|
|
|
@@ -26,15 +26,24 @@ A `NodeJS` library for accessing and downloading Maktabah Shamela v4 APIs. This
|
|
|
26
26
|
- [getBookMetadata](#getbookmetadata)
|
|
27
27
|
- [downloadBook](#downloadbook)
|
|
28
28
|
- [getBook](#getbook)
|
|
29
|
+
- [getCoverUrl](#getcoverurl)
|
|
29
30
|
- [Examples](#examples)
|
|
30
31
|
- [Downloading the Master Database](#downloading-the-master-database)
|
|
31
32
|
- [Downloading a Book](#downloading-a-book)
|
|
32
33
|
- [Retrieving Book Data](#retrieving-book-data)
|
|
34
|
+
- [Getting Book Cover URLs](#getting-book-cover-urls)
|
|
35
|
+
- [Data Structures](#data-structures)
|
|
33
36
|
- [Testing](#testing)
|
|
34
37
|
- [License](#license)
|
|
35
38
|
|
|
36
39
|
## Installation
|
|
37
40
|
|
|
41
|
+
```bash
|
|
42
|
+
bun add shamela
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
or
|
|
46
|
+
|
|
38
47
|
```bash
|
|
39
48
|
npm install shamela
|
|
40
49
|
```
|
|
@@ -55,9 +64,10 @@ pnpm install shamela
|
|
|
55
64
|
|
|
56
65
|
Before using the library, you need to set up some environment variables for API keys and endpoints:
|
|
57
66
|
|
|
58
|
-
`SHAMELA_API_KEY`: Your API key for accessing the Shamela API.
|
|
59
|
-
`SHAMELA_API_MASTER_PATCH_ENDPOINT`: The endpoint URL for the master database patches.
|
|
60
|
-
`SHAMELA_API_BOOKS_ENDPOINT`: The base endpoint URL for book-related API calls.
|
|
67
|
+
- `SHAMELA_API_KEY`: Your API key for accessing the Shamela API.
|
|
68
|
+
- `SHAMELA_API_MASTER_PATCH_ENDPOINT`: The endpoint URL for the master database patches.
|
|
69
|
+
- `SHAMELA_API_BOOKS_ENDPOINT`: The base endpoint URL for book-related API calls.
|
|
70
|
+
|
|
61
71
|
You can set these variables in a `.env` file at the root of your project:
|
|
62
72
|
|
|
63
73
|
```dotenv
|
|
@@ -73,7 +83,14 @@ SHAMELA_API_BOOKS_ENDPOINT=https://shamela.ws/api/books
|
|
|
73
83
|
First, import the library functions into your project:
|
|
74
84
|
|
|
75
85
|
```javascript
|
|
76
|
-
import {
|
|
86
|
+
import {
|
|
87
|
+
getMasterMetadata,
|
|
88
|
+
downloadMasterDatabase,
|
|
89
|
+
getBookMetadata,
|
|
90
|
+
downloadBook,
|
|
91
|
+
getBook,
|
|
92
|
+
getCoverUrl,
|
|
93
|
+
} from 'shamela';
|
|
77
94
|
```
|
|
78
95
|
|
|
79
96
|
### API Functions
|
|
@@ -84,34 +101,54 @@ Fetches metadata for the master database.
|
|
|
84
101
|
|
|
85
102
|
```typescript
|
|
86
103
|
getMasterMetadata(version?: number): Promise<GetMasterMetadataResponsePayload>
|
|
87
|
-
|
|
88
104
|
```
|
|
89
105
|
|
|
90
|
-
- version (optional): The version number of the master database you want to
|
|
106
|
+
- `version` (optional): The version number of the master database you want to check for updates (defaults to 0)
|
|
91
107
|
|
|
92
|
-
|
|
108
|
+
**Returns:** Promise that resolves to master database metadata including download URL and version
|
|
109
|
+
|
|
110
|
+
**Example:**
|
|
93
111
|
|
|
94
112
|
```javascript
|
|
95
113
|
const masterMetadata = await getMasterMetadata();
|
|
114
|
+
console.log(masterMetadata.url); // Download URL for master database patch
|
|
115
|
+
console.log(masterMetadata.version); // Latest version number
|
|
116
|
+
|
|
117
|
+
// Check for updates from a specific version
|
|
118
|
+
const updates = await getMasterMetadata(5);
|
|
96
119
|
```
|
|
97
120
|
|
|
98
121
|
#### downloadMasterDatabase
|
|
99
122
|
|
|
100
|
-
Downloads the master database and saves it to a specified path.
|
|
123
|
+
Downloads the master database and saves it to a specified path. The master database contains comprehensive information about all books, authors, and categories available in the Shamela library.
|
|
101
124
|
|
|
102
125
|
```typescript
|
|
103
126
|
downloadMasterDatabase(options: DownloadMasterOptions): Promise<string>
|
|
104
|
-
|
|
105
127
|
```
|
|
106
128
|
|
|
107
|
-
- options
|
|
108
|
-
- masterMetadata (optional):
|
|
109
|
-
- outputFile
|
|
129
|
+
- `options`: Configuration object containing:
|
|
130
|
+
- `masterMetadata` (optional): Pre-fetched metadata from `getMasterMetadata`
|
|
131
|
+
- `outputFile`: Object with `path` property specifying the output file path
|
|
132
|
+
|
|
133
|
+
**Returns:** Promise that resolves to the path of the created output file
|
|
110
134
|
|
|
111
|
-
Example
|
|
135
|
+
**Example:**
|
|
112
136
|
|
|
113
137
|
```javascript
|
|
138
|
+
// Download as SQLite database
|
|
139
|
+
await downloadMasterDatabase({
|
|
140
|
+
outputFile: { path: './master.db' },
|
|
141
|
+
});
|
|
142
|
+
|
|
143
|
+
// Download as JSON
|
|
144
|
+
await downloadMasterDatabase({
|
|
145
|
+
outputFile: { path: './master.json' },
|
|
146
|
+
});
|
|
147
|
+
|
|
148
|
+
// Use pre-fetched metadata for efficiency
|
|
149
|
+
const masterMetadata = await getMasterMetadata();
|
|
114
150
|
await downloadMasterDatabase({
|
|
151
|
+
masterMetadata,
|
|
115
152
|
outputFile: { path: './master.db' },
|
|
116
153
|
});
|
|
117
154
|
```
|
|
@@ -124,33 +161,101 @@ Fetches metadata for a specific book.
|
|
|
124
161
|
getBookMetadata(id: number, options?: GetBookMetadataOptions): Promise<GetBookMetadataResponsePayload>
|
|
125
162
|
```
|
|
126
163
|
|
|
127
|
-
- id
|
|
128
|
-
- options (optional):
|
|
129
|
-
- majorVersion
|
|
130
|
-
- minorVersion
|
|
164
|
+
- `id`: The unique identifier of the book
|
|
165
|
+
- `options` (optional): Configuration object containing:
|
|
166
|
+
- `majorVersion`: The major version to check against
|
|
167
|
+
- `minorVersion`: The minor version to check against
|
|
168
|
+
|
|
169
|
+
**Returns:** Promise that resolves to book metadata including release URLs and versions
|
|
131
170
|
|
|
132
|
-
Example
|
|
171
|
+
**Example:**
|
|
133
172
|
|
|
134
173
|
```javascript
|
|
135
|
-
await
|
|
136
|
-
|
|
174
|
+
const metadata = await getBookMetadata(26592);
|
|
175
|
+
console.log(metadata.majorReleaseUrl); // URL for downloading the book
|
|
176
|
+
console.log(metadata.majorRelease); // Major version number
|
|
177
|
+
|
|
178
|
+
// Check specific versions
|
|
179
|
+
const versionedMetadata = await getBookMetadata(26592, {
|
|
180
|
+
majorVersion: 1,
|
|
181
|
+
minorVersion: 2,
|
|
182
|
+
});
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
#### downloadBook
|
|
186
|
+
|
|
187
|
+
Downloads and processes a book from the Shamela database. This function downloads the book's database files, applies patches if available, and exports the data to the specified format.
|
|
188
|
+
|
|
189
|
+
```typescript
|
|
190
|
+
downloadBook(id: number, options: DownloadBookOptions): Promise<string>
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
- `id`: The unique identifier of the book to download
|
|
194
|
+
- `options`: Configuration object containing:
|
|
195
|
+
- `bookMetadata` (optional): Pre-fetched metadata from `getBookMetadata`
|
|
196
|
+
- `outputFile`: Object with `path` property specifying the output file path
|
|
197
|
+
|
|
198
|
+
**Returns:** Promise that resolves to the path of the created output file
|
|
199
|
+
|
|
200
|
+
**Example:**
|
|
201
|
+
|
|
202
|
+
```javascript
|
|
203
|
+
// Download as JSON
|
|
204
|
+
await downloadBook(26592, {
|
|
205
|
+
outputFile: { path: './book.json' },
|
|
206
|
+
});
|
|
207
|
+
|
|
208
|
+
// Download as SQLite database
|
|
209
|
+
await downloadBook(26592, {
|
|
210
|
+
outputFile: { path: './book.db' },
|
|
211
|
+
});
|
|
212
|
+
|
|
213
|
+
// Use pre-fetched metadata for efficiency
|
|
214
|
+
const bookMetadata = await getBookMetadata(26592);
|
|
215
|
+
await downloadBook(26592, {
|
|
216
|
+
bookMetadata,
|
|
217
|
+
outputFile: { path: './book.db' },
|
|
137
218
|
});
|
|
138
219
|
```
|
|
139
220
|
|
|
140
221
|
#### getBook
|
|
141
222
|
|
|
142
|
-
Retrieves
|
|
223
|
+
Retrieves complete book data as a JavaScript object. This is a convenience function that handles temporary file creation and cleanup automatically.
|
|
143
224
|
|
|
144
225
|
```typescript
|
|
145
226
|
getBook(id: number): Promise<BookData>
|
|
146
227
|
```
|
|
147
228
|
|
|
148
|
-
- id
|
|
229
|
+
- `id`: The unique identifier of the book to retrieve
|
|
230
|
+
|
|
231
|
+
**Returns:** Promise that resolves to complete book data including pages and titles
|
|
149
232
|
|
|
150
|
-
Example
|
|
233
|
+
**Example:**
|
|
151
234
|
|
|
152
235
|
```javascript
|
|
153
236
|
const bookData = await getBook(26592);
|
|
237
|
+
console.log(bookData.pages.length); // Number of pages in the book
|
|
238
|
+
console.log(bookData.titles?.length); // Number of title entries
|
|
239
|
+
console.log(bookData.pages[0].content); // Content of the first page
|
|
240
|
+
```
|
|
241
|
+
|
|
242
|
+
#### getCoverUrl
|
|
243
|
+
|
|
244
|
+
Generates the URL for a book's cover image.
|
|
245
|
+
|
|
246
|
+
```typescript
|
|
247
|
+
getCoverUrl(bookId: number): string
|
|
248
|
+
```
|
|
249
|
+
|
|
250
|
+
- `bookId`: The unique identifier of the book
|
|
251
|
+
|
|
252
|
+
**Returns:** The complete URL to the book's cover image
|
|
253
|
+
|
|
254
|
+
**Example:**
|
|
255
|
+
|
|
256
|
+
```javascript
|
|
257
|
+
const coverUrl = getCoverUrl(26592);
|
|
258
|
+
console.log(coverUrl); // "https://shamela.ws/covers/26592.jpg"
|
|
154
259
|
```
|
|
155
260
|
|
|
156
261
|
## Examples
|
|
@@ -161,21 +266,47 @@ const bookData = await getBook(26592);
|
|
|
161
266
|
import { downloadMasterDatabase } from 'shamela';
|
|
162
267
|
|
|
163
268
|
(async () => {
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
269
|
+
try {
|
|
270
|
+
// Download as SQLite database
|
|
271
|
+
const dbPath = await downloadMasterDatabase({
|
|
272
|
+
outputFile: { path: './shamela_master.db' },
|
|
273
|
+
});
|
|
274
|
+
console.log(`Master database downloaded to: ${dbPath}`);
|
|
275
|
+
|
|
276
|
+
// Download as JSON for programmatic access
|
|
277
|
+
const jsonPath = await downloadMasterDatabase({
|
|
278
|
+
outputFile: { path: './shamela_master.json' },
|
|
279
|
+
});
|
|
280
|
+
console.log(`Master data exported to: ${jsonPath}`);
|
|
281
|
+
} catch (error) {
|
|
282
|
+
console.error('Error downloading master database:', error);
|
|
283
|
+
}
|
|
167
284
|
})();
|
|
168
285
|
```
|
|
169
286
|
|
|
170
287
|
### Downloading a Book
|
|
171
288
|
|
|
172
289
|
```javascript
|
|
173
|
-
import { downloadBook } from 'shamela';
|
|
290
|
+
import { downloadBook, getBookMetadata } from 'shamela';
|
|
174
291
|
|
|
175
292
|
(async () => {
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
293
|
+
const bookId = 26592;
|
|
294
|
+
|
|
295
|
+
try {
|
|
296
|
+
// Download book as database file
|
|
297
|
+
await downloadBook(bookId, {
|
|
298
|
+
outputFile: { path: `./book_${bookId}.db` },
|
|
299
|
+
});
|
|
300
|
+
|
|
301
|
+
// Download with pre-fetched metadata
|
|
302
|
+
const metadata = await getBookMetadata(bookId);
|
|
303
|
+
await downloadBook(bookId, {
|
|
304
|
+
bookMetadata: metadata,
|
|
305
|
+
outputFile: { path: `./book_${bookId}.json` },
|
|
306
|
+
});
|
|
307
|
+
} catch (error) {
|
|
308
|
+
console.error('Error downloading book:', error);
|
|
309
|
+
}
|
|
179
310
|
})();
|
|
180
311
|
```
|
|
181
312
|
|
|
@@ -185,23 +316,101 @@ import { downloadBook } from 'shamela';
|
|
|
185
316
|
import { getBook } from 'shamela';
|
|
186
317
|
|
|
187
318
|
(async () => {
|
|
188
|
-
|
|
189
|
-
|
|
319
|
+
try {
|
|
320
|
+
const bookData = await getBook(26592);
|
|
321
|
+
|
|
322
|
+
console.log(`Book has ${bookData.pages.length} pages`);
|
|
323
|
+
|
|
324
|
+
if (bookData.titles) {
|
|
325
|
+
console.log(`Book has ${bookData.titles.length} titles/chapters`);
|
|
326
|
+
|
|
327
|
+
// Display table of contents
|
|
328
|
+
bookData.titles.forEach((title) => {
|
|
329
|
+
console.log(`${title.id}: ${title.content} (Page ${title.page})`);
|
|
330
|
+
});
|
|
331
|
+
}
|
|
332
|
+
|
|
333
|
+
// Access page content
|
|
334
|
+
const firstPage = bookData.pages[0];
|
|
335
|
+
console.log(`First page content: ${firstPage.content.substring(0, 100)}...`);
|
|
336
|
+
} catch (error) {
|
|
337
|
+
console.error('Error retrieving book:', error);
|
|
338
|
+
}
|
|
190
339
|
})();
|
|
191
340
|
```
|
|
192
341
|
|
|
342
|
+
### Getting Book Cover URLs
|
|
343
|
+
|
|
344
|
+
```javascript
|
|
345
|
+
import { getCoverUrl, downloadMasterDatabase } from 'shamela';
|
|
346
|
+
|
|
347
|
+
(async () => {
|
|
348
|
+
try {
|
|
349
|
+
// Download master data to get book information
|
|
350
|
+
const masterData = await downloadMasterDatabase({
|
|
351
|
+
outputFile: { path: './master.json' },
|
|
352
|
+
});
|
|
353
|
+
|
|
354
|
+
// Read the master data
|
|
355
|
+
const data = await Bun.file('./master.json').json();
|
|
356
|
+
|
|
357
|
+
// Generate cover URLs for all books
|
|
358
|
+
data.books.forEach((book) => {
|
|
359
|
+
const coverUrl = getCoverUrl(book.id);
|
|
360
|
+
console.log(`${book.name}: ${coverUrl}`);
|
|
361
|
+
});
|
|
362
|
+
} catch (error) {
|
|
363
|
+
console.error('Error processing covers:', error);
|
|
364
|
+
}
|
|
365
|
+
})();
|
|
366
|
+
```
|
|
367
|
+
|
|
368
|
+
## Data Structures
|
|
369
|
+
|
|
370
|
+
The library provides comprehensive TypeScript types for all data structures:
|
|
371
|
+
|
|
372
|
+
### BookData
|
|
373
|
+
|
|
374
|
+
- `pages`: Array of page objects with content, ID, and optional page numbers
|
|
375
|
+
- `titles`: Optional array of title/chapter objects with hierarchical structure
|
|
376
|
+
|
|
377
|
+
### MasterData
|
|
378
|
+
|
|
379
|
+
- `authors`: Array of author objects with biographical information
|
|
380
|
+
- `books`: Array of book objects with metadata, categories, and version information
|
|
381
|
+
- `categories`: Array of category objects for classification
|
|
382
|
+
|
|
383
|
+
### Page
|
|
384
|
+
|
|
385
|
+
- `id`: Unique identifier
|
|
386
|
+
- `content`: Text content of the page
|
|
387
|
+
- `number`, `page`, `part`: Optional numerical references
|
|
388
|
+
|
|
389
|
+
### Title
|
|
390
|
+
|
|
391
|
+
- `id`: Unique identifier
|
|
392
|
+
- `content`: Title text
|
|
393
|
+
- `page`: Page number where title appears
|
|
394
|
+
- `parent`: Optional parent title ID for hierarchical structure
|
|
395
|
+
|
|
193
396
|
## Testing
|
|
194
397
|
|
|
195
|
-
The library includes tests
|
|
398
|
+
The library includes comprehensive tests. To run them, ensure you have the necessary environment variables set, then execute:
|
|
196
399
|
|
|
197
400
|
```bash
|
|
198
|
-
|
|
401
|
+
bun test
|
|
199
402
|
```
|
|
200
403
|
|
|
201
404
|
For end-to-end tests:
|
|
202
405
|
|
|
203
406
|
```bash
|
|
204
|
-
|
|
407
|
+
bun run e2e
|
|
408
|
+
```
|
|
409
|
+
|
|
410
|
+
For CI environment:
|
|
411
|
+
|
|
412
|
+
```bash
|
|
413
|
+
bun run e2e:ci
|
|
205
414
|
```
|
|
206
415
|
|
|
207
416
|
## License
|