extract-cite 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +56 -0
- package/dist/extract-cite.cjs.js +2 -0
- package/dist/extract-cite.cjs.js.map +1 -0
- package/dist/extract-cite.es.js +4 -0
- package/dist/extract-cite.es.js.map +1 -0
- package/dist/html-to-cite/extract-author.d.ts +11 -0
- package/dist/html-to-cite/extract-cite.d.ts +33 -0
- package/dist/html-to-cite/extract-date/date-extractors.d.ts +40 -0
- package/dist/html-to-cite/extract-date/date-validators.d.ts +15 -0
- package/dist/html-to-cite/extract-date/extract-date-quick.d.ts +8 -0
- package/dist/html-to-cite/extract-date/extract-date.d.ts +26 -0
- package/dist/html-to-cite/extract-source.d.ts +11 -0
- package/dist/html-to-cite/extract-title.d.ts +11 -0
- package/dist/html-to-cite/human-names-recognize.d.ts +16 -0
- package/dist/html-to-cite/metadata-to-cite.d.ts +16 -0
- package/dist/html-to-cite/url-to-domain.d.ts +20 -0
- package/dist/index.d.ts +14 -0
- package/dist/llm/call-llm.d.ts +33 -0
- package/dist/llm/extract-cite-llm.d.ts +19 -0
- package/dist/llm/format-citation.d.ts +22 -0
- package/dist/llm/index.d.ts +7 -0
- package/dist/llm/page-text.d.ts +21 -0
- package/dist/llm/parse-reply.d.ts +26 -0
- package/dist/llm/prompt.d.ts +14 -0
- package/dist/llm/types.d.ts +115 -0
- package/package.json +61 -0
- package/src/html-to-cite/extract-author.ts +125 -0
- package/src/html-to-cite/extract-cite.ts +97 -0
- package/src/html-to-cite/extract-date/date-extractors.ts +509 -0
- package/src/html-to-cite/extract-date/date-validators.ts +191 -0
- package/src/html-to-cite/extract-date/extract-date-quick.ts +184 -0
- package/src/html-to-cite/extract-date/extract-date.ts +1051 -0
- package/src/html-to-cite/extract-source.ts +35 -0
- package/src/html-to-cite/extract-title.ts +78 -0
- package/src/html-to-cite/human-names-92k.json +1 -0
- package/src/html-to-cite/human-names-recognize.ts +396 -0
- package/src/html-to-cite/metadata-to-cite.ts +77 -0
- package/src/html-to-cite/url-to-domain.ts +50 -0
- package/src/index.ts +14 -0
- package/src/llm/call-llm.ts +112 -0
- package/src/llm/extract-cite-llm.ts +100 -0
- package/src/llm/format-citation.ts +399 -0
- package/src/llm/index.ts +7 -0
- package/src/llm/page-text.ts +106 -0
- package/src/llm/parse-reply.ts +279 -0
- package/src/llm/prompt.ts +95 -0
- package/src/llm/types.ts +174 -0
package/README.md
ADDED
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
# extract-cite
|
|
2
|
+
|
|
3
|
+
Cite any webpage. Two passes:
|
|
4
|
+
|
|
5
|
+
1. **`extractCite`**: reads author, date, title and source straight off the HTML (meta tags, bylines, JSON-LD, URL patterns, a 92k-name database that tells people from organizations). Instant, offline, partial.
|
|
6
|
+
2. **`extractCiteLLM`**: one model call (OpenRouter by default) that is given the partial citation and the parts still missing, completes the full APA citation, **scores the confidence of every part**, **flags parts for review**, and reads **author qualifications** from the page's bio. Output in APA 7, MLA 9, Chicago, Harvard, IEEE and BibTeX.
|
|
7
|
+
|
|
8
|
+
```bash
|
|
9
|
+
bun add extract-cite
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
```ts
|
|
13
|
+
import { extractCiteLLM } from "extract-cite";
|
|
14
|
+
|
|
15
|
+
const cite = await extractCiteLLM({
|
|
16
|
+
url: "https://www.npr.org/2023/12/28/1221827923/2023-hottest-year-record-climate-change",
|
|
17
|
+
apiKey: process.env.OPENROUTER_API_KEY, // default provider: OpenRouter
|
|
18
|
+
model: "anthropic/claude-haiku-4.5", // default
|
|
19
|
+
// baseUrl: "https://api.openai.com/v1" // any OpenAI-compatible API
|
|
20
|
+
});
|
|
21
|
+
|
|
22
|
+
cite.formatted.apa?.text; // full APA 7 entry
|
|
23
|
+
cite.formatted.mla?.html; // with <i> italics, HTML-escaped
|
|
24
|
+
cite.confidence; // { title: 0.98, publishedDate: 0.95, … }
|
|
25
|
+
cite.needsReview; // [{ field, confidence, reason }]
|
|
26
|
+
cite.citation.authors[0].qualifications; // { jobTitle, affiliation, credentials, evidence, … }
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
| Option | Default | |
|
|
30
|
+
| --- | --- | --- |
|
|
31
|
+
| `url` / `html` / `text` | | The page. A `url` alone is fetched; pass `html` or `text` for sites that block servers. |
|
|
32
|
+
| `apiKey` | `OPENROUTER_API_KEY` | |
|
|
33
|
+
| `model` | `anthropic/claude-haiku-4.5` | |
|
|
34
|
+
| `baseUrl` | `https://openrouter.ai/api/v1` | |
|
|
35
|
+
| `styles` | all | `apa`, `mla`, `chicago`, `harvard`, `ieee`, `bibtex` |
|
|
36
|
+
| `reviewThreshold` | `0.7` | Parts below this go in `needsReview`. |
|
|
37
|
+
| `maxChars` | `12000` | Page text sent to the model (head and tail). |
|
|
38
|
+
| `fetch` | global | Replaces `fetch` for both the page and the model call. |
|
|
39
|
+
|
|
40
|
+
`formatCitation(citation, style)` and `formatCitations(citation, styles)` re-style a result offline.
|
|
41
|
+
|
|
42
|
+
## What it will not do
|
|
43
|
+
|
|
44
|
+
The model's reply is not trusted. Text is fenced as untrusted data; strings are stripped of markup; dates must be real; an author whose name the page never prints is capped at 0.2 confidence and flagged; author qualifications need a verbatim quote that is on the page, or they are dropped. A part the page does not state comes back empty and flagged, not guessed. Still: **check `needsReview` before you cite**.
|
|
45
|
+
|
|
46
|
+
## Moved from extract-webpage
|
|
47
|
+
|
|
48
|
+
`extractCite`, `convertURLToDomain` and `isURLValid` used to live in `extract-webpage` (`src/html-to-cite`). That code moved here unchanged; `extract-webpage` re-exports those three, so existing imports keep working.
|
|
49
|
+
|
|
50
|
+
## Demo
|
|
51
|
+
|
|
52
|
+
The Citation tab of the [extract-pdf site](../extract-pdf/site) (`/demo#cite`) runs this against news, journal, organization and blog URLs.
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
cd packages/extract-cite && bun run test
|
|
56
|
+
```
|