@chronicle.app/hackernews 0.0.0-stage → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Andrew Louis
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md CHANGED
@@ -1,3 +1,59 @@
1
- # Temporary Holding Version
1
+ # @chronicle.app/hackernews
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ Read your Hacker News submissions, comments, and the replies to them through the public [Firebase API](https://github.com/HackerNews/API). No login is needed; `--username` is the only required flag, and the CLI asks for it when it's missing.
4
+
5
+ ## Usage
6
+
7
+ ```sh
8
+ chronicle extract hackernews --username you
9
+ chronicle extract hackernews comments replies --username you --since 2025-01-01
10
+ ```
11
+
12
+ Each kind is one record type:
13
+
14
+ - `submissions` (the default): stories, polls, and job posts you submitted.
15
+ - `comments`: comments you wrote.
16
+ - `replies`: other people's direct replies to your submissions and comments. Your own replies are under `comments`.
17
+
18
+ Your items are read newest first, so `--since` ends the `submissions` and `comments` walks at the first older item. `replies` reads every item you posted, since an old item can get a new reply, and applies `--since` to the reply's own time. Deleted items are skipped, as are removed ones the API returns with no time or text; dead (killed) items are kept, since they were still posted.
19
+
20
+ Upvotes, favorites, and hidden items aren't read: Hacker News records no time for them, and upvotes also need a login and page scraping.
21
+
22
+ ## Schema
23
+
24
+ | Hacker News activity | Chronicle action | `object` | `result` |
25
+ | --------------------------------------- | ---------------- | ------------------------------ | --------- |
26
+ | You submitted a story, poll, or job | `PublishAction` | `Post` | |
27
+ | You commented | `RespondAction` | the parent `Post` or `Comment` | `Comment` |
28
+ | Someone replied to your post or comment | `RespondAction` | your `Post` or `Comment` | `Comment` |
29
+
30
+ | Hacker News thing | Chronicle entity | Key | Properties |
31
+ | ------------------- | ---------------- | ----------------------------- | ------------------------------------------------------------------------ |
32
+ | Story, poll, or job | `Post` | `@type`, `source`, `sourceId` | `name` (title), `body` (text), `url` (item page), `author`, `references` |
33
+ | Comment | `Comment` | `@type`, `source`, `sourceId` | `body`, `url` (item page), `author`, `about` |
34
+ | Submitted link | `Entity` | `url` | `url` |
35
+ | Account | `Agent` | `@type`, `source`, `handle` | `handle`, `url` (profile page), `description` (yours only) |
36
+
37
+ ### Decisions
38
+
39
+ **A comment is a `RespondAction`, not a `CreateAction`.** The schema defines a comment as a prose `Response`, the `result` of a `RespondAction` whose `object` is what it responds to. A reply by someone else is the same shape with a different agent, so `comments` and `replies` share one transform.
40
+
41
+ **Two edges from a comment: `object` for its parent, `about` for its submission.** Hacker News gives each comment only its direct `parent`, which may be another comment. The action's `object` is that parent, preserving the thread's tree. The `Comment` is also `about` the story or poll at the top of the thread, found by walking `parent` links, so a reply deep in a discussion still links to what the discussion is about. YouTube and Are.na comments are likewise `about` their video or block. For a top-level comment, the two edges point at the same `Post`. Each item is fetched once per run, however many comments share it as an ancestor.
42
+
43
+ **The submitted link is in `references`, not `about`.** The schema reserves `about` for subject matter, such as topics and tags, and defines `references` as a link from one resource to another. A link submission is a post pointing at a page, as Are.na's link blocks are. Keeping `about` for "the submission this comment's thread is on" also keeps that edge unambiguous.
44
+
45
+ **The link is the shared page node.** It's an `Entity` keyed by `url` alone, with no `source`, so it is the same node that Safari, Are.na, and Obsidian produce for that page. Pinboard instead keys its page nodes by `url` and `source`, joining this one through their `url`, because its snapshots clear per-source notes and tags; Hacker News asserts no such state about the page.
46
+
47
+ **`author` is set on every `Post` and `Comment`.** Parent comments, the story at the top of a thread, and the item a reply answers have no action of their own in these records, so `author` is the only edge to whoever wrote them. It's set on your own items too, so a post has the same shape wherever it appears. It sits on the Hacker News `Post`, never on the linked page: the submitter wrote the submission, not the article.
48
+
49
+ **Accounts are keyed by `handle`.** A username is the only identity Hacker News gives an account; there is no separate numeric id, so there is no `sourceId`. Every `Agent` carries its profile page, `https://news.ycombinator.com/user?id=<username>`, as `url`. Yours also carries `sameAs: ['@me']` and your profile's about text as `description`.
50
+
51
+ **Items are keyed by their Hacker News id.** Stories, comments, and polls share one id sequence, so the id alone identifies an item. An action and its `Post` or `Comment` share that `sourceId` and differ by `@type`.
52
+
53
+ **Times are the item's post time.** Every action's `timestamp` is the item's `time`, exact to the second. Nothing is dated by when it was read.
54
+
55
+ **Text is plain.** Hacker News sends comment and post text as HTML. It's converted to plain text, with each `<p>` starting a new line and links kept as `label (url)`.
56
+
57
+ ## Tests
58
+
59
+ Tests run the extractors against a fake Firebase API on 127.0.0.1 with synthetic items. They never reach the network.
@@ -0,0 +1,9 @@
1
+ import { Record } from '@chronicle.app/etl';
2
+ import HackerNewsExtractor from './HackerNewsExtractor.js';
3
+ export default class HackerNewsCommentsExtractor extends HackerNewsExtractor {
4
+ static description: string;
5
+ static recordTypes: string[];
6
+ static newestFirst: boolean;
7
+ protected records(): AsyncGenerator<Record>;
8
+ }
9
+ //# sourceMappingURL=HackerNewsCommentsExtractor.d.ts.map
@@ -0,0 +1,17 @@
1
+ import HackerNewsExtractor from './HackerNewsExtractor.js';
2
+ export default class HackerNewsCommentsExtractor extends HackerNewsExtractor {
3
+ static description = 'Comments you wrote';
4
+ static recordTypes = ['comments'];
5
+ static newestFirst = true;
6
+ async *records() {
7
+ for await (const item of this.ownItems({ stopAtSince: true })) {
8
+ if (item.type !== 'comment' || !this.inRange(item))
9
+ continue;
10
+ // The direct parent is what the comment answers; the root is the
11
+ // submission whose discussion it belongs to.
12
+ const parent = item.parent === undefined ? null : await this.proxy.getItem(item.parent);
13
+ const root = await this.proxy.getRoot(item);
14
+ yield this.createRecord(item, this.context({ recordType: 'comments', parent, root }));
15
+ }
16
+ }
17
+ }
@@ -0,0 +1,52 @@
1
+ import { Extractor, Record, z } from '@chronicle.app/etl';
2
+ import HackerNewsProxy, { HackerNewsItem, HackerNewsUser } from '../utils/HackerNewsProxy.js';
3
+ import HackerNewsTransformer from './HackerNewsTransformer.js';
4
+ export default abstract class HackerNewsExtractor extends Extractor<typeof HackerNewsExtractor> {
5
+ static defaultTransformer: typeof HackerNewsTransformer;
6
+ static source: string;
7
+ static delivery: "api";
8
+ static strategy: string;
9
+ static schema: z.ZodObject<{
10
+ since: z.ZodOptional<z.ZodDate>;
11
+ until: z.ZodOptional<z.ZodDate>;
12
+ limit: z.ZodOptional<z.ZodNumber>;
13
+ } & {
14
+ username: z.ZodString;
15
+ }, "strip", z.ZodTypeAny, {
16
+ username: string;
17
+ since?: Date | undefined;
18
+ until?: Date | undefined;
19
+ limit?: number | undefined;
20
+ }, {
21
+ username: string;
22
+ since?: Date | undefined;
23
+ until?: Date | undefined;
24
+ limit?: number | undefined;
25
+ }>;
26
+ protected proxy: HackerNewsProxy;
27
+ protected user: HackerNewsUser;
28
+ setup(): Promise<void>;
29
+ /** The raw item id — Hacker News numbers every item once, across all types. */
30
+ keyOf(record: Record): string | null;
31
+ protected abstract records(): AsyncGenerator<Record>;
32
+ extract(): AsyncGenerator<Record>;
33
+ protected context(extra?: object): {
34
+ user: HackerNewsUser;
35
+ };
36
+ /**
37
+ * The user's own items, newest first. With `stopAtSince`, the walk ends at
38
+ * the first item older than `since`: every later id is older still.
39
+ */
40
+ protected ownItems({ stopAtSince }: {
41
+ stopAtSince: boolean;
42
+ }): AsyncGenerator<HackerNewsItem, void, unknown>;
43
+ /**
44
+ * The items in order, a batch at a time, without the missing and deleted.
45
+ * A removed item can also come back as just its id and type, with no time,
46
+ * author, or text; it is skipped too.
47
+ */
48
+ protected fetchEach(ids: number[]): AsyncGenerator<HackerNewsItem>;
49
+ protected isBeforeSince(item: HackerNewsItem): boolean;
50
+ protected inRange(item: HackerNewsItem): boolean;
51
+ }
52
+ //# sourceMappingURL=HackerNewsExtractor.d.ts.map
@@ -0,0 +1,77 @@
1
+ import { Extractor, InputNotFound, z } from '@chronicle.app/etl';
2
+ import HackerNewsProxy from '../utils/HackerNewsProxy.js';
3
+ import HackerNewsTransformer from './HackerNewsTransformer.js';
4
+ /** Items fetched at once. The Firebase API has no batch endpoint. */
5
+ const CONCURRENCY = 8;
6
+ export default class HackerNewsExtractor extends Extractor {
7
+ static defaultTransformer = HackerNewsTransformer;
8
+ static source = 'hackernews';
9
+ static delivery = 'api';
10
+ static strategy = 'api';
11
+ static schema = Extractor.schema.extend({
12
+ // Hacker News has no API credentials: the username is all it needs.
13
+ username: z.string().describe('Hacker News username'),
14
+ });
15
+ proxy;
16
+ user;
17
+ async setup() {
18
+ const { username } = this.config;
19
+ this.proxy = new HackerNewsProxy();
20
+ const user = await this.proxy.getUser(username);
21
+ if (!user) {
22
+ throw new InputNotFound(`Hacker News has no user named ${username}`);
23
+ }
24
+ this.user = user;
25
+ }
26
+ /** The raw item id — Hacker News numbers every item once, across all types. */
27
+ keyOf(record) {
28
+ const { id } = record.data;
29
+ return id === undefined ? null : String(id);
30
+ }
31
+ async *extract() {
32
+ let count = 0;
33
+ for await (const record of this.records()) {
34
+ if (this.shouldStopExtracting(count))
35
+ return;
36
+ yield record;
37
+ count++;
38
+ }
39
+ }
40
+ context(extra = {}) {
41
+ return { user: this.user, ...extra };
42
+ }
43
+ /**
44
+ * The user's own items, newest first. With `stopAtSince`, the walk ends at
45
+ * the first item older than `since`: every later id is older still.
46
+ */
47
+ async *ownItems({ stopAtSince }) {
48
+ for await (const item of this.fetchEach(this.user.submitted ?? [])) {
49
+ if (stopAtSince && this.isBeforeSince(item))
50
+ return;
51
+ yield item;
52
+ }
53
+ }
54
+ /**
55
+ * The items in order, a batch at a time, without the missing and deleted.
56
+ * A removed item can also come back as just its id and type, with no time,
57
+ * author, or text; it is skipped too.
58
+ */
59
+ async *fetchEach(ids) {
60
+ for (let start = 0; start < ids.length; start += CONCURRENCY) {
61
+ const batch = await Promise.all(ids.slice(start, start + CONCURRENCY).map(id => this.proxy.getItem(id)));
62
+ for (const item of batch) {
63
+ if (item && !item.deleted && item.time !== undefined)
64
+ yield item;
65
+ }
66
+ }
67
+ }
68
+ isBeforeSince(item) {
69
+ return this.config.since !== undefined && time(item) < this.config.since;
70
+ }
71
+ inRange(item) {
72
+ return !this.isBeforeSince(item) && !(this.config.until && time(item) > this.config.until);
73
+ }
74
+ }
75
+ function time(item) {
76
+ return new Date((item.time ?? 0) * 1000);
77
+ }
@@ -0,0 +1,8 @@
1
+ import { Record } from '@chronicle.app/etl';
2
+ import HackerNewsExtractor from './HackerNewsExtractor.js';
3
+ export default class HackerNewsRepliesExtractor extends HackerNewsExtractor {
4
+ static description: string;
5
+ static recordTypes: string[];
6
+ protected records(): AsyncGenerator<Record>;
7
+ }
8
+ //# sourceMappingURL=HackerNewsRepliesExtractor.d.ts.map
@@ -0,0 +1,19 @@
1
+ import HackerNewsExtractor from './HackerNewsExtractor.js';
2
+ export default class HackerNewsRepliesExtractor extends HackerNewsExtractor {
3
+ static description = "Other people's replies to your submissions and comments";
4
+ static recordTypes = ['replies'];
5
+ async *records() {
6
+ // An old item can still get a new reply, so `since` can't end the walk.
7
+ for await (const parent of this.ownItems({ stopAtSince: false })) {
8
+ if (!parent.kids?.length)
9
+ continue;
10
+ const root = parent.type === 'comment' ? await this.proxy.getRoot(parent) : parent;
11
+ for await (const reply of this.fetchEach(parent.kids)) {
12
+ // Your own replies are already under `comments`.
13
+ if (reply.by === this.user.id || !this.inRange(reply))
14
+ continue;
15
+ yield this.createRecord(reply, this.context({ recordType: 'replies', parent, root }));
16
+ }
17
+ }
18
+ }
19
+ }
@@ -0,0 +1,12 @@
1
+ import { Record } from '@chronicle.app/etl';
2
+ import HackerNewsExtractor from './HackerNewsExtractor.js';
3
+ /** Item types a user submits, as opposed to comments on them. */
4
+ export declare const SUBMISSION_TYPES: Set<string>;
5
+ export default class HackerNewsSubmissionsExtractor extends HackerNewsExtractor {
6
+ static description: string;
7
+ static recordTypes: string[];
8
+ static default: boolean;
9
+ static newestFirst: boolean;
10
+ protected records(): AsyncGenerator<Record>;
11
+ }
12
+ //# sourceMappingURL=HackerNewsSubmissionsExtractor.d.ts.map
@@ -0,0 +1,16 @@
1
+ import HackerNewsExtractor from './HackerNewsExtractor.js';
2
+ /** Item types a user submits, as opposed to comments on them. */
3
+ export const SUBMISSION_TYPES = new Set(['story', 'poll', 'job']);
4
+ export default class HackerNewsSubmissionsExtractor extends HackerNewsExtractor {
5
+ static description = 'Stories, polls, and job posts you submitted';
6
+ static recordTypes = ['submissions'];
7
+ static default = true;
8
+ static newestFirst = true;
9
+ async *records() {
10
+ for await (const item of this.ownItems({ stopAtSince: true })) {
11
+ if (!SUBMISSION_TYPES.has(item.type ?? '') || !this.inRange(item))
12
+ continue;
13
+ yield this.createRecord(item, this.context({ recordType: 'submissions' }));
14
+ }
15
+ }
16
+ }
@@ -0,0 +1,21 @@
1
+ import { ChronicleTransformer, Record } from '@chronicle.app/etl';
2
+ import { ActionAndChildren } from '@chronicle.app/schema';
3
+ export default class HackerNewsTransformer extends ChronicleTransformer {
4
+ /** The account being extracted, set per record from context. */
5
+ private user?;
6
+ transform(record: Record): Promise<ActionAndChildren[]>;
7
+ private buildPublish;
8
+ /**
9
+ * A comment answers its direct parent (the action's `object`) and is about
10
+ * the submission at the top of its thread (the comment's `about`). For a
11
+ * top-level comment the two are the same Post.
12
+ */
13
+ private buildRespond;
14
+ private buildItem;
15
+ private buildPost;
16
+ private buildComment;
17
+ /** The submitted page: the shared `{url}` hub, as Are.na's link blocks are. */
18
+ private buildLink;
19
+ private buildUser;
20
+ }
21
+ //# sourceMappingURL=HackerNewsTransformer.d.ts.map
@@ -0,0 +1,114 @@
1
+ import { ChronicleTransformer, htmlToText, selfAgent } from '@chronicle.app/etl';
2
+ const SOURCE = 'hackernews';
3
+ const KEY = ['@type', 'source', 'sourceId'];
4
+ function itemUrl(id) {
5
+ return `https://news.ycombinator.com/item?id=${id}`;
6
+ }
7
+ function userUrl(username) {
8
+ return `https://news.ycombinator.com/user?id=${encodeURIComponent(username)}`;
9
+ }
10
+ function time(item) {
11
+ return new Date((item.time ?? 0) * 1000);
12
+ }
13
+ export default class HackerNewsTransformer extends ChronicleTransformer {
14
+ /** The account being extracted, set per record from context. */
15
+ user;
16
+ async transform(record) {
17
+ // Safe as a per-record field: everything below builds synchronously.
18
+ this.user = record.context.user;
19
+ const item = record.data;
20
+ switch (record.extraction.recordType) {
21
+ case 'submissions': {
22
+ return [this.buildPublish(item)];
23
+ }
24
+ // A reply is a comment by someone else; both answer their parent.
25
+ case 'comments':
26
+ case 'replies': {
27
+ return [this.buildRespond(item, record.context.parent, record.context.root)];
28
+ }
29
+ default: {
30
+ return [];
31
+ }
32
+ }
33
+ }
34
+ buildPublish(item) {
35
+ return {
36
+ '@type': 'PublishAction',
37
+ '@key': KEY,
38
+ source: SOURCE,
39
+ sourceId: String(item.id),
40
+ timestamp: time(item),
41
+ ...(item.by && { agent: this.buildUser(item.by) }),
42
+ object: this.buildPost(item),
43
+ };
44
+ }
45
+ /**
46
+ * A comment answers its direct parent (the action's `object`) and is about
47
+ * the submission at the top of its thread (the comment's `about`). For a
48
+ * top-level comment the two are the same Post.
49
+ */
50
+ buildRespond(item, parent, root) {
51
+ return {
52
+ '@type': 'RespondAction',
53
+ '@key': KEY,
54
+ source: SOURCE,
55
+ sourceId: String(item.id),
56
+ timestamp: time(item),
57
+ ...(item.by && { agent: this.buildUser(item.by) }),
58
+ ...(parent && { object: this.buildItem(parent, root) }),
59
+ result: this.buildComment(item, root),
60
+ };
61
+ }
62
+ buildItem(item, root) {
63
+ return item.type === 'comment' ? this.buildComment(item, root) : this.buildPost(item);
64
+ }
65
+ buildPost(item) {
66
+ const body = htmlToText(item.text);
67
+ return {
68
+ '@type': 'Post',
69
+ '@key': KEY,
70
+ source: SOURCE,
71
+ sourceId: String(item.id),
72
+ url: itemUrl(item.id),
73
+ ...(item.title && { name: item.title }),
74
+ ...(body && { body }),
75
+ ...(item.by && { author: [this.buildUser(item.by)] }),
76
+ ...(item.url && { references: [this.buildLink(item.url)] }),
77
+ };
78
+ }
79
+ buildComment(item, root) {
80
+ const body = htmlToText(item.text);
81
+ return {
82
+ '@type': 'Comment',
83
+ '@key': KEY,
84
+ source: SOURCE,
85
+ sourceId: String(item.id),
86
+ url: itemUrl(item.id),
87
+ ...(body && { body }),
88
+ ...(item.by && { author: [this.buildUser(item.by)] }),
89
+ ...(root && { about: [this.buildPost(root)] }),
90
+ };
91
+ }
92
+ /** The submitted page: the shared `{url}` hub, as Are.na's link blocks are. */
93
+ buildLink(url) {
94
+ return { '@type': 'Entity', '@key': ['url'], url };
95
+ }
96
+ buildUser(username) {
97
+ if (username !== this.user?.id) {
98
+ // A username is the only identity Hacker News gives an account.
99
+ return {
100
+ '@type': 'Agent',
101
+ '@key': ['@type', 'source', 'handle'],
102
+ source: SOURCE,
103
+ handle: username,
104
+ url: userUrl(username),
105
+ };
106
+ }
107
+ const description = htmlToText(this.user.about);
108
+ return {
109
+ ...selfAgent({ type: 'Agent', source: SOURCE, handle: username }),
110
+ url: userUrl(username),
111
+ ...(description && { description }),
112
+ };
113
+ }
114
+ }
@@ -0,0 +1,6 @@
1
+ export { default as HackerNewsExtractor } from './connectors/HackerNewsExtractor.js';
2
+ export { default as HackerNewsCommentsExtractor } from './connectors/HackerNewsCommentsExtractor.js';
3
+ export { default as HackerNewsRepliesExtractor } from './connectors/HackerNewsRepliesExtractor.js';
4
+ export { default as HackerNewsSubmissionsExtractor } from './connectors/HackerNewsSubmissionsExtractor.js';
5
+ export { default as HackerNewsTransformer } from './connectors/HackerNewsTransformer.js';
6
+ //# sourceMappingURL=index.d.ts.map
package/dist/index.js ADDED
@@ -0,0 +1,5 @@
1
+ export { default as HackerNewsExtractor } from './connectors/HackerNewsExtractor.js';
2
+ export { default as HackerNewsCommentsExtractor } from './connectors/HackerNewsCommentsExtractor.js';
3
+ export { default as HackerNewsRepliesExtractor } from './connectors/HackerNewsRepliesExtractor.js';
4
+ export { default as HackerNewsSubmissionsExtractor } from './connectors/HackerNewsSubmissionsExtractor.js';
5
+ export { default as HackerNewsTransformer } from './connectors/HackerNewsTransformer.js';
@@ -0,0 +1,37 @@
1
+ import { ApiProxy } from '@chronicle.app/etl';
2
+ /** A user profile from the Firebase API. */
3
+ export interface HackerNewsUser {
4
+ id: string;
5
+ created: number;
6
+ karma?: number;
7
+ about?: string;
8
+ /** Every item the user posted, newest first. */
9
+ submitted?: number[];
10
+ }
11
+ /** An item from the Firebase API: a story, comment, poll, poll option, or job. */
12
+ export interface HackerNewsItem {
13
+ id: number;
14
+ type?: 'story' | 'comment' | 'poll' | 'pollopt' | 'job';
15
+ by?: string;
16
+ /** Unix time in seconds. */
17
+ time?: number;
18
+ text?: string;
19
+ url?: string;
20
+ title?: string;
21
+ parent?: number;
22
+ kids?: number[];
23
+ deleted?: boolean;
24
+ dead?: boolean;
25
+ }
26
+ export default class HackerNewsProxy extends ApiProxy {
27
+ private items;
28
+ constructor();
29
+ initialize(): Promise<void>;
30
+ /** The profile, or null when no such user exists. */
31
+ getUser(username: string): Promise<HackerNewsUser | null>;
32
+ /** The item, or null when the id doesn't exist. */
33
+ getItem(id: number): Promise<HackerNewsItem | null>;
34
+ /** The story, poll, or job at the top of the item's thread. */
35
+ getRoot(item: HackerNewsItem): Promise<HackerNewsItem | undefined>;
36
+ }
37
+ //# sourceMappingURL=HackerNewsProxy.d.ts.map
@@ -0,0 +1,39 @@
1
+ import { ApiProxy } from '@chronicle.app/etl';
2
+ export default class HackerNewsProxy extends ApiProxy {
3
+ // Items never move between threads, so one fetch per run serves every
4
+ // comment that shares an ancestor.
5
+ items = new Map();
6
+ constructor() {
7
+ super({ baseURL: 'https://hacker-news.firebaseio.com/v0' });
8
+ }
9
+ // The API is public; there is nothing to load.
10
+ async initialize() { }
11
+ /** The profile, or null when no such user exists. */
12
+ async getUser(username) {
13
+ return this.request({
14
+ url: `/user/${encodeURIComponent(username)}.json`,
15
+ method: 'GET',
16
+ });
17
+ }
18
+ /** The item, or null when the id doesn't exist. */
19
+ getItem(id) {
20
+ let item = this.items.get(id);
21
+ if (!item) {
22
+ item = this.request({ url: `/item/${id}.json`, method: 'GET' });
23
+ this.items.set(id, item);
24
+ }
25
+ return item;
26
+ }
27
+ /** The story, poll, or job at the top of the item's thread. */
28
+ async getRoot(item) {
29
+ let current = item;
30
+ const seen = new Set();
31
+ while (current?.type === 'comment' && current.parent !== undefined) {
32
+ if (seen.has(current.id))
33
+ return undefined;
34
+ seen.add(current.id);
35
+ current = await this.getItem(current.parent);
36
+ }
37
+ return current && current.type !== 'comment' ? current : undefined;
38
+ }
39
+ }
package/package.json CHANGED
@@ -1,6 +1,73 @@
1
1
  {
2
2
  "name": "@chronicle.app/hackernews",
3
- "version": "0.0.0-stage",
4
- "stub": true,
5
- "description": "Temporary package placeholder for staged publishing"
6
- }
3
+ "version": "0.4.0",
4
+ "main": "./dist/index.js",
5
+ "types": "./dist/index.d.ts",
6
+ "files": [
7
+ "dist/**/*.js",
8
+ "dist/**/*.d.ts"
9
+ ],
10
+ "scripts": {
11
+ "build": "tsc -b tsconfig.lib.json",
12
+ "typecheck": "tsc -p tsconfig.json",
13
+ "lint": "eslint src --ext .ts,.js",
14
+ "test": "node --test src/*.test.js",
15
+ "shapes": "node src/shapes.test.js --update"
16
+ },
17
+ "keywords": [
18
+ "chronicle",
19
+ "hackernews"
20
+ ],
21
+ "author": "Andrew Louis",
22
+ "license": "MIT",
23
+ "type": "module",
24
+ "description": "Chronicle plugin for Hacker News submissions, comments, and replies",
25
+ "devDependencies": {
26
+ "@chronicle.app/etl": "0.4.0",
27
+ "@chronicle.app/schema": "0.4.0",
28
+ "@chronicle.app/tsconfig": "0.4.0",
29
+ "@types/node": "^22.10.1",
30
+ "typescript": "^5.7.2"
31
+ },
32
+ "homepage": "https://chronicle.app",
33
+ "repository": {
34
+ "type": "git",
35
+ "url": "https://github.com/chronicle-app/chronicle.git",
36
+ "directory": "plugins/hackernews"
37
+ },
38
+ "publishConfig": {
39
+ "access": "public",
40
+ "registry": "https://registry.npmjs.org/"
41
+ },
42
+ "peerDependencies": {
43
+ "@chronicle.app/etl": ">=0.1.0 <1.0.0",
44
+ "@chronicle.app/schema": ">=0.1.0 <1.0.0"
45
+ },
46
+ "engines": {
47
+ "node": ">=22.13.0"
48
+ },
49
+ "exports": {
50
+ ".": {
51
+ "types": "./dist/index.d.ts",
52
+ "import": "./dist/index.js"
53
+ }
54
+ },
55
+ "chronicle": {
56
+ "plugin": true,
57
+ "sources": {
58
+ "hackernews": {
59
+ "strategies": {
60
+ "api": {
61
+ "delivery": "api",
62
+ "recordTypes": [
63
+ "comments",
64
+ "replies",
65
+ "submissions"
66
+ ],
67
+ "default": true
68
+ }
69
+ }
70
+ }
71
+ }
72
+ }
73
+ }