webring 1.0.3 → 1.1.1

package/CHANGELOG.md

@@ -5,6 +5,24 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.1.1](https://github.com/shepherdjerred/webring/compare/v1.1.0...v1.1.1) (2024-07-07)
+
+
+### Bug Fixes
+
+* **deps:** update dependency astro to v4.11.5 ([a61f733](https://github.com/shepherdjerred/webring/commit/a61f733cbd3f7bd659d4a608f58473885a5bcc61))
+* **deps:** update dependency remeda to v2.2.2 ([97a6c06](https://github.com/shepherdjerred/webring/commit/97a6c064138e0c204e231efb2ca45defd82b61f9))
+* **deps:** update dependency remeda to v2.3.0 ([7a4c8d5](https://github.com/shepherdjerred/webring/commit/7a4c8d51ad5722be0a8dced10eb0b14a279b0bf6))
+* update snapshot ([c73cdf7](https://github.com/shepherdjerred/webring/commit/c73cdf779433e6e7e8c5f2beee3fb2aefec9a0e0))
+* use import instead of triple-slash ([971a77e](https://github.com/shepherdjerred/webring/commit/971a77ecd0c612850faeb9d16f7775d3e7ca7253))
+
+## [1.1.0](https://github.com/shepherdjerred/webring/compare/v1.0.3...v1.1.0) (2024-06-28)
+
+
+### Features
+
+* source mappings ([095fe2b](https://github.com/shepherdjerred/webring/commit/095fe2be44e25547271730a5611d4710609cdf8d))
+
 ## [1.0.3](https://github.com/shepherdjerred/webring/compare/v1.0.2...v1.0.3) (2024-06-24)
 
 

package/README.md

@@ -1,13 +1,17 @@
-# webring
+<div align="center">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset="https://cdn.rawgit.com/shepherdjerred/webring/main/assets/logo-dark.png">
+    <source media="(prefers-color-scheme: light)" srcset="https://cdn.rawgit.com/shepherdjerred/webring/main/assets/logo-light.png">
+    <img alt="webring logo" src="https://cdn.rawgit.com/shepherdjerred/webring/main/assets/logo-light.png" height=150>
+  </picture>
 
 [![webring](https://img.shields.io/npm/v/webring.svg)](https://www.npmjs.com/package/webring)
 
-`webring` gathers the latest posts from your favorite RSS feeds so that you can embed them on your site.
+`webring` fetches the latest updates from your favorite RSS feeds.
 
-Inspired by:
+This project is actively maintained. If you have a feature request or need help, please [create an issue](https://github.com/shepherdjerred/webring/issues/new).
 
-- https://github.com/lukehsiao/openring-rs
-- https://git.sr.ht/~sircmpwn/openring
+</div>
 
 ## Installation
 
@@ -15,13 +19,92 @@ Inspired by:
 npm i webring
 ```
 
+## Features
+
+- Written in TypeScript
+- Caching
+- HTML sanitization and truncation
+
 ## Quick Start
 
-This library is meant to be used with static site generators. It is framework agnostic. I use this with [Astro](https://astro.build/) on my [personal website](https://github.com/shepherdjerred/sjer.red/blob/main/src/components/BlogWebring.astro#L17-L22).
+This library is intended to be used with a static site generator. I use this with [Astro](https://astro.build/) on my [personal website](https://github.com/shepherdjerred/sjer.red/blob/1220ebef2e43956ba385402ed8529870e9084de8/src/components/BlogWebring.astro#L17-L22).
 
 ```typescript
 import { run } from "webring";
-import { type Configuration, type Result } from "webring";
+
+const result = await run({
+  sources: [
+    {
+      url: "https://drewdevault.com/blog/index.xml",
+      title: "Drew DeVault",
+    },
+    {
+      url: "https://danluu.com/atom.xml",
+      title: "Dan Luu",
+    },
+    {
+      url: "https://jakelazaroff.com/rss.xml",
+      title: "Jake Lazaroff",
+    },
+  ],
+});
+
+console.log(result);
+// [
+//   {
+//     title: 'A discussion of discussions on AI bias',
+//     url: 'https://danluu.com/ai-bias/',
+//     date: 2024-06-16T00:00:00.000Z,
+//     source: { url: 'https://danluu.com/atom.xml', title: 'Dan Luu' },
+//     preview: `There've been regular viral stories about ML/AI bias with LLMs and generative AI for the past couple years. One thing I find interesting about discussions of bias is how different the reaction is in the LLM and generative AI case when compared to "classical" bugs in cases where there's a clear bug. ...`
+//   },
+//   {
+//     title: 'Writing a Unix clone in about a month',
+//     url: 'https://drewdevault.com/2024/05/24/2024-05-24-Bunnix.html',
+//     date: 2024-05-24T00:00:00.000Z,
+//     source: {
+//       url: 'https://drewdevault.com/blog/index.xml',
+//       title: 'Drew DeVault'
+//     },
+//     preview: 'I needed a bit of a break from “real work” recently, so I started a new programming project that was low-stakes and purely recreational. On April 21st, I set out to see how much of a Unix-like operating system for x86_64 targets that I could put together in about a month. The result is Bunnix. Not i...'
+//   },
+//   {
+//     title: 'The Web Component Success Story',
+//     url: 'https://jakelazaroff.com/words/the-web-component-success-story/',
+//     date: 2024-01-29T00:00:00.000Z,
+//     source: { url: 'https://jakelazaroff.com/rss.xml', title: 'Jake Lazaroff' },
+//     preview: "Web components won't take web development by storm, or show us the One True Way to build websites. What they will do is let us collectively build a rich ecosystem of dynamic components that work with any web stack."
+//   }
+// ]
+```
+
+## Configuration
+
+`webring` is configured by passing in a `Configuration` object into the `run` method.
+
+All possible configuration values can be seen by looking at the [`typedoc` site](https://shepherdjerred.github.io/webring/types/Configuration.html).
+
+## Example
+
+An example of using this project with Astro is located in `example`. The relevant file is [`src/pages/blog/[...slug].astro`](https://github.com/shepherdjerred/webring/blob/971a77ecd0c612850faeb9d16f7775d3e7ca7253/example/src/pages/blog/%5B...slug%5D.astro#L18).
+
+```typescript
+---
+import { type CollectionEntry, getCollection } from "astro:content";
+import BlogPost from "../../layouts/BlogPost.astro";
+import { type Configuration, type Result, run } from "webring";
+
+export async function getStaticPaths() {
+  const posts = await getCollection("blog");
+  return posts.map((post) => ({
+    params: { slug: post.slug },
+    props: post,
+  }));
+}
+type Props = CollectionEntry<"blog">;
+
+const post = Astro.props;
+const { Content } = await post.render();
 
 export const config: Configuration = {
   sources: [
@@ -38,39 +121,33 @@ export const config: Configuration = {
       title: "Jake Lazaroff",
     },
   ],
-  // the output will return the three most recent posts from the above sources
   number: 3,
-  // the output will return santized HTML truncated to 300 characters
   truncate: 300,
-  // if this is defined, we'll cache the results
   cache: {
-    // the file to use as a cache
     cache_file: "webring.json",
-    // how long the cache should remain valid for
     cache_duration_minutes: 60,
   },
 };
 
-// type Result = {
-//   title: string,
-//   url: string,
-//   date: Date,
-//   source: {
-//     url: string,
-//     title: string,
-//   },
-//   // this will be undefined if the RSS feed is empty
-//   preview?: string
-// }[]
 export const result: Result = await run(config);
+---
 
-result.map((entry) => {
-  // do something with the results
-  // for example, you might render each item as an HTML block
-});
+<BlogPost {...post.data}>
+  <Content />
+  <h2>Posts from blogs I read</h2>
+  <ul>
+    {
+      result.map((post) => (
+        <li>
+          <a href={post.url}>{post.title}</a>
+        </li>
+      ))
+    }
+  </ul>
+</BlogPost>
 ```
 
-Here's what I do for my blog:
+## Inspiration
 
-1. Run `map` against the resulting array ([code](https://github.com/shepherdjerred/sjer.red/blob/f72b2b75bf0722ba8ff0fdd45f31c02c2ee5089d/src/components/BlogWebring.astro#L17-L22)).
-1. Render a component for each entry ([code](https://github.com/shepherdjerred/sjer.red/blob/f72b2b75bf0722ba8ff0fdd45f31c02c2ee5089d/src/components/WebringEntry.astro)).
+- https://github.com/lukehsiao/openring-rs
+- https://git.sr.ht/~sircmpwn/openring

package/dist/cache.d.ts

@@ -1,3 +1,4 @@
 import { type Cache, type Result, type ResultEntry, type Source, type CachedConfiguration } from "./types.js";
 export declare function fetchAllCached(config: CachedConfiguration): Promise<Result>;
 export declare function fetchWithCache(source: Source, cache: Cache, config: CachedConfiguration): Promise<ResultEntry | undefined>;
+//# sourceMappingURL=cache.d.ts.map

package/dist/cache.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"cache.d.ts","sourceRoot":"","sources":["../src/cache.ts"],"names":[],"mappings":"AACA,OAAO,EACL,KAAK,KAAK,EACV,KAAK,MAAM,EACX,KAAK,WAAW,EAEhB,KAAK,MAAM,EACX,KAAK,mBAAmB,EAGzB,MAAM,YAAY,CAAC;AAyCpB,wBAAsB,cAAc,CAAC,MAAM,EAAE,mBAAmB,GAAG,OAAO,CAAC,MAAM,CAAC,CAQjF;AAED,wBAAsB,cAAc,CAClC,MAAM,EAAE,MAAM,EACd,KAAK,EAAE,KAAK,EACZ,MAAM,EAAE,mBAAmB,GAC1B,OAAO,CAAC,WAAW,GAAG,SAAS,CAAC,CAgBlC"}

package/dist/cache.js

@@ -1,8 +1,8 @@
 import * as R from "remeda";
 import { CacheSchema, } from "./types.js";
 import { fetch } from "./fetch.js";
-import fs from "fs/promises";
 import { asyncMapFilterUndefined } from "./util.js";
+import fs from "fs/promises";
 async function loadCache({ cache_file }) {
     try {
         await fs.access(cache_file);
@@ -14,7 +14,10 @@ async function loadCache({ cache_file }) {
     }
 }
 async function saveCache({ cache_file }, cache) {
-    await fs.mkdir(cache_file.split("/").slice(0, -1).join("/"), { recursive: true });
+    const dir = cache_file.split("/").slice(0, -1).join("/");
+    if (dir !== "") {
+        await fs.mkdir(cache_file.split("/").slice(0, -1).join("/"), { recursive: true });
+    }
     await fs.writeFile(cache_file, JSON.stringify(cache));
 }
 function toCacheEntry(result, now) {
@@ -52,3 +55,4 @@ export async function fetchWithCache(source, cache, config) {
     console.log(`Fetching ${source.url}`);
     return fetch(source, config.truncate);
 }
+//# sourceMappingURL=cache.js.map

package/dist/cache.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"cache.js","sourceRoot":"","sources":["../src/cache.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,CAAC,MAAM,QAAQ,CAAC;AAC5B,OAAO,EAOL,WAAW,GAEZ,MAAM,YAAY,CAAC;AACpB,OAAO,EAAE,KAAK,EAAE,MAAM,YAAY,CAAC;AACnC,OAAO,EAAE,uBAAuB,EAAE,MAAM,WAAW,CAAC;AACpD,OAAO,EAAE,MAAM,aAAa,CAAC;AAE7B,KAAK,UAAU,SAAS,CAAC,EAAE,UAAU,EAAsB;IACzD,IAAI,CAAC;QACH,MAAM,EAAE,CAAC,MAAM,CAAC,UAAU,CAAC,CAAC;QAC5B,MAAM,SAAS,GAAG,MAAM,EAAE,CAAC,QAAQ,CAAC,UAAU,CAAC,CAAC;QAChD,OAAO,WAAW,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,SAAS,CAAC,QAAQ,EAAE,CAAC,CAAC,CAAC;IAC7D,CAAC;IAAC,OAAO,CAAC,EAAE,CAAC;QACX,OAAO,EAAE,CAAC;IACZ,CAAC;AACH,CAAC;AAED,KAAK,UAAU,SAAS,CAAC,EAAE,UAAU,EAAsB,EAAE,KAAY;IACvE,MAAM,GAAG,GAAG,UAAU,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;IACzD,IAAI,GAAG,KAAK,EAAE,EAAE,CAAC;QACf,MAAM,EAAE,CAAC,KAAK,CAAC,UAAU,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;IACpF,CAAC;IACD,MAAM,EAAE,CAAC,SAAS,CAAC,UAAU,EAAE,IAAI,CAAC,SAAS,CAAC,KAAK,CAAC,CAAC,CAAC;AACxD,CAAC;AAED,SAAS,YAAY,CAAC,MAAmB,EAAE,GAAS;IAClD,OAAO,CAAC,MAAM,CAAC,MAAM,CAAC,GAAG,EAAE,EAAE,SAAS,EAAE,GAAG,EAAE,IAAI,EAAE,MAAM,EAAE,CAAC,CAAC;AAC/D,CAAC;AAED,SAAS,OAAO,CAAC,OAAsB,EAAE,GAAS;IAChD,OAAO,CAAC,CAAC,IAAI,CACX,OAAO,EACP,CAAC,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,YAAY,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC,EAC5C,CAAC,CAAC,WAAW,EAAE,CAChB,CAAC;AACJ,CAAC;AAED,SAAS,WAAW,CAAC,OAAsB,EAAE,MAA2B;IACtE,MAAM,GAAG,GAAG,IAAI,IAAI,EAAE,CAAC;IACvB,MAAM,YAAY,GAAG,OAAO,CAAC,OAAO,EAAE,GAAG,CAAC,CAAC;IAC3C,OAAO,SAAS,CAAC,MAAM,CAAC,KAAK,EAAE,YAAY,CAAC,CAAC;AAC/C,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,cAAc,CAAC,MAA2B;IAC9D,MAAM,KAAK,GAAG,MAAM,SAAS,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;IAE5C,MAAM,OAAO,GAAG,MAAM,uBAAuB,CAAC,MAAM,CAAC,OAAO,EAAE,CAAC,MAAM,EAAE,EAAE,CAAC,cAAc,CAAC,MAAM,EAAE,KAAK,EAAE,MAAM,CAAC,CAAC,CAAC;IAEjH,MAAM,WAAW,CAAC,OAAO,EAAE,MAAM,CAAC,CAAC;IAEnC,OAAO,OAAO,CAAC;AACjB,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,cAAc,CAClC,MAAc,EACd,KAAY,EACZ,MAA2B;IAE3B,MAAM,UAAU,GAAG,KAAK,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;IACrC,IAAI,UAAU,EAAE,CAAC;QACf,MAAM,GAAG,GAAG,IAAI,IAAI,EAAE,CAAC;QACvB,IAAI,GAAG,CAAC,OAAO,EAAE,GAAG,UAAU,CAAC,SAAS,CAAC,OAAO,EAAE,GAAG,MAAM,CAAC,KAAK,CAAC,sBAAsB,GAAG,EAAE,GAAG,IAAI,EAAE,CAAC;YACrG,OAAO,CAAC,GAAG,CAAC,yBAAyB,MAAM,CAAC,GAAG,GAAG,CAAC,CAAC;YACpD,OAAO,OAAO,CAAC,OAAO,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC;QAC1C,CAAC;aAAM,CAAC;YACN,OAAO,CAAC,GAAG,CAAC,mBAAmB,MAAM,CAAC,GAAG,cAAc,CAAC,CAAC;QAC3D,CAAC;IACH,CAAC;SAAM,CAAC;QACN,OAAO,CAAC,GAAG,CAAC,sBAAsB,MAAM,CAAC,GAAG,GAAG,CAAC,CAAC;IACnD,CAAC;IAED,OAAO,CAAC,GAAG,CAAC,YAAY,MAAM,CAAC,GAAG,EAAE,CAAC,CAAC;IACtC,OAAO,KAAK,CAAC,MAAM,EAAE,MAAM,CAAC,QAAQ,CAAC,CAAC;AACxC,CAAC"}

package/dist/fetch.d.ts

@@ -10,3 +10,4 @@ export declare function fetchAll(config: Configuration): Promise<{
     preview?: string | undefined;
 }[]>;
 export declare function fetch(source: Source, length: number): Promise<ResultEntry | undefined>;
+//# sourceMappingURL=fetch.d.ts.map

package/dist/fetch.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"fetch.d.ts","sourceRoot":"","sources":["../src/fetch.ts"],"names":[],"mappings":"AAGA,OAAO,EAAE,KAAK,MAAM,EAAE,KAAK,WAAW,EAAmB,KAAK,aAAa,EAAE,MAAM,YAAY,CAAC;AAOhG,wBAAsB,QAAQ,CAAC,MAAM,EAAE,aAAa;;;;;;;;;KAEnD;AAED,wBAAsB,KAAK,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,GAAG,OAAO,CAAC,WAAW,GAAG,SAAS,CAAC,CAsC5F"}

package/dist/fetch.js

@@ -35,3 +35,4 @@ export async function fetch(source, length) {
         return undefined;
     }
 }
+//# sourceMappingURL=fetch.js.map

package/dist/fetch.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"fetch.js","sourceRoot":"","sources":["../src/fetch.ts"],"names":[],"mappings":"AAAA,OAAO,MAAM,MAAM,YAAY,CAAC;AAChC,OAAO,YAAY,MAAM,eAAe,CAAC;AACzC,OAAO,QAAQ,MAAM,eAAe,CAAC;AACrC,OAAO,EAAiC,eAAe,EAAsB,MAAM,YAAY,CAAC;AAChG,OAAO,KAAK,CAAC,MAAM,QAAQ,CAAC;AAC5B,OAAO,EAAE,uBAAuB,EAAE,MAAM,WAAW,CAAC;AAEpD,sFAAsF;AACtF,MAAM,UAAU,GAA4B,QAA8C,CAAC;AAE3F,MAAM,CAAC,KAAK,UAAU,QAAQ,CAAC,MAAqB;IAClD,OAAO,MAAM,uBAAuB,CAAC,MAAM,CAAC,OAAO,EAAE,CAAC,MAAM,EAAE,EAAE,CAAC,KAAK,CAAC,MAAM,EAAE,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC;AACnG,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,KAAK,CAAC,MAAc,EAAE,MAAc;IACxD,MAAM,MAAM,GAAG,IAAI,MAAM,EAAE,CAAC;IAC5B,IAAI,CAAC;QACH,MAAM,IAAI,GAAG,MAAM,MAAM,CAAC,QAAQ,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;QAE/C,MAAM,SAAS,GAAG,CAAC,CAAC,IAAI,CACtB,IAAI,CAAC,KAAK,EACV,CAAC,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,eAAe,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,EAC5C,CAAC,CAAC,MAAM,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,OAAO,EAAE,CAAC,EACjD,CAAC,CAAC,OAAO,EAAE,EACX,CAAC,CAAC,KAAK,EAAE,CACV,CAAC;QAEF,IAAI,CAAC,SAAS,EAAE,CAAC;YACf,MAAM,IAAI,KAAK,CAAC,wBAAwB,CAAC,CAAC;QAC5C,CAAC;QAED,MAAM,OAAO,GACX,SAAS,CAAC,cAAc,IAAI,SAAS,CAAC,OAAO,IAAI,SAAS,CAAC,WAAW,IAAI,SAAS,CAAC,iBAAiB,CAAC,CAAC;QAEzG,OAAO;YACL,KAAK,EAAE,SAAS,CAAC,KAAK;YACtB,GAAG,EAAE,SAAS,CAAC,IAAI;YACnB,IAAI,EAAE,IAAI,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC;YAC9B,MAAM;YACN,OAAO,EAAE,OAAO;gBACd,CAAC,CAAC,UAAU,CACR,YAAY,CAAC,OAAO,EAAE;oBACpB,oBAAoB,EAAE,KAAK;iBAC5B,CAAC,EACF,MAAM,CACP;gBACH,CAAC,CAAC,SAAS;SACd,CAAC;IACJ,CAAC;IAAC,OAAO,CAAC,EAAE,CAAC;QACX,OAAO,CAAC,KAAK,CAAC,kBAAkB,MAAM,CAAC,GAAG,KAAK,CAAW,EAAE,CAAC,CAAC;QAC9D,OAAO,SAAS,CAAC;IACnB,CAAC;AACH,CAAC"}

package/dist/index.d.ts

@@ -1,3 +1,4 @@
 import { type Configuration, type Result } from "./types.js";
 export declare function run(config: Configuration): Promise<Result>;
 export * from "./types.js";
+//# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAGA,OAAO,EAAE,KAAK,aAAa,EAAE,KAAK,MAAM,EAA6B,MAAM,YAAY,CAAC;AAExF,wBAAsB,GAAG,CAAC,MAAM,EAAE,aAAa,GAAG,OAAO,CAAC,MAAM,CAAC,CAoBhE;AAED,cAAc,YAAY,CAAC"}

package/dist/index.js

@@ -17,3 +17,4 @@ export async function run(config) {
     return topResults;
 }
 export * from "./types.js";
+//# sourceMappingURL=index.js.map

package/dist/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,CAAC,MAAM,QAAQ,CAAC;AAC5B,OAAO,EAAE,cAAc,IAAI,cAAc,EAAE,MAAM,YAAY,CAAC;AAC9D,OAAO,EAAE,QAAQ,IAAI,gBAAgB,EAAE,MAAM,YAAY,CAAC;AAC1D,OAAO,EAAmC,yBAAyB,EAAE,MAAM,YAAY,CAAC;AAExF,MAAM,CAAC,KAAK,UAAU,GAAG,CAAC,MAAqB;IAC7C,MAAM,EAAE,OAAO,EAAE,IAAI,EAAE,GAAG,yBAAyB,CAAC,SAAS,CAAC,MAAM,CAAC,CAAC;IAEtE,IAAI,MAAc,CAAC;IACnB,IAAI,OAAO,EAAE,CAAC;QACZ,OAAO,CAAC,GAAG,CAAC,kBAAkB,IAAI,CAAC,KAAK,CAAC,UAAU,GAAG,CAAC,CAAC;QACxD,MAAM,GAAG,MAAM,cAAc,CAAC,IAAI,CAAC,CAAC;IACtC,CAAC;SAAM,CAAC;QACN,OAAO,CAAC,GAAG,CAAC,iBAAiB,CAAC,CAAC;QAC/B,MAAM,GAAG,MAAM,gBAAgB,CAAC,MAAM,CAAC,CAAC;IAC1C,CAAC;IAED,MAAM,UAAU,GAAG,CAAC,CAAC,IAAI,CACvB,MAAM,EACN,CAAC,CAAC,MAAM,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC,OAAO,EAAE,CAAC,EAC3C,CAAC,CAAC,OAAO,EAAE,EACX,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,CACtB,CAAC;IAEF,OAAO,UAAU,CAAC;AACpB,CAAC;AAED,cAAc,YAAY,CAAC"}

package/dist/index.test.d.ts

@@ -1 +1,2 @@
 export {};
+//# sourceMappingURL=index.test.d.ts.map

package/dist/index.test.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.test.d.ts","sourceRoot":"","sources":["../src/index.test.ts"],"names":[],"mappings":""}

package/dist/index.test.js

@@ -4,7 +4,7 @@ import { tmpdir } from "os";
 import { mkdtemp } from "fs/promises";
 import { join } from "path";
 // TODO: intercept network requests
-test("it should fetch an RSS feed without caching", async () => {
+test("it should fetch an RSS feed without caching", { timeout: 30000 }, async () => {
     const config = {
         sources: [
             {
@@ -18,7 +18,7 @@ test("it should fetch an RSS feed without caching", async () => {
     const result = await run(config);
     expect(result).toMatchSnapshot();
 });
-test("it should fetch several RSS feeds", async () => {
+test("it should fetch several RSS feeds", { timeout: 30000 }, async () => {
     const config = {
         sources: [
             {
@@ -29,58 +29,14 @@ test("it should fetch several RSS feeds", async () => {
                 url: "https://danluu.com/atom.xml",
                 title: "Dan Luu",
             },
-            {
-                url: "https://jakelazaroff.com/rss.xml",
-                title: "Jake Lazaroff",
-            },
-            {
-                url: "https://awesomekling.github.io/feed.xml",
-                title: "Andreas Kling",
-            },
-            {
-                url: "https://xeiaso.net/blog.rss",
-                title: "Xe Iaso",
-            },
-            {
-                url: "https://ciechanow.ski/atom.xml",
-                title: "Bartosz Ciechanowski",
-            },
-            {
-                url: "https://explained-from-first-principles.com/feed.xml",
-                title: "Explained From First Principles",
-            },
-            {
-                url: "http://www.aaronsw.com/2002/feeds/pgessays.rss",
-                title: "Paul Graham",
-            },
-            {
-                url: "https://samwho.dev/rss.xml",
-                title: "Sam Rose",
-            },
-            {
-                url: "https://rachelbythebay.com/w/atom.xml",
-                title: "Rachel Kroll",
-            },
-            {
-                url: "https://brr.fyi/feed.xml",
-                title: "brr.fyi",
-            },
-            {
-                url: "https://devblogs.microsoft.com/oldnewthing/feed",
-                title: "The Old New Thing",
-            },
-            {
-                url: "https://ludic.mataroa.blog/rss/",
-                title: "Ludicity",
-            },
         ],
-        number: 20,
+        number: 3,
         truncate: 300,
     };
     const result = await run(config);
     expect(result).toMatchSnapshot();
 });
-test("it should fetch an RSS feed with caching", async () => {
+test("it should fetch an RSS feed with caching", { timeout: 30000 }, async () => {
     const config = {
         sources: [
             {
@@ -104,3 +60,4 @@ async function createTempDir() {
     const dir = join(ostmpdir, "unit-test-");
     return await mkdtemp(dir);
 }
+//# sourceMappingURL=index.test.js.map

package/dist/index.test.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.test.js","sourceRoot":"","sources":["../src/index.test.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,QAAQ,CAAC;AAEtC,OAAO,EAAE,GAAG,EAAE,MAAM,YAAY,CAAC;AACjC,OAAO,EAAE,MAAM,EAAE,MAAM,IAAI,CAAC;AAC5B,OAAO,EAAE,OAAO,EAAE,MAAM,aAAa,CAAC;AACtC,OAAO,EAAE,IAAI,EAAE,MAAM,MAAM,CAAC;AAE5B,mCAAmC;AACnC,IAAI,CAAC,6CAA6C,EAAE,EAAE,OAAO,EAAE,KAAK,EAAE,EAAE,KAAK,IAAI,EAAE;IACjF,MAAM,MAAM,GAAkB;QAC5B,OAAO,EAAE;YACP;gBACE,KAAK,EAAE,iBAAiB;gBACxB,GAAG,EAAE,0BAA0B;aAChC;SACF;QACD,MAAM,EAAE,CAAC;QACT,QAAQ,EAAE,GAAG;KACd,CAAC;IAEF,MAAM,MAAM,GAAG,MAAM,GAAG,CAAC,MAAM,CAAC,CAAC;IACjC,MAAM,CAAC,MAAM,CAAC,CAAC,eAAe,EAAE,CAAC;AACnC,CAAC,CAAC,CAAC;AAEH,IAAI,CAAC,mCAAmC,EAAE,EAAE,OAAO,EAAE,KAAK,EAAE,EAAE,KAAK,IAAI,EAAE;IACvE,MAAM,MAAM,GAAkB;QAC5B,OAAO,EAAE;YACP;gBACE,GAAG,EAAE,wCAAwC;gBAC7C,KAAK,EAAE,cAAc;aACtB;YACD;gBACE,GAAG,EAAE,6BAA6B;gBAClC,KAAK,EAAE,SAAS;aACjB;SACF;QACD,MAAM,EAAE,CAAC;QACT,QAAQ,EAAE,GAAG;KACd,CAAC;IAEF,MAAM,MAAM,GAAG,MAAM,GAAG,CAAC,MAAM,CAAC,CAAC;IACjC,MAAM,CAAC,MAAM,CAAC,CAAC,eAAe,EAAE,CAAC;AACnC,CAAC,CAAC,CAAC;AAEH,IAAI,CAAC,0CAA0C,EAAE,EAAE,OAAO,EAAE,KAAK,EAAE,EAAE,KAAK,IAAI,EAAE;IAC9E,MAAM,MAAM,GAAkB;QAC5B,OAAO,EAAE;YACP;gBACE,KAAK,EAAE,iBAAiB;gBACxB,GAAG,EAAE,0BAA0B;aAChC;SACF;QACD,MAAM,EAAE,CAAC;QACT,QAAQ,EAAE,GAAG;QACb,KAAK,EAAE;YACL,UAAU,EAAE,GAAG,MAAM,aAAa,EAAE,aAAa;YACjD,sBAAsB,EAAE,CAAC;SAC1B;KACF,CAAC;IAEF,MAAM,MAAM,GAAG,MAAM,GAAG,CAAC,MAAM,CAAC,CAAC;IACjC,MAAM,CAAC,MAAM,CAAC,CAAC,eAAe,EAAE,CAAC;AACnC,CAAC,CAAC,CAAC;AAEH,oDAAoD;AACpD,KAAK,UAAU,aAAa;IAC1B,MAAM,QAAQ,GAAG,MAAM,EAAE,CAAC;IAC1B,MAAM,GAAG,GAAG,IAAI,CAAC,QAAQ,EAAE,YAAY,CAAC,CAAC;IACzC,OAAO,MAAM,OAAO,CAAC,GAAG,CAAC,CAAC;AAC5B,CAAC"}

package/dist/types.d.ts

@@ -1,7 +1,10 @@
 import { z } from "zod";
 export type Source = z.infer<typeof SourceSchema>;
+/** An RSS source */
 declare const SourceSchema: z.ZodObject<{
+    /** The URL of an RSS feed */
     url: z.ZodString;
+    /** A title to describe the feed */
     title: z.ZodString;
 }, "strip", z.ZodTypeAny, {
     url: string;
@@ -11,8 +14,11 @@ declare const SourceSchema: z.ZodObject<{
     title: string;
 }>;
 export type CacheConfiguration = z.infer<typeof CacheConfigurationSchema>;
+/** Configuration for the cache */
 declare const CacheConfigurationSchema: z.ZodObject<{
+    /** How long to cache a result for */
     cache_duration_minutes: z.ZodDefault<z.ZodNumber>;
+    /** The location of a file to use as a cache */
     cache_file: z.ZodDefault<z.ZodString>;
 }, "strip", z.ZodTypeAny, {
     cache_duration_minutes: number;
@@ -22,9 +28,13 @@ declare const CacheConfigurationSchema: z.ZodObject<{
     cache_file?: string | undefined;
 }>;
 export type Configuration = z.infer<typeof ConfigurationSchema>;
+/** A configuration object with caching possibly configured */
 declare const ConfigurationSchema: z.ZodObject<{
+    /** A list of sources to fetch */
     sources: z.ZodArray<z.ZodObject<{
+        /** The URL of an RSS feed */
         url: z.ZodString;
+        /** A title to describe the feed */
         title: z.ZodString;
     }, "strip", z.ZodTypeAny, {
         url: string;
@@ -33,10 +43,15 @@ declare const ConfigurationSchema: z.ZodObject<{
         url: string;
         title: string;
     }>, "many">;
+    /** Return the n latest updates from the source list. */
     number: z.ZodDefault<z.ZodNumber>;
+    /** How many words the preview field should be truncated to in characters after HTML has been sanitized and parsed. */
     truncate: z.ZodDefault<z.ZodNumber>;
+    /** Configuration for the cache */
     cache: z.ZodOptional<z.ZodObject<{
+        /** How long to cache a result for */
         cache_duration_minutes: z.ZodDefault<z.ZodNumber>;
+        /** The location of a file to use as a cache */
         cache_file: z.ZodDefault<z.ZodString>;
     }, "strip", z.ZodTypeAny, {
         cache_duration_minutes: number;
@@ -69,9 +84,13 @@ declare const ConfigurationSchema: z.ZodObject<{
     } | undefined;
 }>;
 export type CachedConfiguration = z.infer<typeof CachedConfigurationSchema>;
+/** A configuration object with caching definitely configured */
 export declare const CachedConfigurationSchema: z.ZodObject<z.objectUtil.extendShape<{
+    /** A list of sources to fetch */
     sources: z.ZodArray<z.ZodObject<{
+        /** The URL of an RSS feed */
         url: z.ZodString;
+        /** A title to describe the feed */
         title: z.ZodString;
     }, "strip", z.ZodTypeAny, {
         url: string;
@@ -80,10 +99,15 @@ export declare const CachedConfigurationSchema: z.ZodObject<z.objectUtil.extendS
         url: string;
         title: string;
     }>, "many">;
+    /** Return the n latest updates from the source list. */
     number: z.ZodDefault<z.ZodNumber>;
+    /** How many words the preview field should be truncated to in characters after HTML has been sanitized and parsed. */
     truncate: z.ZodDefault<z.ZodNumber>;
+    /** Configuration for the cache */
     cache: z.ZodOptional<z.ZodObject<{
+        /** How long to cache a result for */
         cache_duration_minutes: z.ZodDefault<z.ZodNumber>;
+        /** The location of a file to use as a cache */
         cache_file: z.ZodDefault<z.ZodString>;
     }, "strip", z.ZodTypeAny, {
         cache_duration_minutes: number;
@@ -93,8 +117,11 @@ export declare const CachedConfigurationSchema: z.ZodObject<z.objectUtil.extendS
         cache_file?: string | undefined;
     }>>;
 }, {
+    /** Configuration for the cache */
     cache: z.ZodObject<{
+        /** How long to cache a result for */
         cache_duration_minutes: z.ZodDefault<z.ZodNumber>;
+        /** The location of a file to use as a cache */
         cache_file: z.ZodDefault<z.ZodString>;
     }, "strip", z.ZodTypeAny, {
         cache_duration_minutes: number;
@@ -127,12 +154,19 @@ export declare const CachedConfigurationSchema: z.ZodObject<z.objectUtil.extendS
     truncate?: number | undefined;
 }>;
 export type ResultEntry = z.infer<typeof ResultEntrySchema>;
+/** A single entry from an RSS feed */
 declare const ResultEntrySchema: z.ZodObject<{
+    /** The title of the entry */
     title: z.ZodString;
+    /** A direct link to the entry */
     url: z.ZodString;
+    /** The date of the entry */
     date: z.ZodDate;
+    /** The source the entry is from */
     source: z.ZodObject<{
+        /** The URL of an RSS feed */
         url: z.ZodString;
+        /** A title to describe the feed */
         title: z.ZodString;
     }, "strip", z.ZodTypeAny, {
         url: string;
@@ -141,6 +175,7 @@ declare const ResultEntrySchema: z.ZodObject<{
         url: string;
         title: string;
     }>;
+    /** A preview of the entry. This may contain sanitized HTML. */
     preview: z.ZodOptional<z.ZodString>;
 }, "strip", z.ZodTypeAny, {
     url: string;
@@ -162,12 +197,19 @@ declare const ResultEntrySchema: z.ZodObject<{
     preview?: string | undefined;
 }>;
 export type Result = z.infer<typeof ResultSchema>;
+/** A list of results */
 declare const ResultSchema: z.ZodArray<z.ZodObject<{
+    /** The title of the entry */
     title: z.ZodString;
+    /** A direct link to the entry */
     url: z.ZodString;
+    /** The date of the entry */
     date: z.ZodDate;
+    /** The source the entry is from */
     source: z.ZodObject<{
+        /** The URL of an RSS feed */
         url: z.ZodString;
+        /** A title to describe the feed */
         title: z.ZodString;
     }, "strip", z.ZodTypeAny, {
         url: string;
@@ -176,6 +218,7 @@ declare const ResultSchema: z.ZodArray<z.ZodObject<{
         url: string;
         title: string;
     }>;
+    /** A preview of the entry. This may contain sanitized HTML. */
     preview: z.ZodOptional<z.ZodString>;
 }, "strip", z.ZodTypeAny, {
     url: string;
@@ -197,14 +240,23 @@ declare const ResultSchema: z.ZodArray<z.ZodObject<{
     preview?: string | undefined;
 }>, "many">;
 export type CacheEntry = z.infer<typeof CacheEntrySchema>;
+/** A single cache entry */
 export declare const CacheEntrySchema: z.ZodObject<{
+    /** The time a source was last checked */
     timestamp: z.ZodDate;
+    /** The data from the source */
     data: z.ZodObject<{
+        /** The title of the entry */
         title: z.ZodString;
+        /** A direct link to the entry */
         url: z.ZodString;
+        /** The date of the entry */
         date: z.ZodDate;
+        /** The source the entry is from */
         source: z.ZodObject<{
+            /** The URL of an RSS feed */
             url: z.ZodString;
+            /** A title to describe the feed */
             title: z.ZodString;
         }, "strip", z.ZodTypeAny, {
             url: string;
@@ -213,6 +265,7 @@ export declare const CacheEntrySchema: z.ZodObject<{
             url: string;
             title: string;
         }>;
+        /** A preview of the entry. This may contain sanitized HTML. */
         preview: z.ZodOptional<z.ZodString>;
     }, "strip", z.ZodTypeAny, {
         url: string;
@@ -259,14 +312,23 @@ export declare const CacheEntrySchema: z.ZodObject<{
     };
 }>;
 export type Cache = z.infer<typeof CacheSchema>;
+/** A mapping of source URLs to cache entries */
 export declare const CacheSchema: z.ZodRecord<z.ZodString, z.ZodObject<{
+    /** The time a source was last checked */
     timestamp: z.ZodDate;
+    /** The data from the source */
     data: z.ZodObject<{
+        /** The title of the entry */
         title: z.ZodString;
+        /** A direct link to the entry */
         url: z.ZodString;
+        /** The date of the entry */
         date: z.ZodDate;
+        /** The source the entry is from */
         source: z.ZodObject<{
+            /** The URL of an RSS feed */
             url: z.ZodString;
+            /** A title to describe the feed */
             title: z.ZodString;
         }, "strip", z.ZodTypeAny, {
             url: string;
@@ -275,6 +337,7 @@ export declare const CacheSchema: z.ZodRecord<z.ZodString, z.ZodObject<{
             url: string;
             title: string;
         }>;
+        /** A preview of the entry. This may contain sanitized HTML. */
         preview: z.ZodOptional<z.ZodString>;
     }, "strip", z.ZodTypeAny, {
         url: string;
@@ -320,6 +383,7 @@ export declare const CacheSchema: z.ZodRecord<z.ZodString, z.ZodObject<{
         preview?: string | undefined;
     };
 }>>;
+/** The expected format fetched RSS feed entries */
 export declare const FeedEntrySchema: z.ZodEffects<z.ZodObject<{
     title: z.ZodString;
     link: z.ZodString;
@@ -366,3 +430,4 @@ export declare const FeedEntrySchema: z.ZodEffects<z.ZodObject<{
     description?: string | undefined;
 }>;
 export {};
+//# sourceMappingURL=types.d.ts.map

package/dist/types.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"types.d.ts","sourceRoot":"","sources":["../src/types.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AAExB,MAAM,MAAM,MAAM,GAAG,CAAC,CAAC,KAAK,CAAC,OAAO,YAAY,CAAC,CAAC;AAClD,oBAAoB;AACpB,QAAA,MAAM,YAAY;IAChB,6BAA6B;;IAE7B,mCAAmC;;;;;;;;EAEnC,CAAC;AAEH,MAAM,MAAM,kBAAkB,GAAG,CAAC,CAAC,KAAK,CAAC,OAAO,wBAAwB,CAAC,CAAC;AAC1E,kCAAkC;AAClC,QAAA,MAAM,wBAAwB;IAC5B,qCAAqC;;IAErC,+CAA+C;;;;;;;;EAE/C,CAAC;AAEH,MAAM,MAAM,aAAa,GAAG,CAAC,CAAC,KAAK,CAAC,OAAO,mBAAmB,CAAC,CAAC;AAChE,8DAA8D;AAC9D,QAAA,MAAM,mBAAmB;IACvB,iCAAiC;;QAlBjC,6BAA6B;;QAE7B,mCAAmC;;;;;;;;;IAkBnC,wDAAwD;;IAExD,sHAAsH;;IAEtH,kCAAkC;;QAflC,qCAAqC;;QAErC,+CAA+C;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;EAe/C,CAAC;AAEH,MAAM,MAAM,mBAAmB,GAAG,CAAC,CAAC,KAAK,CAAC,OAAO,yBAAyB,CAAC,CAAC;AAC5E,gEAAgE;AAChE,eAAO,MAAM,yBAAyB;IAZpC,iCAAiC;;QAlBjC,6BAA6B;;QAE7B,mCAAmC;;;;;;;;;IAkBnC,wDAAwD;;IAExD,sHAAsH;;IAEtH,kCAAkC;;QAflC,qCAAqC;;QAErC,+CAA+C;;;;;;;;;;IAoB/C,kCAAkC;;QAtBlC,qCAAqC;;QAErC,+CAA+C;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;EAsB/C,CAAC;AAEH,MAAM,MAAM,WAAW,GAAG,CAAC,CAAC,KAAK,CAAC,OAAO,iBAAiB,CAAC,CAAC;AAC5D,sCAAsC;AACtC,QAAA,MAAM,iBAAiB;IACrB,6BAA6B;;IAE7B,iCAAiC;;IAEjC,4BAA4B;;IAE5B,mCAAmC;;QA5CnC,6BAA6B;;QAE7B,mCAAmC;;;;;;;;;IA4CnC,+DAA+D;;;;;;;;;;;;;;;;;;;;EAE/D,CAAC;AAEH,MAAM,MAAM,MAAM,GAAG,CAAC,CAAC,KAAK,CAAC,OAAO,YAAY,CAAC,CAAC;AAClD,wBAAwB;AACxB,QAAA,MAAM,YAAY;IAdhB,6BAA6B;;IAE7B,iCAAiC;;IAEjC,4BAA4B;;IAE5B,mCAAmC;;QA5CnC,6BAA6B;;QAE7B,mCAAmC;;;;;;;;;IA4CnC,+DAA+D;;;;;;;;;;;;;;;;;;;;WAMlB,CAAC;AAEhD,MAAM,MAAM,UAAU,GAAG,CAAC,CAAC,KAAK,CAAC,OAAO,gBAAgB,CAAC,CAAC;AAC1D,2BAA2B;AAC3B,eAAO,MAAM,gBAAgB;IAC3B,yCAAyC;;IAEzC,+BAA+B;;QArB/B,6BAA6B;;QAE7B,iCAAiC;;QAEjC,4BAA4B;;QAE5B,mCAAmC;;YA5CnC,6BAA6B;;YAE7B,mCAAmC;;;;;;;;;QA4CnC,+DAA+D;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;EAe/D,CAAC;AAEH,MAAM,MAAM,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC,OAAO,WAAW,CAAC,CAAC;AAChD,gDAAgD;AAChD,eAAO,MAAM,WAAW;IARtB,yCAAyC;;IAEzC,+BAA+B;;QArB/B,6BAA6B;;QAE7B,iCAAiC;;QAEjC,4BAA4B;;QAE5B,mCAAmC;;YA5CnC,6BAA6B;;YAE7B,mCAAmC;;;;;;;;;QA4CnC,+DAA+D;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAmBZ,CAAC;AAEtD,mDAAmD;AACnD,eAAO,MAAM,eAAe;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;EAyBxB,CAAC"}

package/dist/types.js

@@ -1,40 +1,59 @@
 import { z } from "zod";
+/** An RSS source */
 const SourceSchema = z.object({
-    // the url of the feed
+    /** The URL of an RSS feed */
     url: z.string(),
-    // a title for the feed
-    title: z.string(),
+    /** A title to describe the feed */
+    title: z.string().describe("A title for the feed"),
 });
+/** Configuration for the cache */
 const CacheConfigurationSchema = z.object({
-    // how long to cache a results for
+    /** How long to cache a result for */
     cache_duration_minutes: z.number().default(60),
+    /** The location of a file to use as a cache */
     cache_file: z.string().default("cache.json"),
 });
+/** A configuration object with caching possibly configured */
 const ConfigurationSchema = z.object({
-    // list of sources to fetch
+    /** A list of sources to fetch */
     sources: SourceSchema.array(),
-    // how many entries to return
+    /** Return the n latest updates from the source list. */
     number: z.number().default(3),
-    // how many words to truncate the preview to
+    /** How many words the preview field should be truncated to in characters after HTML has been sanitized and parsed. */
     truncate: z.number().default(300),
+    /** Configuration for the cache */
     cache: CacheConfigurationSchema.optional(),
 });
+/** A configuration object with caching definitely configured */
 export const CachedConfigurationSchema = ConfigurationSchema.extend({
+    /** Configuration for the cache */
     cache: CacheConfigurationSchema,
 });
+/** A single entry from an RSS feed */
 const ResultEntrySchema = z.object({
+    /** The title of the entry */
     title: z.string(),
+    /** A direct link to the entry */
     url: z.string(),
+    /** The date of the entry */
     date: z.coerce.date(),
+    /** The source the entry is from */
     source: SourceSchema,
+    /** A preview of the entry. This may contain sanitized HTML. */
     preview: z.string().optional(),
 });
+/** A list of results */
 const ResultSchema = z.array(ResultEntrySchema);
+/** A single cache entry */
 export const CacheEntrySchema = z.object({
+    /** The time a source was last checked */
     timestamp: z.coerce.date(),
+    /** The data from the source */
     data: ResultEntrySchema,
 });
+/** A mapping of source URLs to cache entries */
 export const CacheSchema = z.record(CacheEntrySchema);
+/** The expected format fetched RSS feed entries */
 export const FeedEntrySchema = z
     .object({
     title: z.string(),
@@ -61,3 +80,4 @@ export const FeedEntrySchema = z
         "content:encoded": entry["content:encoded"],
     };
 });
+//# sourceMappingURL=types.js.map

package/dist/types.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"types.js","sourceRoot":"","sources":["../src/types.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AAGxB,oBAAoB;AACpB,MAAM,YAAY,GAAG,CAAC,CAAC,MAAM,CAAC;IAC5B,6BAA6B;IAC7B,GAAG,EAAE,CAAC,CAAC,MAAM,EAAE;IACf,mCAAmC;IACnC,KAAK,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,CAAC,sBAAsB,CAAC;CACnD,CAAC,CAAC;AAGH,kCAAkC;AAClC,MAAM,wBAAwB,GAAG,CAAC,CAAC,MAAM,CAAC;IACxC,qCAAqC;IACrC,sBAAsB,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,EAAE,CAAC;IAC9C,+CAA+C;IAC/C,UAAU,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,YAAY,CAAC;CAC7C,CAAC,CAAC;AAGH,8DAA8D;AAC9D,MAAM,mBAAmB,GAAG,CAAC,CAAC,MAAM,CAAC;IACnC,iCAAiC;IACjC,OAAO,EAAE,YAAY,CAAC,KAAK,EAAE;IAC7B,wDAAwD;IACxD,MAAM,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,CAAC,CAAC;IAC7B,sHAAsH;IACtH,QAAQ,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,GAAG,CAAC;IACjC,kCAAkC;IAClC,KAAK,EAAE,wBAAwB,CAAC,QAAQ,EAAE;CAC3C,CAAC,CAAC;AAGH,gEAAgE;AAChE,MAAM,CAAC,MAAM,yBAAyB,GAAG,mBAAmB,CAAC,MAAM,CAAC;IAClE,kCAAkC;IAClC,KAAK,EAAE,wBAAwB;CAChC,CAAC,CAAC;AAGH,sCAAsC;AACtC,MAAM,iBAAiB,GAAG,CAAC,CAAC,MAAM,CAAC;IACjC,6BAA6B;IAC7B,KAAK,EAAE,CAAC,CAAC,MAAM,EAAE;IACjB,iCAAiC;IACjC,GAAG,EAAE,CAAC,CAAC,MAAM,EAAE;IACf,4BAA4B;IAC5B,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,IAAI,EAAE;IACrB,mCAAmC;IACnC,MAAM,EAAE,YAAY;IACpB,+DAA+D;IAC/D,OAAO,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE;CAC/B,CAAC,CAAC;AAGH,wBAAwB;AACxB,MAAM,YAAY,GAAG,CAAC,CAAC,KAAK,CAAC,iBAAiB,CAAC,CAAC;AAGhD,2BAA2B;AAC3B,MAAM,CAAC,MAAM,gBAAgB,GAAG,CAAC,CAAC,MAAM,CAAC;IACvC,yCAAyC;IACzC,SAAS,EAAE,CAAC,CAAC,MAAM,CAAC,IAAI,EAAE;IAC1B,+BAA+B;IAC/B,IAAI,EAAE,iBAAiB;CACxB,CAAC,CAAC;AAGH,gDAAgD;AAChD,MAAM,CAAC,MAAM,WAAW,GAAG,CAAC,CAAC,MAAM,CAAC,gBAAgB,CAAC,CAAC;AAEtD,mDAAmD;AACnD,MAAM,CAAC,MAAM,eAAe,GAAG,CAAC;KAC7B,MAAM,CAAC;IACN,KAAK,EAAE,CAAC,CAAC,MAAM,EAAE;IACjB,IAAI,EAAE,CAAC,CAAC,MAAM,EAAE;IAChB,OAAO,EAAE,CAAC,CAAC,MAAM,CAAC,IAAI,EAAE,CAAC,QAAQ,EAAE;IACnC,OAAO,EAAE,CAAC,CAAC,MAAM,CAAC,IAAI,EAAE,CAAC,QAAQ,EAAE;IACnC,OAAO,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE;IAC9B,cAAc,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE;IACrC,iBAAiB,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE;IACxC,WAAW,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE;CACnC,CAAC;KACD,SAAS,CAAC,CAAC,KAAK,EAAE,EAAE;IACnB,MAAM,IAAI,GAAG,KAAK,CAAC,OAAO,IAAI,KAAK,CAAC,OAAO,CAAC;IAC5C,IAAI,CAAC,IAAI,EAAE,CAAC;QACV,MAAM,IAAI,KAAK,CAAC,6BAA6B,CAAC,CAAC;IACjD,CAAC;IACD,OAAO;QACL,KAAK,EAAE,KAAK,CAAC,KAAK;QAClB,IAAI,EAAE,KAAK,CAAC,IAAI;QAChB,IAAI;QACJ,OAAO,EAAE,KAAK,CAAC,OAAO;QACtB,cAAc,EAAE,KAAK,CAAC,cAAc;QACpC,WAAW,EAAE,KAAK,CAAC,WAAW;QAC9B,iBAAiB,EAAE,KAAK,CAAC,iBAAiB,CAAC;KAC5C,CAAC;AACJ,CAAC,CAAC,CAAC"}

package/dist/util.d.ts

@@ -1,3 +1,4 @@
 export declare function asyncMapFilterUndefined<T, U>(input: T[], fn: (x: T) => Promise<U | undefined>): Promise<U[]>;
 export declare function asyncMap<T, U>(input: T[], fn: (x: T) => Promise<U>): Promise<U[]>;
 export declare function filterUndefined<T>(input: (T | undefined)[]): T[];
+//# sourceMappingURL=util.d.ts.map

package/dist/util.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"util.d.ts","sourceRoot":"","sources":["../src/util.ts"],"names":[],"mappings":"AAGA,wBAAsB,uBAAuB,CAAC,CAAC,EAAE,CAAC,EAAE,KAAK,EAAE,CAAC,EAAE,EAAE,EAAE,EAAE,CAAC,CAAC,EAAE,CAAC,KAAK,OAAO,CAAC,CAAC,GAAG,SAAS,CAAC,GAAG,OAAO,CAAC,CAAC,EAAE,CAAC,CAGlH;AAED,wBAAsB,QAAQ,CAAC,CAAC,EAAE,CAAC,EAAE,KAAK,EAAE,CAAC,EAAE,EAAE,EAAE,EAAE,CAAC,CAAC,EAAE,CAAC,KAAK,OAAO,CAAC,CAAC,CAAC,GAAG,OAAO,CAAC,CAAC,EAAE,CAAC,CAOvF;AAED,wBAAgB,eAAe,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC,GAAG,SAAS,CAAC,EAAE,GAAG,CAAC,EAAE,CAKhE"}

package/dist/util.js

@@ -11,3 +11,4 @@ export async function asyncMap(input, fn) {
 export function filterUndefined(input) {
     return R.pipe(input, R.filter((result) => result !== undefined));
 }
+//# sourceMappingURL=util.js.map

package/dist/util.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"util.js","sourceRoot":"","sources":["../src/util.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,CAAC,MAAM,QAAQ,CAAC;AAE5B,8DAA8D;AAC9D,MAAM,CAAC,KAAK,UAAU,uBAAuB,CAAO,KAAU,EAAE,EAAoC;IAClG,MAAM,OAAO,GAAG,MAAM,QAAQ,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC;IAC1C,OAAO,eAAe,CAAC,OAAO,CAAC,CAAC;AAClC,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,QAAQ,CAAO,KAAU,EAAE,EAAwB;IACvE,MAAM,QAAQ,GAAG,CAAC,CAAC,IAAI,CACrB,KAAK,EACL,CAAC,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,EAAE,CAAC,IAAI,CAAC,CAAC,CAC1B,CAAC;IAEF,OAAO,OAAO,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;AAC/B,CAAC;AAED,MAAM,UAAU,eAAe,CAAI,KAAwB;IACzD,OAAO,CAAC,CAAC,IAAI,CACX,KAAK,EACL,CAAC,CAAC,MAAM,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,MAAM,KAAK,SAAS,CAAC,CACpC,CAAC;AACX,CAAC"}

package/package.json

@@ -1,12 +1,29 @@
 {
   "name": "webring",
+  "description": "Collect the latest RSS items from your favorite feeds.",
+  "author": {
+    "name": "Jerred Shepherd",
+    "email": "npm@sjer.red",
+    "url": "https://sjer.red"
+  },
+  "homepage": "https://github.com/shepherdjerred/webring",
+  "license": "GPL-3.0-only",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/shepherdjerred/webring.git"
+  },
+  "bugs": {
+    "url": "https://github.com/shepherdjerred/webring/issues"
+  },
   "type": "module",
-  "version": "1.0.3",
+  "version": "1.1.1",
   "scripts": {
     "prepare": "husky",
     "lint": "eslint src",
     "build": "tsc",
-    "test": "vitest --disable-console-intercept"
+    "watch": "tsc -w",
+    "test": "vitest --disable-console-intercept",
+    "typedoc": "typedoc src/index.ts"
   },
   "main": "dist/index.js",
   "types": "dist/index.d.ts",
@@ -23,7 +40,6 @@
     "@eslint/js": "^9.4.0",
     "@tsconfig/node20": "^20.1.4",
     "@tsconfig/strictest": "^2.0.5",
-    "@types/eslint__js": "^8.42.3",
     "@types/node": "^20.14.0",
     "@types/sanitize-html": "^2.11.0",
     "@typescript-eslint/eslint-plugin": "^7.11.0",
@@ -32,6 +48,8 @@
     "husky": "^9.0.11",
     "lint-staged": "^15.2.5",
     "prettier": "^3.3.0",
+    "typedoc": "^0.26.3",
+    "typedoc-plugin-zod": "^1.2.0",
     "typescript": "^5.4.5",
     "typescript-eslint": "^7.11.0",
     "vitest": "^1.6.0"
@@ -50,6 +68,7 @@
   },
   "files": [
     "dist",
+    "src",
     "package.json",
     "README.md",
     "LICENSE",

package/src/__snapshots__/index.test.ts.snap

@@ -0,0 +1,56 @@
+// Vitest Snapshot v1, https://vitest.dev/guide/snapshot.html
+
+exports[`it should fetch an RSS feed with caching 1`] = `
+[
+  {
+    "date": 2024-07-01T00:00:00.000Z,
+    "preview": "TIL: Using Twoslash with Shiki and Astro",
+    "source": {
+      "title": "Jerred Shepherd",
+      "url": "https://sjer.red/rss.xml",
+    },
+    "title": "TIL: Using Twoslash with Shiki and  Astro",
+    "url": "https://sjer.red/blog/til/2024-07-01/",
+  },
+]
+`;
+
+exports[`it should fetch an RSS feed without caching 1`] = `
+[
+  {
+    "date": 2024-07-01T00:00:00.000Z,
+    "preview": "TIL: Using Twoslash with Shiki and Astro",
+    "source": {
+      "title": "Jerred Shepherd",
+      "url": "https://sjer.red/rss.xml",
+    },
+    "title": "TIL: Using Twoslash with Shiki and  Astro",
+    "url": "https://sjer.red/blog/til/2024-07-01/",
+  },
+]
+`;
+
+exports[`it should fetch several RSS feeds 1`] = `
+[
+  {
+    "date": 2024-06-16T00:00:00.000Z,
+    "preview": "There've been regular viral stories about ML/AI bias with LLMs and generative AI for the past couple years. One thing I find interesting about discussions of bias is how different the reaction is in the LLM and generative AI case when compared to "classical" bugs in cases where there's a clear bug. ...",
+    "source": {
+      "title": "Dan Luu",
+      "url": "https://danluu.com/atom.xml",
+    },
+    "title": "A discussion of discussions on AI bias",
+    "url": "https://danluu.com/ai-bias/",
+  },
+  {
+    "date": 2024-05-24T00:00:00.000Z,
+    "preview": "I needed a bit of a break from “real work” recently, so I started a new programming project that was low-stakes and purely recreational. On April 21st, I set out to see how much of a Unix-like operating system for x86_64 targets that I could put together in about a month. The result is Bunnix. Not i...",
+    "source": {
+      "title": "Drew DeVault",
+      "url": "https://drewdevault.com/blog/index.xml",
+    },
+    "title": "Writing a Unix clone in about a month",
+    "url": "https://drewdevault.com/2024/05/24/2024-05-24-Bunnix.html",
+  },
+]
+`;

package/src/cache.ts

@@ -0,0 +1,82 @@
+import * as R from "remeda";
+import {
+  type Cache,
+  type Result,
+  type ResultEntry,
+  type CacheEntry,
+  type Source,
+  type CachedConfiguration,
+  CacheSchema,
+  type CacheConfiguration,
+} from "./types.js";
+import { fetch } from "./fetch.js";
+import { asyncMapFilterUndefined } from "./util.js";
+import fs from "fs/promises";
+
+async function loadCache({ cache_file }: CacheConfiguration): Promise<Cache> {
+  try {
+    await fs.access(cache_file);
+    const cacheFile = await fs.readFile(cache_file);
+    return CacheSchema.parse(JSON.parse(cacheFile.toString()));
+  } catch (e) {
+    return {};
+  }
+}
+
+async function saveCache({ cache_file }: CacheConfiguration, cache: Cache) {
+  const dir = cache_file.split("/").slice(0, -1).join("/");
+  if (dir !== "") {
+    await fs.mkdir(cache_file.split("/").slice(0, -1).join("/"), { recursive: true });
+  }
+  await fs.writeFile(cache_file, JSON.stringify(cache));
+}
+
+function toCacheEntry(result: ResultEntry, now: Date): [string, CacheEntry] {
+  return [result.source.url, { timestamp: now, data: result }];
+}
+
+function toCache(results: ResultEntry[], now: Date): Cache {
+  return R.pipe(
+    results,
+    R.map((result) => toCacheEntry(result, now)),
+    R.fromEntries(),
+  );
+}
+
+function updateCache(results: ResultEntry[], config: CachedConfiguration): Promise<void> {
+  const now = new Date();
+  const updatedCache = toCache(results, now);
+  return saveCache(config.cache, updatedCache);
+}
+
+export async function fetchAllCached(config: CachedConfiguration): Promise<Result> {
+  const cache = await loadCache(config.cache);
+
+  const results = await asyncMapFilterUndefined(config.sources, (source) => fetchWithCache(source, cache, config));
+
+  await updateCache(results, config);
+
+  return results;
+}
+
+export async function fetchWithCache(
+  source: Source,
+  cache: Cache,
+  config: CachedConfiguration,
+): Promise<ResultEntry | undefined> {
+  const cacheEntry = cache[source.url];
+  if (cacheEntry) {
+    const now = new Date();
+    if (now.getTime() - cacheEntry.timestamp.getTime() < config.cache.cache_duration_minutes * 60 * 1000) {
+      console.log(`Cache entry found for ${source.url}.`);
+      return Promise.resolve(cacheEntry.data);
+    } else {
+      console.log(`Cache entry for ${source.url} is too old.`);
+    }
+  } else {
+    console.log(`No cache entry for ${source.url}.`);
+  }
+
+  console.log(`Fetching ${source.url}`);
+  return fetch(source, config.truncate);
+}

package/src/fetch.ts

@@ -0,0 +1,53 @@
+import Parser from "rss-parser";
+import sanitizeHtml from "sanitize-html";
+import truncate from "truncate-html";
+import { type Source, type ResultEntry, FeedEntrySchema, type Configuration } from "./types.js";
+import * as R from "remeda";
+import { asyncMapFilterUndefined } from "./util.js";
+
+// for some reason, TypeScript does not infer the type of the default export correctly
+const truncateFn: typeof truncate.default = truncate as unknown as typeof truncate.default;
+
+export async function fetchAll(config: Configuration) {
+  return await asyncMapFilterUndefined(config.sources, (source) => fetch(source, config.truncate));
+}
+
+export async function fetch(source: Source, length: number): Promise<ResultEntry | undefined> {
+  const parser = new Parser();
+  try {
+    const feed = await parser.parseURL(source.url);
+
+    const firstItem = R.pipe(
+      feed.items,
+      R.map((item) => FeedEntrySchema.parse(item)),
+      R.sortBy((item) => new Date(item.date).getTime()),
+      R.reverse(),
+      R.first(),
+    );
+
+    if (!firstItem) {
+      throw new Error("no items found in feed");
+    }
+
+    const preview =
+      firstItem.contentSnippet ?? firstItem.content ?? firstItem.description ?? firstItem["content:encoded"];
+
+    return {
+      title: firstItem.title,
+      url: firstItem.link,
+      date: new Date(firstItem.date),
+      source,
+      preview: preview
+        ? truncateFn(
+            sanitizeHtml(preview, {
+              parseStyleAttributes: false,
+            }),
+            length,
+          )
+        : undefined,
+    };
+  } catch (e) {
+    console.error(`Error fetching ${source.url}: ${e as string}`);
+    return undefined;
+  }
+}

package/src/index.test.ts

@@ -0,0 +1,70 @@
+import { expect, test } from "vitest";
+import type { Configuration } from "./types.js";
+import { run } from "./index.js";
+import { tmpdir } from "os";
+import { mkdtemp } from "fs/promises";
+import { join } from "path";
+
+// TODO: intercept network requests
+test("it should fetch an RSS feed without caching", { timeout: 30000 }, async () => {
+  const config: Configuration = {
+    sources: [
+      {
+        title: "Jerred Shepherd",
+        url: "https://sjer.red/rss.xml",
+      },
+    ],
+    number: 1,
+    truncate: 300,
+  };
+
+  const result = await run(config);
+  expect(result).toMatchSnapshot();
+});
+
+test("it should fetch several RSS feeds", { timeout: 30000 }, async () => {
+  const config: Configuration = {
+    sources: [
+      {
+        url: "https://drewdevault.com/blog/index.xml",
+        title: "Drew DeVault",
+      },
+      {
+        url: "https://danluu.com/atom.xml",
+        title: "Dan Luu",
+      },
+    ],
+    number: 3,
+    truncate: 300,
+  };
+
+  const result = await run(config);
+  expect(result).toMatchSnapshot();
+});
+
+test("it should fetch an RSS feed with caching", { timeout: 30000 }, async () => {
+  const config: Configuration = {
+    sources: [
+      {
+        title: "Jerred Shepherd",
+        url: "https://sjer.red/rss.xml",
+      },
+    ],
+    number: 1,
+    truncate: 300,
+    cache: {
+      cache_file: `${await createTempDir()}/cache.json`,
+      cache_duration_minutes: 1,
+    },
+  };
+
+  const result = await run(config);
+  expect(result).toMatchSnapshot();
+});
+
+// https://sdorra.dev/posts/2024-02-12-vitest-tmpdir
+async function createTempDir() {
+  const ostmpdir = tmpdir();
+  const dir = join(ostmpdir, "unit-test-");
+  return await mkdtemp(dir);
+}

package/src/index.ts

@@ -0,0 +1,28 @@
+import * as R from "remeda";
+import { fetchAllCached as fetchAllCached } from "./cache.js";
+import { fetchAll as fetchAllUncached } from "./fetch.js";
+import { type Configuration, type Result, CachedConfigurationSchema } from "./types.js";
+
+export async function run(config: Configuration): Promise<Result> {
+  const { success, data } = CachedConfigurationSchema.safeParse(config);
+
+  let result: Result;
+  if (success) {
+    console.log(`Using cache at ${data.cache.cache_file}.`);
+    result = await fetchAllCached(data);
+  } else {
+    console.log("Cache disabled.");
+    result = await fetchAllUncached(config);
+  }
+
+  const topResults = R.pipe(
+    result,
+    R.sortBy((result) => result.date.getTime()),
+    R.reverse(),
+    R.take(config.number),
+  );
+
+  return topResults;
+}
+
+export * from "./types.js";

package/src/types.ts

@@ -0,0 +1,99 @@
+import { z } from "zod";
+
+export type Source = z.infer<typeof SourceSchema>;
+/** An RSS source */
+const SourceSchema = z.object({
+  /** The URL of an RSS feed */
+  url: z.string(),
+  /** A title to describe the feed */
+  title: z.string().describe("A title for the feed"),
+});
+
+export type CacheConfiguration = z.infer<typeof CacheConfigurationSchema>;
+/** Configuration for the cache */
+const CacheConfigurationSchema = z.object({
+  /** How long to cache a result for */
+  cache_duration_minutes: z.number().default(60),
+  /** The location of a file to use as a cache */
+  cache_file: z.string().default("cache.json"),
+});
+
+export type Configuration = z.infer<typeof ConfigurationSchema>;
+/** A configuration object with caching possibly configured */
+const ConfigurationSchema = z.object({
+  /** A list of sources to fetch */
+  sources: SourceSchema.array(),
+  /** Return the n latest updates from the source list. */
+  number: z.number().default(3),
+  /** How many words the preview field should be truncated to in characters after HTML has been sanitized and parsed. */
+  truncate: z.number().default(300),
+  /** Configuration for the cache */
+  cache: CacheConfigurationSchema.optional(),
+});
+
+export type CachedConfiguration = z.infer<typeof CachedConfigurationSchema>;
+/** A configuration object with caching definitely configured */
+export const CachedConfigurationSchema = ConfigurationSchema.extend({
+  /** Configuration for the cache */
+  cache: CacheConfigurationSchema,
+});
+
+export type ResultEntry = z.infer<typeof ResultEntrySchema>;
+/** A single entry from an RSS feed */
+const ResultEntrySchema = z.object({
+  /** The title of the entry */
+  title: z.string(),
+  /** A direct link to the entry */
+  url: z.string(),
+  /** The date of the entry */
+  date: z.coerce.date(),
+  /** The source the entry is from */
+  source: SourceSchema,
+  /** A preview of the entry. This may contain sanitized HTML. */
+  preview: z.string().optional(),
+});
+
+export type Result = z.infer<typeof ResultSchema>;
+/** A list of results */
+const ResultSchema = z.array(ResultEntrySchema);
+
+export type CacheEntry = z.infer<typeof CacheEntrySchema>;
+/** A single cache entry */
+export const CacheEntrySchema = z.object({
+  /** The time a source was last checked */
+  timestamp: z.coerce.date(),
+  /** The data from the source */
+  data: ResultEntrySchema,
+});
+
+export type Cache = z.infer<typeof CacheSchema>;
+/** A mapping of source URLs to cache entries */
+export const CacheSchema = z.record(CacheEntrySchema);
+
+/** The expected format fetched RSS feed entries */
+export const FeedEntrySchema = z
+  .object({
+    title: z.string(),
+    link: z.string(),
+    isoDate: z.coerce.date().optional(),
+    pubDate: z.coerce.date().optional(),
+    content: z.string().optional(),
+    contentSnippet: z.string().optional(),
+    "content:encoded": z.string().optional(),
+    description: z.string().optional(),
+  })
+  .transform((entry) => {
+    const date = entry.isoDate ?? entry.pubDate;
+    if (!date) {
+      throw new Error("no date found in feed entry");
+    }
+    return {
+      title: entry.title,
+      link: entry.link,
+      date,
+      content: entry.content,
+      contentSnippet: entry.contentSnippet,
+      description: entry.description,
+      "content:encoded": entry["content:encoded"],
+    };
+  });

package/src/util.ts

@@ -0,0 +1,23 @@
+import * as R from "remeda";
+
+// run an async map operation, filtering out undefined results
+export async function asyncMapFilterUndefined<T, U>(input: T[], fn: (x: T) => Promise<U | undefined>): Promise<U[]> {
+  const results = await asyncMap(input, fn);
+  return filterUndefined(results);
+}
+
+export async function asyncMap<T, U>(input: T[], fn: (x: T) => Promise<U>): Promise<U[]> {
+  const promises = R.pipe(
+    input,
+    R.map((item) => fn(item)),
+  );
+
+  return Promise.all(promises);
+}
+
+export function filterUndefined<T>(input: (T | undefined)[]): T[] {
+  return R.pipe(
+    input,
+    R.filter((result) => result !== undefined),
+  ) as T[];
+}