@evcraddock/slug-cli 0.9.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +84 -3
- package/dist/banner-import.d.ts +12 -0
- package/dist/banner-import.js +62 -0
- package/dist/commands.js +226 -80
- package/dist/link-metadata.d.ts +18 -0
- package/dist/link-metadata.js +234 -0
- package/package.json +2 -1
package/README.md
CHANGED
|
@@ -55,7 +55,7 @@ Use `slug --site <name> post list`, `slug --site <name> post show <slug>`, `slug
|
|
|
55
55
|
|
|
56
56
|
## Markdown imports
|
|
57
57
|
|
|
58
|
-
Import one Markdown file with `slug --site <name> post import <file.md>`. Imports require YAML frontmatter delimited by `---` with `slug` and `type`; articles also require `title`. Supported frontmatter fields are `title`, `slug`, `type`, `excerpt`, `date`, `publishedAt`, `tags`, `url`, and `
|
|
58
|
+
Import one Markdown file with `slug --site <name> post import <file.md>`. Imports require YAML frontmatter delimited by `---` with `slug` and `type`; articles also require `title`. Supported frontmatter fields are `title`, `slug`, `type`, `excerpt`, `date`, `publishedAt`, `tags`, `url`, `banner`, `banner_url`, `source`, and `authors`. `source` and `authors` are supported only for links. `date` is used when `publishedAt` is absent. Dates must be ISO dates or datetimes. Tags support either a YAML list or a flow-style list such as `[writing, notes]`.
|
|
59
59
|
|
|
60
60
|
```markdown
|
|
61
61
|
---
|
|
@@ -67,13 +67,94 @@ date: 2020-01-02
|
|
|
67
67
|
tags:
|
|
68
68
|
- writing
|
|
69
69
|
- updates
|
|
70
|
-
banner: images/hero.png
|
|
70
|
+
banner: ./images/hero.png
|
|
71
71
|
---
|
|
72
72
|
|
|
73
73
|
The imported body.
|
|
74
74
|
```
|
|
75
75
|
|
|
76
|
-
|
|
76
|
+
An article `banner` starting with `./` is a local file path resolved from the Markdown file. It is uploaded through the Slugkit media API under `posts/<slug>/banner.<extension>` and saved as the post's explicit `bannerUrl` without modifying Markdown content. Re-importing that local banner uploads the current file again to refresh the deterministic media object key. A banner without `./` is an existing same-site media key, not a local file.
|
|
77
|
+
|
|
78
|
+
**Migration:** change existing local banner references such as `banner: images/hero.png` to `banner: ./images/hero.png`. There is no local-file fallback for missing media keys. Full URLs, absolute filesystem paths, `..` path segments (including `../`), empty values, backslashes, and URL-encoded or reserved path characters (`:`, `%`, `?`, `#`) are rejected.
|
|
79
|
+
|
|
80
|
+
The command looks up the post by slug: it creates a missing post and updates an existing one, applying the supplied metadata, body, and tags. A new article without `date` or `publishedAt` is unpublished. Re-importing an existing article preserves its ID, slug, and publication timestamp when dates are omitted. Omitting `banner` leaves an existing banner unchanged; for a new article it means no banner. Link imports continue to map `banner_url` directly to `bannerUrl`.
|
|
81
|
+
|
|
82
|
+
### Upload once and reuse an article banner
|
|
83
|
+
|
|
84
|
+
Upload a shared image with a stable key:
|
|
85
|
+
|
|
86
|
+
```bash
|
|
87
|
+
slug --site my-site media upload ./technology.png \
|
|
88
|
+
--key banners/technology.png \
|
|
89
|
+
--alt "Technology banner"
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
Use that key in any number of articles:
|
|
93
|
+
|
|
94
|
+
```markdown
|
|
95
|
+
---
|
|
96
|
+
title: Technology notes
|
|
97
|
+
slug: technology-notes
|
|
98
|
+
type: article
|
|
99
|
+
banner: banners/technology.png
|
|
100
|
+
---
|
|
101
|
+
|
|
102
|
+
The article body.
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
```bash
|
|
106
|
+
slug --site my-site post import ./technology-notes.md
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
For an API base of `https://your-site.example/api/v1`, the importer verifies `https://your-site.example/media/banners/technology.png`. It uses the API's origin and the site's `/media/` route, not a separately configured external media hostname. It sends an unauthenticated `HEAD` request with a five-second timeout and requires HTTP 200 with an `image/*` content type. No image body is downloaded, no media database lookup is required, and no new upload is made. Redirects are rejected rather than followed.
|
|
110
|
+
|
|
111
|
+
A missing shared image or local file stops import before post or media mutations. Non-image responses, unsupported HEAD requests, network failures, and server errors also stop import with contextual errors; they are not silently treated as missing images or replaced with defaults.
|
|
112
|
+
|
|
113
|
+
Replacing an image at a shared key changes it for every article using that key, subject to browser or proxy caching. Deleting it affects all those articles too: when viewed in a JavaScript-enabled browser, unavailable article banners are removed without a broken-image placeholder or default banner. Articles with no banner have none. URL verification at import time is only a point-in-time availability check; it does not prevent later deletion.
|
|
114
|
+
|
|
115
|
+
Link imports may include one source and one or more authors:
|
|
116
|
+
|
|
117
|
+
```markdown
|
|
118
|
+
---
|
|
119
|
+
type: link
|
|
120
|
+
slug: how-to-stop-being-boring
|
|
121
|
+
title: How to stop being boring
|
|
122
|
+
url: https://www.joanwestenberg.com/how-to-stop-being-boring/
|
|
123
|
+
banner_url: https://example.com/banner.jpg
|
|
124
|
+
excerpt: A short description.
|
|
125
|
+
tags: [adulting]
|
|
126
|
+
publishedAt: 2026-02-07
|
|
127
|
+
source:
|
|
128
|
+
name: Joan Westenberg
|
|
129
|
+
url: https://www.joanwestenberg.com
|
|
130
|
+
authors:
|
|
131
|
+
- name: Joan Westenberg
|
|
132
|
+
url: https://www.joanwestenberg.com/about
|
|
133
|
+
---
|
|
134
|
+
|
|
135
|
+
Optional commentary about the link.
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
Link files may omit either or both relationship fields:
|
|
139
|
+
|
|
140
|
+
```markdown
|
|
141
|
+
---
|
|
142
|
+
type: link
|
|
143
|
+
slug: discovered-link
|
|
144
|
+
title: Discovered link
|
|
145
|
+
url: https://publication.example/posts/discovered-link
|
|
146
|
+
---
|
|
147
|
+
|
|
148
|
+
Optional commentary.
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
Explicit `source` and `authors` values override discovery independently. For each missing field, the importer fetches the link page without sending Slugkit API credentials. Source discovery prefers JSON-LD publisher data, then standard site-name metadata. Author discovery uses JSON-LD author data and standard author metadata. If source metadata is unavailable, the normalized destination hostname and origin become the source. Missing or malformed author metadata does not block the import; the link is imported without author contacts.
|
|
152
|
+
|
|
153
|
+
Remote discovery accepts HTML responses, uses a five-second timeout, and reads at most 1 MiB. Network errors, non-HTML responses, malformed metadata, timeouts, and oversized responses use the hostname source fallback. Duplicate discovered authors are collapsed by canonical URL or normalized name, preferring a direct author URL over fragment identifiers.
|
|
154
|
+
|
|
155
|
+
When a relationship URL is available, the importer matches existing sources and contacts by a canonical URL with fragments and trailing slashes removed. Without a URL, it matches a unique name after normalizing case and whitespace. Ambiguous matches fail instead of silently selecting a record. Missing authors are created as contacts, then associated with the resolved or newly created source and credited on the post. Existing source contacts and post credits are preserved. Re-importing the same file reuses records and relationships without duplicates.
|
|
156
|
+
|
|
157
|
+
All explicit source and author metadata is validated before API mutations. For relationship imports, missing contacts are created first, the source is created or updated second, and the post is created or updated last. The API does not provide a bulk transaction, so a later failure stops the import but records created by earlier successful requests remain; relationship mutation errors identify the failed frontmatter path.
|
|
77
158
|
|
|
78
159
|
## Source imports
|
|
79
160
|
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
export interface ArticleBanner {
|
|
2
|
+
bannerPath?: string;
|
|
3
|
+
bannerKey?: string;
|
|
4
|
+
}
|
|
5
|
+
interface VerifyBannerKeyOptions {
|
|
6
|
+
apiBaseUrl: string;
|
|
7
|
+
key: string;
|
|
8
|
+
fetchImpl?: typeof fetch;
|
|
9
|
+
}
|
|
10
|
+
export declare function readArticleBanner(frontmatter: Record<string, unknown>, filePath: string): ArticleBanner;
|
|
11
|
+
export declare function verifyBannerKey(options: VerifyBannerKeyOptions): Promise<string>;
|
|
12
|
+
export {};
|
|
@@ -0,0 +1,62 @@
|
|
|
1
|
+
import { dirname, resolve } from "node:path";
|
|
2
|
+
import { CliError, createInvalidUsageError, ExitCode } from "./errors.js";
|
|
3
|
+
export function readArticleBanner(frontmatter, filePath) {
|
|
4
|
+
if (!("banner" in frontmatter))
|
|
5
|
+
return {};
|
|
6
|
+
const value = frontmatter.banner;
|
|
7
|
+
if (typeof value !== "string" || value.trim() === "") {
|
|
8
|
+
throw createInvalidUsageError("frontmatter.banner must be a non-empty string");
|
|
9
|
+
}
|
|
10
|
+
const local = value.startsWith("./");
|
|
11
|
+
assertBannerKey(local ? value.slice(2) : value);
|
|
12
|
+
return local ? { bannerPath: resolve(dirname(filePath), value) } : { bannerKey: value };
|
|
13
|
+
}
|
|
14
|
+
function assertBannerKey(key) {
|
|
15
|
+
const invalidSegment = key
|
|
16
|
+
.split("/")
|
|
17
|
+
.some((part) => part === "" || part === "." || part === "..");
|
|
18
|
+
const hasControlCharacter = [...key].some((char) => char.charCodeAt(0) < 32 || char.charCodeAt(0) === 127);
|
|
19
|
+
if (invalidSegment || hasControlCharacter || key !== key.trim() || /[:\\%?#]/u.test(key)) {
|
|
20
|
+
throw createInvalidUsageError("frontmatter.banner must be a media key or ./local-file path; URLs, absolute paths, traversal, and URL-encoded or reserved path characters are not allowed");
|
|
21
|
+
}
|
|
22
|
+
}
|
|
23
|
+
export async function verifyBannerKey(options) {
|
|
24
|
+
assertBannerKey(options.key);
|
|
25
|
+
const key = options.key.split("/").map(encodeURIComponent).join("/");
|
|
26
|
+
const url = new URL(`/media/${key}`, options.apiBaseUrl).toString();
|
|
27
|
+
let response;
|
|
28
|
+
try {
|
|
29
|
+
response = await (options.fetchImpl ?? fetch)(url, {
|
|
30
|
+
method: "HEAD",
|
|
31
|
+
headers: { accept: "image/*" },
|
|
32
|
+
credentials: "omit",
|
|
33
|
+
redirect: "manual",
|
|
34
|
+
signal: AbortSignal.timeout(5_000),
|
|
35
|
+
});
|
|
36
|
+
}
|
|
37
|
+
catch {
|
|
38
|
+
throw new CliError(`Cannot verify banner: network request failed or timed out for ${url}`, ExitCode.NetworkError);
|
|
39
|
+
}
|
|
40
|
+
assertBannerResponse(response, url);
|
|
41
|
+
return url;
|
|
42
|
+
}
|
|
43
|
+
function assertBannerResponse(response, url) {
|
|
44
|
+
let reason;
|
|
45
|
+
if (response.status === 404 || response.status === 410) {
|
|
46
|
+
reason = "Banner image not found";
|
|
47
|
+
}
|
|
48
|
+
else if (response.status >= 300 && response.status < 400) {
|
|
49
|
+
reason = "Cannot verify banner: redirects are not allowed";
|
|
50
|
+
}
|
|
51
|
+
else if (response.status === 405 || response.status === 501) {
|
|
52
|
+
reason = "Cannot verify banner: HEAD requests are not supported";
|
|
53
|
+
}
|
|
54
|
+
else if (response.status !== 200) {
|
|
55
|
+
reason = `Cannot verify banner: HTTP ${response.status}`;
|
|
56
|
+
}
|
|
57
|
+
else if (!/^image\/[a-z0-9.+-]+(?:\s*;|$)/iu.test(response.headers.get("content-type") ?? "")) {
|
|
58
|
+
reason = "Cannot verify banner: response is not an image";
|
|
59
|
+
}
|
|
60
|
+
if (reason !== undefined)
|
|
61
|
+
throw new CliError(`${reason}: ${url}`, ExitCode.ApiError, response.status);
|
|
62
|
+
}
|
package/dist/commands.js
CHANGED
|
@@ -5,8 +5,10 @@ import AdmZip from "adm-zip";
|
|
|
5
5
|
import { parse as parseYaml } from "yaml";
|
|
6
6
|
import { SLUGKIT_API_MAJOR_VERSION, SLUGKIT_API_NAME, compareSlugkitApiVersions, normalizeApiBaseUrl, readSlugkitApiMajorVersion, } from "@evcraddock/slug-core";
|
|
7
7
|
import { isValidSiteName, readConfig, removeConfigSite, setConfigSiteApiBaseUrl, setConfigSiteApiKey, toDisplayConfig, writeConfig, } from "./config.js";
|
|
8
|
+
import { readArticleBanner, verifyBannerKey } from "./banner-import.js";
|
|
8
9
|
import { CliError, createInvalidUsageError, ExitCode } from "./errors.js";
|
|
9
10
|
import { SlugHttpClient } from "./http.js";
|
|
11
|
+
import { discoverLinkRelationships } from "./link-metadata.js";
|
|
10
12
|
import { writeJson } from "./output.js";
|
|
11
13
|
const HELP_TEXT = `slug - manage Slugkit sites
|
|
12
14
|
|
|
@@ -21,8 +23,8 @@ Usage:
|
|
|
21
23
|
slug [--config <file>] --site <name> post list [--type article|link|note] [--status draft|published|all] [--tag <slug>] [--json]
|
|
22
24
|
slug [--config <file>] post show <slug> [--json]
|
|
23
25
|
slug [--config <file>] post import <file.md> [--json]
|
|
24
|
-
slug [--config <file>] post create --type article|link|note --slug <slug> --content <text> [--title <text>] [--url <url>] [--excerpt <text>] [--published-at <datetime>] [--tag <slug>]... [--source-id <id>] [--credit-contact-id <id>]... [--json]
|
|
25
|
-
slug [--config <file>] post edit <slug> [--slug <new-slug>] [--title <text>] [--content <text>] [--url <url>] [--excerpt <text>] [--published-at <datetime>] [--tag <slug>]... [--source-id <id>] [--credit-contact-id <id>]... [--json]
|
|
26
|
+
slug [--config <file>] post create --type article|link|note --slug <slug> --content <text> [--title <text>] [--url <url>] [--excerpt <text>] [--banner-url <url>] [--published-at <datetime>] [--tag <slug>]... [--source-id <id>] [--credit-contact-id <id>]... [--json]
|
|
27
|
+
slug [--config <file>] post edit <slug> [--slug <new-slug>] [--title <text>] [--content <text>] [--url <url>] [--excerpt <text>] [--banner-url <url>] [--published-at <datetime>] [--tag <slug>]... [--source-id <id>] [--credit-contact-id <id>]... [--json]
|
|
26
28
|
slug [--config <file>] post delete <slug> [--json]
|
|
27
29
|
slug [--config <file>] post publish <slug> [--published-at <datetime>] [--json]
|
|
28
30
|
slug [--config <file>] post unpublish <slug> [--json]
|
|
@@ -142,7 +144,7 @@ const GENERATED_SITE_SLUGKIT_DEPENDENCIES = {
|
|
|
142
144
|
"@evcraddock/slug-auth": "0.1.0",
|
|
143
145
|
"@evcraddock/slug-core": "0.1.1",
|
|
144
146
|
"@evcraddock/slug-federation": "0.1.0",
|
|
145
|
-
"@evcraddock/slug-media": "0.1.
|
|
147
|
+
"@evcraddock/slug-media": "0.1.1",
|
|
146
148
|
};
|
|
147
149
|
const INIT_HELP_TEXT = `slug init - create a standalone Slugkit-compatible website
|
|
148
150
|
|
|
@@ -924,24 +926,52 @@ async function runPostsCommand(context, args) {
|
|
|
924
926
|
}
|
|
925
927
|
async function runPostsImportCommand(context, args) {
|
|
926
928
|
const { filePath, json } = readPostImportArgs(args);
|
|
927
|
-
|
|
929
|
+
let post = await readMarkdownPostImport(filePath);
|
|
930
|
+
if (post.type === "link" && post.url !== undefined) {
|
|
931
|
+
const discovered = await discoverLinkRelationships({
|
|
932
|
+
url: post.url,
|
|
933
|
+
source: post.source === undefined,
|
|
934
|
+
authors: post.authors === undefined,
|
|
935
|
+
fetchImpl: context.fetchImpl,
|
|
936
|
+
});
|
|
937
|
+
post = {
|
|
938
|
+
...post,
|
|
939
|
+
...(post.source !== undefined || discovered.source === undefined
|
|
940
|
+
? {}
|
|
941
|
+
: { source: discovered.source }),
|
|
942
|
+
...(post.authors !== undefined || discovered.authors === undefined
|
|
943
|
+
? {}
|
|
944
|
+
: { authors: discovered.authors }),
|
|
945
|
+
};
|
|
946
|
+
}
|
|
928
947
|
const api = await createConfiguredApiContext(context);
|
|
929
|
-
const
|
|
948
|
+
const uploadedBannerUrl = post.bannerPath === undefined
|
|
930
949
|
? undefined
|
|
931
950
|
: await uploadPostImportBanner(api, post, post.bannerPath);
|
|
932
|
-
const
|
|
951
|
+
const bannerUrl = post.bannerKey === undefined
|
|
952
|
+
? (uploadedBannerUrl ?? post.bannerUrl)
|
|
953
|
+
: await verifyBannerKey({
|
|
954
|
+
apiBaseUrl: api.apiBaseUrl,
|
|
955
|
+
key: post.bannerKey,
|
|
956
|
+
fetchImpl: context.fetchImpl,
|
|
957
|
+
});
|
|
958
|
+
writeMutationTarget(context.writer, api.apiBaseUrl, json);
|
|
959
|
+
const existing = await findImportedPost(api.client, post.slug);
|
|
960
|
+
const relationships = await resolveMarkdownImportRelationships(api.client, post);
|
|
961
|
+
const creditContactIds = mergeIds(existing?.creditContactIds ?? [], relationships.authorContactIds);
|
|
933
962
|
const body = {
|
|
934
963
|
type: post.type,
|
|
935
964
|
slug: post.slug,
|
|
936
|
-
content,
|
|
965
|
+
content: post.content,
|
|
937
966
|
...(post.title === undefined ? {} : { title: post.title }),
|
|
938
967
|
...(post.excerpt === undefined ? {} : { excerpt: post.excerpt }),
|
|
939
968
|
...(post.url === undefined ? {} : { url: post.url }),
|
|
969
|
+
...(bannerUrl === undefined ? {} : { bannerUrl }),
|
|
970
|
+
...(relationships.sourceId === undefined ? {} : { sourceId: relationships.sourceId }),
|
|
971
|
+
...(post.authors === undefined ? {} : { creditContactIds }),
|
|
940
972
|
...(post.tags === undefined ? {} : { tagSlugs: post.tags }),
|
|
941
973
|
...(post.publishedAt === undefined ? {} : { publishedAt: post.publishedAt }),
|
|
942
974
|
};
|
|
943
|
-
writeMutationTarget(context.writer, api.apiBaseUrl, json);
|
|
944
|
-
const existing = await findImportedPost(api.client, post.slug);
|
|
945
975
|
const response = existing === undefined
|
|
946
976
|
? await api.client.requestJson({ method: "POST", path: "/posts", body })
|
|
947
977
|
: await api.client.requestJson({
|
|
@@ -977,21 +1007,32 @@ async function readMarkdownPostImport(filePath) {
|
|
|
977
1007
|
if (match === null) {
|
|
978
1008
|
throw createInvalidUsageError("Markdown import requires YAML frontmatter delimited by ---");
|
|
979
1009
|
}
|
|
980
|
-
const frontmatter =
|
|
981
|
-
const slug =
|
|
982
|
-
const type =
|
|
1010
|
+
const frontmatter = parseMarkdownPostFrontmatter(match[1]);
|
|
1011
|
+
const slug = readMarkdownRequiredString(frontmatter, "slug", "frontmatter");
|
|
1012
|
+
const type = readMarkdownRequiredString(frontmatter, "type", "frontmatter");
|
|
983
1013
|
if (type !== "article" && type !== "link" && type !== "note") {
|
|
984
|
-
throw createInvalidUsageError("
|
|
1014
|
+
throw createInvalidUsageError("frontmatter.type must be article, link, or note");
|
|
985
1015
|
}
|
|
986
|
-
const title =
|
|
1016
|
+
const title = readMarkdownOptionalString(frontmatter, "title", "frontmatter");
|
|
987
1017
|
if (type === "article" && title === undefined) {
|
|
988
|
-
throw createInvalidUsageError("Article imports require frontmatter
|
|
989
|
-
}
|
|
990
|
-
const
|
|
991
|
-
const
|
|
992
|
-
|
|
993
|
-
|
|
994
|
-
|
|
1018
|
+
throw createInvalidUsageError("Article imports require frontmatter.title");
|
|
1019
|
+
}
|
|
1020
|
+
const source = readMarkdownRelationship(frontmatter, "source", "frontmatter");
|
|
1021
|
+
const authors = readMarkdownAuthors(frontmatter);
|
|
1022
|
+
if (type !== "link" && (source !== undefined || authors !== undefined)) {
|
|
1023
|
+
throw createInvalidUsageError("frontmatter.source and frontmatter.authors require type: link");
|
|
1024
|
+
}
|
|
1025
|
+
const excerpt = readMarkdownOptionalString(frontmatter, "excerpt", "frontmatter");
|
|
1026
|
+
const url = readMarkdownOptionalString(frontmatter, "url", "frontmatter");
|
|
1027
|
+
if (type === "link") {
|
|
1028
|
+
if (url === undefined)
|
|
1029
|
+
throw createInvalidUsageError("Link imports require frontmatter.url");
|
|
1030
|
+
assertSourceImportUrl(url, "frontmatter.url", ["http:", "https:"]);
|
|
1031
|
+
}
|
|
1032
|
+
const tags = readMarkdownTags(frontmatter);
|
|
1033
|
+
const publishedAt = readMarkdownPublishedAt(frontmatter);
|
|
1034
|
+
const banner = type === "article" ? readArticleBanner(frontmatter, filePath) : {};
|
|
1035
|
+
const linkBannerUrl = readMarkdownOptionalString(frontmatter, "banner_url", "frontmatter");
|
|
995
1036
|
return {
|
|
996
1037
|
slug,
|
|
997
1038
|
type,
|
|
@@ -1000,75 +1041,179 @@ async function readMarkdownPostImport(filePath) {
|
|
|
1000
1041
|
...(url === undefined ? {} : { url }),
|
|
1001
1042
|
...(tags === undefined ? {} : { tags }),
|
|
1002
1043
|
...(publishedAt === undefined ? {} : { publishedAt }),
|
|
1003
|
-
...
|
|
1044
|
+
...banner,
|
|
1045
|
+
...(type !== "link" || linkBannerUrl === undefined ? {} : { bannerUrl: linkBannerUrl }),
|
|
1046
|
+
...(source === undefined ? {} : { source }),
|
|
1047
|
+
...(authors === undefined ? {} : { authors }),
|
|
1004
1048
|
content: markdown.slice(match[0].length),
|
|
1005
1049
|
};
|
|
1006
1050
|
}
|
|
1007
|
-
function
|
|
1008
|
-
|
|
1009
|
-
|
|
1010
|
-
|
|
1011
|
-
|
|
1012
|
-
|
|
1013
|
-
|
|
1014
|
-
|
|
1015
|
-
|
|
1016
|
-
|
|
1017
|
-
|
|
1018
|
-
|
|
1019
|
-
|
|
1020
|
-
|
|
1021
|
-
|
|
1022
|
-
|
|
1023
|
-
|
|
1051
|
+
function parseMarkdownPostFrontmatter(value) {
|
|
1052
|
+
let parsed;
|
|
1053
|
+
try {
|
|
1054
|
+
parsed = parseYaml(value);
|
|
1055
|
+
}
|
|
1056
|
+
catch (error) {
|
|
1057
|
+
const message = error instanceof Error ? error.message : "Unable to parse YAML";
|
|
1058
|
+
throw createInvalidUsageError(`Invalid Markdown frontmatter YAML: ${message}`);
|
|
1059
|
+
}
|
|
1060
|
+
if (!isRecord(parsed))
|
|
1061
|
+
throw createInvalidUsageError("frontmatter must be an object");
|
|
1062
|
+
return parsed;
|
|
1063
|
+
}
|
|
1064
|
+
function readMarkdownRequiredString(record, key, path) {
|
|
1065
|
+
const value = readMarkdownOptionalString(record, key, path);
|
|
1066
|
+
if (value === undefined) {
|
|
1067
|
+
throw createInvalidUsageError(`${path}.${key} must be a non-empty string`);
|
|
1068
|
+
}
|
|
1069
|
+
return value;
|
|
1070
|
+
}
|
|
1071
|
+
function readMarkdownOptionalString(record, key, path) {
|
|
1072
|
+
if (!(key in record))
|
|
1073
|
+
return undefined;
|
|
1074
|
+
const value = record[key];
|
|
1075
|
+
if (typeof value !== "string") {
|
|
1076
|
+
throw createInvalidUsageError(`${path}.${key} must be a string`);
|
|
1077
|
+
}
|
|
1078
|
+
return value.trim() === "" ? undefined : value;
|
|
1079
|
+
}
|
|
1080
|
+
function readMarkdownTags(record) {
|
|
1081
|
+
if (!("tags" in record))
|
|
1082
|
+
return undefined;
|
|
1083
|
+
if (!Array.isArray(record.tags)) {
|
|
1084
|
+
throw createInvalidUsageError("frontmatter.tags must be an array");
|
|
1085
|
+
}
|
|
1086
|
+
return record.tags.map((tag, index) => {
|
|
1087
|
+
if (typeof tag !== "string" || tag.trim() === "") {
|
|
1088
|
+
throw createInvalidUsageError(`frontmatter.tags[${index}] must be a non-empty string`);
|
|
1024
1089
|
}
|
|
1025
|
-
|
|
1026
|
-
|
|
1090
|
+
return tag;
|
|
1091
|
+
});
|
|
1092
|
+
}
|
|
1093
|
+
function readMarkdownRelationship(record, key, path) {
|
|
1094
|
+
if (!(key in record))
|
|
1095
|
+
return undefined;
|
|
1096
|
+
return parseMarkdownRelationship(record[key], `${path}.${key}`);
|
|
1097
|
+
}
|
|
1098
|
+
function parseMarkdownRelationship(value, path) {
|
|
1099
|
+
if (!isRecord(value))
|
|
1100
|
+
throw createInvalidUsageError(`${path} must be an object`);
|
|
1101
|
+
for (const field of Object.keys(value)) {
|
|
1102
|
+
if (field !== "name" && field !== "url") {
|
|
1103
|
+
throw createInvalidUsageError(`${path}.${field} is not supported`);
|
|
1027
1104
|
}
|
|
1028
1105
|
}
|
|
1029
|
-
|
|
1106
|
+
const name = readMarkdownRequiredString(value, "name", path);
|
|
1107
|
+
const url = readMarkdownOptionalString(value, "url", path);
|
|
1108
|
+
if (url !== undefined)
|
|
1109
|
+
assertSourceImportUrl(url, `${path}.url`, ["http:", "https:"]);
|
|
1110
|
+
return { name, ...(url === undefined ? {} : { url }) };
|
|
1030
1111
|
}
|
|
1031
|
-
function
|
|
1032
|
-
if (
|
|
1033
|
-
|
|
1034
|
-
|
|
1035
|
-
|
|
1036
|
-
|
|
1037
|
-
|
|
1038
|
-
|
|
1039
|
-
|
|
1112
|
+
function readMarkdownAuthors(record) {
|
|
1113
|
+
if (!("authors" in record))
|
|
1114
|
+
return undefined;
|
|
1115
|
+
if (!Array.isArray(record.authors) || record.authors.length === 0) {
|
|
1116
|
+
throw createInvalidUsageError("frontmatter.authors must be a non-empty array");
|
|
1117
|
+
}
|
|
1118
|
+
return record.authors.map((author, index) => parseMarkdownRelationship(author, `frontmatter.authors[${index}]`));
|
|
1119
|
+
}
|
|
1120
|
+
function readMarkdownPublishedAt(record) {
|
|
1121
|
+
const publishedAt = readMarkdownOptionalString(record, "publishedAt", "frontmatter") ??
|
|
1122
|
+
readMarkdownOptionalString(record, "date", "frontmatter");
|
|
1123
|
+
return publishedAt === undefined ? undefined : normalizeCliPublishedAt(publishedAt);
|
|
1124
|
+
}
|
|
1125
|
+
async function resolveMarkdownImportRelationships(client, post) {
|
|
1126
|
+
if (post.source === undefined && post.authors === undefined)
|
|
1127
|
+
return { authorContactIds: [] };
|
|
1128
|
+
const sources = post.source === undefined
|
|
1129
|
+
? []
|
|
1130
|
+
: (await client.requestJson({ path: "/sources" })).data;
|
|
1131
|
+
const contacts = post.authors === undefined
|
|
1132
|
+
? []
|
|
1133
|
+
: (await client.requestJson({ path: "/contacts" })).data;
|
|
1134
|
+
if (post.source !== undefined) {
|
|
1135
|
+
findMarkdownRelationshipMatch(sources, post.source, "frontmatter.source", "sources");
|
|
1136
|
+
}
|
|
1137
|
+
post.authors?.forEach((author, index) => findMarkdownRelationshipMatch(contacts, author, `frontmatter.authors[${index}]`, "contacts"));
|
|
1138
|
+
const authorContactIds = await resolveMarkdownAuthors(client, post.authors ?? [], contacts);
|
|
1139
|
+
const sourceId = await resolveMarkdownSource(client, post.source, sources, authorContactIds);
|
|
1140
|
+
return { ...(sourceId === undefined ? {} : { sourceId }), authorContactIds };
|
|
1141
|
+
}
|
|
1142
|
+
async function resolveMarkdownAuthors(client, authors, contacts) {
|
|
1143
|
+
const ids = [];
|
|
1144
|
+
for (const [index, author] of authors.entries()) {
|
|
1145
|
+
let contact = findMarkdownRelationshipMatch(contacts, author, `frontmatter.authors[${index}]`, "contacts");
|
|
1146
|
+
if (contact === undefined) {
|
|
1147
|
+
const response = await requestMarkdownRelationship(client, "POST", "/contacts", { name: author.name, ...(author.url === undefined ? {} : { url: author.url }) }, `frontmatter.authors[${index}]`);
|
|
1148
|
+
contact = response.data;
|
|
1149
|
+
contacts.push(contact);
|
|
1040
1150
|
}
|
|
1151
|
+
ids.push(contact.id);
|
|
1041
1152
|
}
|
|
1042
|
-
|
|
1043
|
-
|
|
1153
|
+
return [...new Set(ids)];
|
|
1154
|
+
}
|
|
1155
|
+
async function resolveMarkdownSource(client, input, sources, authorContactIds) {
|
|
1156
|
+
if (input === undefined)
|
|
1157
|
+
return undefined;
|
|
1158
|
+
const existing = findMarkdownRelationshipMatch(sources, input, "frontmatter.source", "sources");
|
|
1159
|
+
if (existing === undefined)
|
|
1160
|
+
return createMarkdownSource(client, input, authorContactIds);
|
|
1161
|
+
await addMarkdownSourceContacts(client, existing, authorContactIds);
|
|
1162
|
+
return existing.id;
|
|
1163
|
+
}
|
|
1164
|
+
async function createMarkdownSource(client, input, contactIds) {
|
|
1165
|
+
const response = await requestMarkdownRelationship(client, "POST", "/sources", {
|
|
1166
|
+
name: input.name,
|
|
1167
|
+
...(input.url === undefined ? {} : { url: input.url }),
|
|
1168
|
+
...(contactIds.length === 0 ? {} : { contactIds }),
|
|
1169
|
+
}, "frontmatter.source");
|
|
1170
|
+
return response.data.id;
|
|
1171
|
+
}
|
|
1172
|
+
async function addMarkdownSourceContacts(client, source, authorContactIds) {
|
|
1173
|
+
const existingIds = source.contacts?.map((contact) => contact.id) ?? [];
|
|
1174
|
+
const contactIds = mergeIds(existingIds, authorContactIds);
|
|
1175
|
+
if (contactIds.length === existingIds.length)
|
|
1176
|
+
return;
|
|
1177
|
+
await requestMarkdownRelationship(client, "PUT", `/sources/${encodeURIComponent(source.id.toString())}`, { contactIds }, "frontmatter.source.contacts");
|
|
1178
|
+
}
|
|
1179
|
+
function findMarkdownRelationshipMatch(records, input, path, label) {
|
|
1180
|
+
const matches = findMarkdownRelationshipMatches(records, input);
|
|
1181
|
+
if (matches.length > 1) {
|
|
1182
|
+
throw createInvalidUsageError(`${path} matches multiple ${label} by ${input.url === undefined ? "name" : "URL"}`);
|
|
1044
1183
|
}
|
|
1045
|
-
return
|
|
1184
|
+
return matches[0];
|
|
1046
1185
|
}
|
|
1047
|
-
function
|
|
1048
|
-
|
|
1049
|
-
|
|
1050
|
-
|
|
1051
|
-
|
|
1052
|
-
|
|
1053
|
-
|
|
1054
|
-
function readRequiredFrontmatterString(frontmatter, key) {
|
|
1055
|
-
const value = readOptionalFrontmatterString(frontmatter, key);
|
|
1056
|
-
if (value === undefined)
|
|
1057
|
-
throw createInvalidUsageError(`Markdown frontmatter requires ${key}`);
|
|
1058
|
-
return value;
|
|
1186
|
+
function findMarkdownRelationshipMatches(records, input) {
|
|
1187
|
+
if (input.url === undefined) {
|
|
1188
|
+
const name = normalizeImportName(input.name);
|
|
1189
|
+
return records.filter((record) => normalizeImportName(record.name) === name);
|
|
1190
|
+
}
|
|
1191
|
+
const url = canonicalizeImportUrl(input.url);
|
|
1192
|
+
return records.filter((record) => record.url !== undefined && record.url !== null && canonicalizeImportUrl(record.url) === url);
|
|
1059
1193
|
}
|
|
1060
|
-
function
|
|
1061
|
-
const
|
|
1062
|
-
|
|
1194
|
+
function canonicalizeImportUrl(value) {
|
|
1195
|
+
const url = new URL(value);
|
|
1196
|
+
url.hash = "";
|
|
1197
|
+
if (url.pathname !== "/")
|
|
1198
|
+
url.pathname = url.pathname.replace(/\/+$/u, "");
|
|
1199
|
+
return url.toString();
|
|
1063
1200
|
}
|
|
1064
|
-
function
|
|
1065
|
-
|
|
1066
|
-
return Array.isArray(value) ? value : undefined;
|
|
1201
|
+
function normalizeImportName(value) {
|
|
1202
|
+
return value.trim().replace(/\s+/gu, " ").toLocaleLowerCase("en-US");
|
|
1067
1203
|
}
|
|
1068
|
-
function
|
|
1069
|
-
|
|
1070
|
-
|
|
1071
|
-
|
|
1204
|
+
function mergeIds(existing, added) {
|
|
1205
|
+
return [...new Set([...existing, ...added])];
|
|
1206
|
+
}
|
|
1207
|
+
async function requestMarkdownRelationship(client, method, path, body, importPath) {
|
|
1208
|
+
try {
|
|
1209
|
+
return await client.requestJson({ method, path, body });
|
|
1210
|
+
}
|
|
1211
|
+
catch (error) {
|
|
1212
|
+
if (error instanceof CliError) {
|
|
1213
|
+
throw new CliError(`Failed to import ${importPath}: ${error.message}`, error.exitCode, error.status);
|
|
1214
|
+
}
|
|
1215
|
+
throw error;
|
|
1216
|
+
}
|
|
1072
1217
|
}
|
|
1073
1218
|
async function uploadPostImportBanner(api, post, bannerPath) {
|
|
1074
1219
|
const file = await readMediaUploadFile(bannerPath);
|
|
@@ -1082,10 +1227,7 @@ async function uploadPostImportBanner(api, post, bannerPath) {
|
|
|
1082
1227
|
path: "/media",
|
|
1083
1228
|
body,
|
|
1084
1229
|
});
|
|
1085
|
-
return response.data.url;
|
|
1086
|
-
}
|
|
1087
|
-
function createBannerContent(post, bannerUrl) {
|
|
1088
|
-
return `\n\n${post.content}`;
|
|
1230
|
+
return new URL(response.data.url, api.apiBaseUrl).toString();
|
|
1089
1231
|
}
|
|
1090
1232
|
async function findImportedPost(client, slug) {
|
|
1091
1233
|
try {
|
|
@@ -1142,6 +1284,7 @@ async function runPostsCreateCommand(context, args) {
|
|
|
1142
1284
|
"title",
|
|
1143
1285
|
"url",
|
|
1144
1286
|
"excerpt",
|
|
1287
|
+
"banner-url",
|
|
1145
1288
|
"tag",
|
|
1146
1289
|
"source-id",
|
|
1147
1290
|
"credit-contact-id",
|
|
@@ -1169,6 +1312,7 @@ async function runPostsEditCommand(context, args) {
|
|
|
1169
1312
|
"title",
|
|
1170
1313
|
"url",
|
|
1171
1314
|
"excerpt",
|
|
1315
|
+
"banner-url",
|
|
1172
1316
|
"tag",
|
|
1173
1317
|
"source-id",
|
|
1174
1318
|
"credit-contact-id",
|
|
@@ -1255,6 +1399,7 @@ function createPostMutationInput(options, requireCreateFields) {
|
|
|
1255
1399
|
copyStringOption(options, input, "content", "content");
|
|
1256
1400
|
copyStringOption(options, input, "url", "url");
|
|
1257
1401
|
copyStringOption(options, input, "excerpt", "excerpt");
|
|
1402
|
+
copyStringOption(options, input, "banner-url", "bannerUrl");
|
|
1258
1403
|
copyNumberOption(options, input, "source-id", "sourceId");
|
|
1259
1404
|
if (typeof options["published-at"] === "string") {
|
|
1260
1405
|
input.publishedAt = normalizeCliPublishedAt(options["published-at"]);
|
|
@@ -1351,6 +1496,7 @@ function writePost(writer, post) {
|
|
|
1351
1496
|
writer.stdout(`slug: ${post.slug}`);
|
|
1352
1497
|
writer.stdout(`type: ${post.type}`);
|
|
1353
1498
|
writer.stdout(`title: ${post.title ?? ""}`);
|
|
1499
|
+
writer.stdout(`bannerUrl: ${post.bannerUrl ?? ""}`);
|
|
1354
1500
|
writer.stdout(`publishedAt: ${post.publishedAt ?? "draft"}`);
|
|
1355
1501
|
}
|
|
1356
1502
|
function writePostMutationOutput(writer, response, json) {
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
export interface LinkRelationshipMetadata {
|
|
2
|
+
name: string;
|
|
3
|
+
url?: string;
|
|
4
|
+
}
|
|
5
|
+
export interface DiscoveredLinkRelationships {
|
|
6
|
+
source?: LinkRelationshipMetadata;
|
|
7
|
+
authors?: LinkRelationshipMetadata[];
|
|
8
|
+
}
|
|
9
|
+
interface DiscoverLinkRelationshipsOptions {
|
|
10
|
+
url: string;
|
|
11
|
+
source: boolean;
|
|
12
|
+
authors: boolean;
|
|
13
|
+
fetchImpl?: typeof fetch;
|
|
14
|
+
timeoutMs?: number;
|
|
15
|
+
maxResponseBytes?: number;
|
|
16
|
+
}
|
|
17
|
+
export declare function discoverLinkRelationships(options: DiscoverLinkRelationshipsOptions): Promise<DiscoveredLinkRelationships>;
|
|
18
|
+
export {};
|
|
@@ -0,0 +1,234 @@
|
|
|
1
|
+
import { load } from "cheerio";
|
|
2
|
+
const defaultTimeoutMs = 5_000;
|
|
3
|
+
const defaultMaxResponseBytes = 1_048_576;
|
|
4
|
+
export async function discoverLinkRelationships(options) {
|
|
5
|
+
if (!options.source && !options.authors)
|
|
6
|
+
return {};
|
|
7
|
+
const fallbackSource = options.source ? createHostnameSource(options.url) : undefined;
|
|
8
|
+
const html = await fetchLinkHtml(options);
|
|
9
|
+
if (html === undefined) {
|
|
10
|
+
return fallbackSource === undefined ? {} : { source: fallbackSource };
|
|
11
|
+
}
|
|
12
|
+
const document = load(html);
|
|
13
|
+
const jsonLd = readJsonLdRecords(document("script[type='application/ld+json']")
|
|
14
|
+
.toArray()
|
|
15
|
+
.map((element) => document(element).text()));
|
|
16
|
+
const source = options.source
|
|
17
|
+
? (readJsonLdSource(jsonLd) ?? readMetaSource(document, options.url) ?? fallbackSource)
|
|
18
|
+
: undefined;
|
|
19
|
+
const authors = options.authors
|
|
20
|
+
? deduplicateRelationships([...readJsonLdAuthors(jsonLd), ...readMetaAuthors(document)])
|
|
21
|
+
: [];
|
|
22
|
+
return {
|
|
23
|
+
...(source === undefined ? {} : { source }),
|
|
24
|
+
...(authors.length === 0 ? {} : { authors }),
|
|
25
|
+
};
|
|
26
|
+
}
|
|
27
|
+
async function fetchLinkHtml(options) {
|
|
28
|
+
const fetchImpl = options.fetchImpl ?? fetch;
|
|
29
|
+
let response;
|
|
30
|
+
try {
|
|
31
|
+
response = await fetchImpl(options.url, {
|
|
32
|
+
headers: {
|
|
33
|
+
accept: "text/html,application/xhtml+xml",
|
|
34
|
+
"user-agent": "Slugkit link importer",
|
|
35
|
+
},
|
|
36
|
+
signal: AbortSignal.timeout(options.timeoutMs ?? defaultTimeoutMs),
|
|
37
|
+
});
|
|
38
|
+
}
|
|
39
|
+
catch {
|
|
40
|
+
return undefined;
|
|
41
|
+
}
|
|
42
|
+
if (!response.ok || !isHtmlResponse(response))
|
|
43
|
+
return undefined;
|
|
44
|
+
try {
|
|
45
|
+
return await readBoundedText(response, options.maxResponseBytes ?? defaultMaxResponseBytes);
|
|
46
|
+
}
|
|
47
|
+
catch {
|
|
48
|
+
return undefined;
|
|
49
|
+
}
|
|
50
|
+
}
|
|
51
|
+
function isHtmlResponse(response) {
|
|
52
|
+
const contentType = response.headers.get("content-type");
|
|
53
|
+
return (contentType !== null &&
|
|
54
|
+
(contentType.toLowerCase().includes("text/html") ||
|
|
55
|
+
contentType.toLowerCase().includes("application/xhtml+xml")));
|
|
56
|
+
}
|
|
57
|
+
async function readBoundedText(response, maxBytes) {
|
|
58
|
+
const declaredLength = Number.parseInt(response.headers.get("content-length") ?? "0", 10);
|
|
59
|
+
if (declaredLength > maxBytes)
|
|
60
|
+
throw new Error("Link metadata response is too large");
|
|
61
|
+
if (response.body === null)
|
|
62
|
+
return "";
|
|
63
|
+
const reader = response.body.getReader();
|
|
64
|
+
const chunks = [];
|
|
65
|
+
let totalBytes = 0;
|
|
66
|
+
while (true) {
|
|
67
|
+
const result = await reader.read();
|
|
68
|
+
if (result.done)
|
|
69
|
+
break;
|
|
70
|
+
totalBytes += result.value.byteLength;
|
|
71
|
+
if (totalBytes > maxBytes) {
|
|
72
|
+
await reader.cancel();
|
|
73
|
+
throw new Error("Link metadata response is too large");
|
|
74
|
+
}
|
|
75
|
+
chunks.push(result.value);
|
|
76
|
+
}
|
|
77
|
+
const body = new Uint8Array(totalBytes);
|
|
78
|
+
let offset = 0;
|
|
79
|
+
for (const chunk of chunks) {
|
|
80
|
+
body.set(chunk, offset);
|
|
81
|
+
offset += chunk.byteLength;
|
|
82
|
+
}
|
|
83
|
+
return new TextDecoder().decode(body);
|
|
84
|
+
}
|
|
85
|
+
function readJsonLdRecords(values) {
|
|
86
|
+
const records = [];
|
|
87
|
+
for (const value of values) {
|
|
88
|
+
try {
|
|
89
|
+
collectJsonLdRecords(JSON.parse(value), records);
|
|
90
|
+
}
|
|
91
|
+
catch {
|
|
92
|
+
// Ignore malformed structured metadata and continue with other discovery signals.
|
|
93
|
+
}
|
|
94
|
+
}
|
|
95
|
+
return records;
|
|
96
|
+
}
|
|
97
|
+
function collectJsonLdRecords(value, records) {
|
|
98
|
+
if (Array.isArray(value)) {
|
|
99
|
+
value.forEach((item) => collectJsonLdRecords(item, records));
|
|
100
|
+
return;
|
|
101
|
+
}
|
|
102
|
+
if (!isRecord(value))
|
|
103
|
+
return;
|
|
104
|
+
records.push(value);
|
|
105
|
+
if ("@graph" in value)
|
|
106
|
+
collectJsonLdRecords(value["@graph"], records);
|
|
107
|
+
}
|
|
108
|
+
function readJsonLdSource(records) {
|
|
109
|
+
for (const record of records) {
|
|
110
|
+
const relationships = readJsonLdRelationships(record.publisher, records);
|
|
111
|
+
if (relationships.length > 0)
|
|
112
|
+
return relationships[0];
|
|
113
|
+
}
|
|
114
|
+
return undefined;
|
|
115
|
+
}
|
|
116
|
+
function readJsonLdAuthors(records) {
|
|
117
|
+
return records.flatMap((record) => readJsonLdRelationships(record.author, records));
|
|
118
|
+
}
|
|
119
|
+
function readJsonLdRelationships(value, records, visitedIds = new Set()) {
|
|
120
|
+
if (Array.isArray(value)) {
|
|
121
|
+
return value.flatMap((item) => readJsonLdRelationships(item, records, visitedIds));
|
|
122
|
+
}
|
|
123
|
+
if (typeof value === "string" && value.trim() !== "") {
|
|
124
|
+
if (!isHttpUrl(value))
|
|
125
|
+
return [{ name: value }];
|
|
126
|
+
return readJsonLdReference(value, records, visitedIds);
|
|
127
|
+
}
|
|
128
|
+
if (!isRecord(value))
|
|
129
|
+
return [];
|
|
130
|
+
if (typeof value.name !== "string" || value.name.trim() === "") {
|
|
131
|
+
const id = typeof value["@id"] === "string" ? value["@id"] : undefined;
|
|
132
|
+
return id === undefined ? [] : readJsonLdReference(id, records, visitedIds);
|
|
133
|
+
}
|
|
134
|
+
const url = readJsonLdUrl(value);
|
|
135
|
+
return [{ name: value.name, ...(url === undefined ? {} : { url }) }];
|
|
136
|
+
}
|
|
137
|
+
function readJsonLdReference(id, records, visitedIds) {
|
|
138
|
+
if (visitedIds.has(id))
|
|
139
|
+
return [];
|
|
140
|
+
const referenced = records.find((record) => record["@id"] === id);
|
|
141
|
+
if (referenced === undefined)
|
|
142
|
+
return [];
|
|
143
|
+
const visited = new Set(visitedIds);
|
|
144
|
+
visited.add(id);
|
|
145
|
+
return readJsonLdRelationships(referenced, records, visited);
|
|
146
|
+
}
|
|
147
|
+
function readJsonLdUrl(value) {
|
|
148
|
+
for (const candidate of [value.url, value["@id"]]) {
|
|
149
|
+
if (typeof candidate === "string" && isHttpUrl(candidate))
|
|
150
|
+
return candidate;
|
|
151
|
+
}
|
|
152
|
+
return undefined;
|
|
153
|
+
}
|
|
154
|
+
function readMetaSource(document, destinationUrl) {
|
|
155
|
+
const name = readFirstMeta(document, [
|
|
156
|
+
"meta[property='og:site_name']",
|
|
157
|
+
"meta[name='application-name']",
|
|
158
|
+
"meta[name='apple-mobile-web-app-title']",
|
|
159
|
+
]);
|
|
160
|
+
return name === undefined ? undefined : { name, url: new URL(destinationUrl).origin };
|
|
161
|
+
}
|
|
162
|
+
function readMetaAuthors(document) {
|
|
163
|
+
return [
|
|
164
|
+
...readMetaValues(document, "meta[name='author']"),
|
|
165
|
+
...readMetaValues(document, "meta[property='article:author']"),
|
|
166
|
+
]
|
|
167
|
+
.filter((value) => !isHttpUrl(value))
|
|
168
|
+
.map((name) => ({ name }));
|
|
169
|
+
}
|
|
170
|
+
function readFirstMeta(document, selectors) {
|
|
171
|
+
for (const selector of selectors) {
|
|
172
|
+
const value = document(selector).first().attr("content")?.trim();
|
|
173
|
+
if (value !== undefined && value !== "")
|
|
174
|
+
return value;
|
|
175
|
+
}
|
|
176
|
+
return undefined;
|
|
177
|
+
}
|
|
178
|
+
function readMetaValues(document, selector) {
|
|
179
|
+
return document(selector)
|
|
180
|
+
.toArray()
|
|
181
|
+
.map((element) => document(element).attr("content")?.trim() ?? "")
|
|
182
|
+
.filter((value) => value !== "");
|
|
183
|
+
}
|
|
184
|
+
function createHostnameSource(value) {
|
|
185
|
+
const url = new URL(value);
|
|
186
|
+
return { name: url.hostname.replace(/^www\./iu, ""), url: url.origin };
|
|
187
|
+
}
|
|
188
|
+
function deduplicateRelationships(relationships) {
|
|
189
|
+
const unique = [];
|
|
190
|
+
for (const relationship of relationships) {
|
|
191
|
+
const index = unique.findIndex((candidate) => relationshipsMatch(candidate, relationship));
|
|
192
|
+
if (index === -1) {
|
|
193
|
+
unique.push(relationship);
|
|
194
|
+
}
|
|
195
|
+
else if (relationshipUrlScore(relationship) > relationshipUrlScore(unique[index])) {
|
|
196
|
+
unique[index] = relationship;
|
|
197
|
+
}
|
|
198
|
+
}
|
|
199
|
+
return unique;
|
|
200
|
+
}
|
|
201
|
+
function relationshipsMatch(left, right) {
|
|
202
|
+
if (normalizeName(left.name) === normalizeName(right.name))
|
|
203
|
+
return true;
|
|
204
|
+
return (left.url !== undefined &&
|
|
205
|
+
right.url !== undefined &&
|
|
206
|
+
canonicalizeUrl(left.url) === canonicalizeUrl(right.url));
|
|
207
|
+
}
|
|
208
|
+
function relationshipUrlScore(relationship) {
|
|
209
|
+
if (relationship.url === undefined)
|
|
210
|
+
return 0;
|
|
211
|
+
return new URL(relationship.url).hash === "" ? 2 : 1;
|
|
212
|
+
}
|
|
213
|
+
function normalizeName(value) {
|
|
214
|
+
return value.trim().replace(/\s+/gu, " ").toLocaleLowerCase("en-US");
|
|
215
|
+
}
|
|
216
|
+
function canonicalizeUrl(value) {
|
|
217
|
+
const url = new URL(value);
|
|
218
|
+
url.hash = "";
|
|
219
|
+
if (url.pathname !== "/")
|
|
220
|
+
url.pathname = url.pathname.replace(/\/+$/u, "");
|
|
221
|
+
return url.toString();
|
|
222
|
+
}
|
|
223
|
+
function isHttpUrl(value) {
|
|
224
|
+
try {
|
|
225
|
+
const url = new URL(value);
|
|
226
|
+
return url.protocol === "http:" || url.protocol === "https:";
|
|
227
|
+
}
|
|
228
|
+
catch {
|
|
229
|
+
return false;
|
|
230
|
+
}
|
|
231
|
+
}
|
|
232
|
+
function isRecord(value) {
|
|
233
|
+
return typeof value === "object" && value !== null && !Array.isArray(value);
|
|
234
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@evcraddock/slug-cli",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.11.0",
|
|
4
4
|
"description": "Command-line tool for Slugkit sites.",
|
|
5
5
|
"private": false,
|
|
6
6
|
"type": "module",
|
|
@@ -28,6 +28,7 @@
|
|
|
28
28
|
"dependencies": {
|
|
29
29
|
"@evcraddock/slug-core": "0.1.1",
|
|
30
30
|
"adm-zip": "^0.5.18",
|
|
31
|
+
"cheerio": "^1.2.0",
|
|
31
32
|
"yaml": "^2.9.0"
|
|
32
33
|
}
|
|
33
34
|
}
|