@sillsdev/docu-notion 0.14.0-alpha.13 → 0.14.0-alpha.15

package/README.md

@@ -6,7 +6,7 @@ Example Site: https://sillsdev.github.io/docu-notion-sample-site/
 
 # Instructions
 
-## 1. Set up your documentation site.
+## 1. Set up your documentation site
 
 First, prepare your markdown-based static file system like [Docusaurus](https://docusaurus.io/). For a shortcut with github actions, search, and deployment to github pages, you can just copy [this template](https://github.com/sillsdev/docu-notion-sample-site).
 
@@ -27,15 +27,15 @@ Go to the page that will be the root of your site. This page should have, as dir
 
 <img width="318" alt="image" src="https://github.com/sillsdev/docu-notion/assets/8448/810c6dca-f9ab-4370-93b4-dc1479332af7">
 
-## 5. Add your pages under your Outline page.
+## 5. Add your pages under your Outline page
 
 Currently, docu-notion expects that each page has only one of the following: sub-pages, links to other pages, or normal content. Do not mix them. You can add content pages directly here, but then you won't be able to make use of the workflow features. If those matter to you, instead make new pages under the "Database" and then link to them in your outline pages.
 
 ## 6. Pull your pages
 
-First, determine the id of your root page by clicking "Share" and looking at the url it gives you. E.g.
-https://www.notion.so/hattonjohn/My-Docs-0456aa5842946bdbea3a4f37c97a0e5
-means that the id is "0456aa5842946PRETEND4f37c97a0e5".
+First, determine the ID of your root page by clicking "Share" and looking at the url it gives you. E.g.
+`https://www.notion.so/hattonjohn/My-Docs-0456aa5842946PRETEND4f37c97a0e5`
+means that the ID is `0456aa5842946PRETEND4f37c97a0e5`.
 
 Try it out:
 
@@ -90,7 +90,9 @@ One of the big attractions of Notion for large documentation projects is that yo
 
 ## Slugs
 
-By default, pages will be given a slug based on the Notion id. For a human-readable URL, add a notion property named `Slug` to your database pages and enter a value in there that will work well in a URL. That is, no spaces, ?, #, /, etc.
+By default, pages will be given a slug based on the Notion ID. For a human-readable URL, add a notion property named `Slug` to your database pages and enter a value in there that will work well in a URL. That is, no spaces, ?, #, /, etc.
+
+See `Options` to require slugs in Notion.
 
 ## Known Limitations
 
@@ -112,25 +114,27 @@ NOTE: if you just localize an image, it will not get picked up. You also must lo
 
 # Automated builds with Github Actions
 
-Here is a working Github Action script to copy and customize: https://github.com/BloomBooks/bloom-docs/blob/master/.github/workflows/release.yml
+Here is a [working Github Action script to copy and customize](https://github.com/BloomBooks/bloom-docs/blob/master/.github/workflows/release.yml).
 
 # Command line
 
-Usage: docu-notion -n <token> -r <root> [options]
+Usage: `docu-notion -n <token> -r <root> [options]`
 
 Options:
 
 | flag                                  | required? | description                                                                                                                                                                                                        |
 | ------------------------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
-| -n, --notion-token <string>           | required  | notion api token, which looks like `secret_3bc1b50XFYb15123RHF243x43450XFY33250XFYa343`                                                                                                                            |
-| -r, --root-page <string>              | required  | The 31 character ID of the page which is the root of your docs page in notion. The code will look like `9120ec9960244ead80fa2ef4bc1bba25`. This page must have a child page named 'Outline'                        |
-| -m, --markdown-output-path <string>   |           | Root of the hierarchy for md files. WARNING: node-pull-mdx will delete files from this directory. Note also that if it finds localized images, it will create an i18n/ directory as a sibling. (default: "./docs") |
-| -t, --status-tag <string>             |           | Database pages without a Notion page property 'status' matching this will be ignored. Use '\*' to ignore status altogether. (default: `Publish`)                                                                   |
-| --locales <codes>                     |           | Comma-separated list of iso 639-2 codes, the same list as in docusaurus.config.js, minus the primary (i.e. 'en'). This is needed for image localization. (default: [])                                             |
-| -l, --log-level <level>               |           | Log level (choices: `info`, `verbose`, `debug`)                                                                                                                                                                    |
-| -i, --img-output-path <string>        |           | Path to directory where images will be stored. If this is not included, images will be placed in the same directory as the document that uses them, which then allows for localization of screenshots.             |
-| -p, --img-prefix-in-markdown <string> |           | When referencing an image from markdown, prefix with this path instead of the full img-output-path. Should be used only in conjunction with --img-output-path.                                                     |
-| -h, --help                            |           | display help for command                                                                                                                                                                                           |
+| `-n, --notion-token <string>`           | required  | notion api token, which looks like `secret_3bc1b50XFYb15123RHF243x43450XFY33250XFYa343`                                                                                                                            |
+| `-r, --root-page <string>`              | required  | The 31 character ID of the page which is the root of your docs page in notion. The code will look like `9120ec9960244ead80fa2ef4bc1bba25`. This page must have a child page named 'Outline'                        |
+| `-m, --markdown-output-path <string>`   |           | Root of the hierarchy for md files. WARNING: node-pull-mdx will delete files from this directory. Note also that if it finds localized images, it will create an i18n/ directory as a sibling. (default: `./docs`) |
+| `-t, --status-tag <string>`             |           | Database pages without a Notion page property 'status' matching this will be ignored. Use '\*' to ignore status altogether. (default: `Publish`)                                                                   |
+| `--locales <codes>`                     |           | Comma-separated list of iso 639-2 codes, the same list as in docusaurus.config.js, minus the primary (i.e. 'en'). This is needed for image localization. (default: `[]`)                                             |
+| `-l, --log-level <level>`               |           | Log level (choices: `info`, `verbose`, `debug`)                                                                                                                                                                    |
+| `-i, --img-output-path <string>`        |           | Path to directory where images will be stored. If this is not included, images will be placed in the same directory as the document that uses them, which then allows for localization of screenshots.             |
+| `-p, --img-prefix-in-markdown <string>` |           | When referencing an image from markdown, prefix with this path instead of the full img-output-path. Should be used only in conjunction with --img-output-path.                                                     |
+| `--require-slugs`                       |           | If set, docu-notion will fail if any pages it would otherwise publish are missing a slug in Notion. |
+| `--image-file-name-format <format>`     |           | choices:<ul><li>`default`: {page slug (if any)}.{image block ID}</li><li>`content-hash`: Use a hash of the image content.</li><li>`legacy`: Use the legacy (before v0.16) method of determining file names. Set this to maintain backward compatibility.</li></ul>All formats will use the original file extension. |
+| `-h, --help`                            |           | display help for command                              |
 
 # Plugins
 
@@ -152,8 +156,10 @@ The default admonition type, if no matching icon is found, is "note".
 # Known Workarounds
 
 ### Start a numbered list at a number other than 1
+
 In Notion, make sure the block is "Text," not "Numbered List".
+
 - But make sure the number does NOT have a space in front of it. This can/will cause issues with sub-list items.
 - One way to get Notion to let you do this:
-    - Create a numbered list item where the text duplicates the number you want. Convert that numbered list item to "Text."
-    - i.e. Type "1. 1. Item one." Notion makes the first "1." into a number in a list. When you convert back to "Text," you're left with plain text "1. Item one."
+  - Create a numbered list item where the text duplicates the number you want. Convert that numbered list item to "Text."
+  - i.e. Type "1. 1. Item one." Notion makes the first "1." into a number in a list. When you convert back to "Text," you're left with plain text "1. Item one."

package/dist/MakeImagePersistencePlan.d.ts

@@ -1,3 +1,4 @@
 import { ImageSet } from "./images";
-export declare function makeImagePersistencePlan(imageSet: ImageSet, imageOutputRootPath: string, imagePrefix: string): void;
+import { DocuNotionOptions } from "./pull";
+export declare function makeImagePersistencePlan(options: DocuNotionOptions, imageSet: ImageSet, imageBlockId: string, imageOutputRootPath: string, imagePrefix: string): void;
 export declare function hashOfString(s: string): number;

package/dist/MakeImagePersistencePlan.js

@@ -22,13 +22,17 @@ var __importStar = (this && this.__importStar) || function (mod) {
     __setModuleDefault(result, mod);
     return result;
 };
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.hashOfString = exports.makeImagePersistencePlan = void 0;
 const Path = __importStar(require("path"));
 const log_1 = require("./log");
 const process_1 = require("process");
-function makeImagePersistencePlan(imageSet, imageOutputRootPath, imagePrefix) {
-    var _a, _b;
+const crypto_1 = __importDefault(require("crypto"));
+function makeImagePersistencePlan(options, imageSet, imageBlockId, imageOutputRootPath, imagePrefix) {
+    var _a, _b, _c;
     const urlBeforeQuery = imageSet.primaryUrl.split("?")[0];
     let imageFileExtension = (_a = imageSet.fileType) === null || _a === void 0 ? void 0 : _a.ext;
     if (!imageFileExtension) {
@@ -39,18 +43,52 @@ function makeImagePersistencePlan(imageSet, imageOutputRootPath, imagePrefix) {
             (0, process_1.exit)(1);
         }
     }
-    // Since most images come from pasting screenshots, there isn't normally a filename. That's fine, we just make a hash of the url
-    // Images that are stored by notion come to us with a complex url that changes over time, so we pick out the UUID that doesn't change. Example:
-    //    https://s3.us-west-2.amazonaws.com/secure.notion-static.com/d1058f46-4d2f-4292-8388-4ad393383439/Untitled.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIAT73L2G45EIPT3X45%2F20220516%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20220516T233630Z&X-Amz-Expires=3600&X-Amz-Signature=f215704094fcc884d37073b0b108cf6d1c9da9b7d57a898da38bc30c30b4c4b5&X-Amz-SignedHeaders=host&x-id=GetObject
-    // But around Sept 2023, they changed the url to be something like:
-    //    https://prod-files-secure.s3.us-west-2.amazonaws.com/d9a2b712-cf69-4bd6-9d65-87a4ceeacca2/d1bcdc8c-b065-4e40-9a11-392aabeb220e/Untitled.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIAT73L2G45EIPT3X45%2F20230915%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20230915T161258Z&X-Amz-Expires=3600&X-Amz-Signature=28fca48e65fba86d539c3c4b7676fce1fa0857aa194f7b33dd4a468ecca6ab24&X-Amz-SignedHeaders=host&x-id=GetObject
-    // The thing we want is the last UUID before the ?
-    const thingToHash = (_b = findLastUuid(urlBeforeQuery)) !== null && _b !== void 0 ? _b : urlBeforeQuery;
-    const hash = hashOfString(thingToHash);
-    imageSet.outputFileName = `${hash}.${imageFileExtension}`;
+    if (options.imageFileNameFormat === "legacy") {
+        // Original behavior and comment:
+        //   Since most images come from pasting screenshots, there isn't normally a filename. That's fine, we just make a hash of the url
+        //   Images that are stored by notion come to us with a complex url that changes over time, so we pick out the UUID that doesn't change. Example:
+        //      https://s3.us-west-2.amazonaws.com/secure.notion-static.com/d1058f46-4d2f-4292-8388-4ad393383439/Untitled.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIAT73L2G45EIPT3X45%2F20220516%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20220516T233630Z&X-Amz-Expires=3600&X-Amz-Signature=f215704094fcc884d37073b0b108cf6d1c9da9b7d57a898da38bc30c30b4c4b5&X-Amz-SignedHeaders=host&x-id=GetObject
+        //   But around Sept 2023, they changed the url to be something like:
+        //      https://prod-files-secure.s3.us-west-2.amazonaws.com/d9a2b712-cf69-4bd6-9d65-87a4ceeacca2/d1bcdc8c-b065-4e40-9a11-392aabeb220e/Untitled.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIAT73L2G45EIPT3X45%2F20230915%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20230915T161258Z&X-Amz-Expires=3600&X-Amz-Signature=28fca48e65fba86d539c3c4b7676fce1fa0857aa194f7b33dd4a468ecca6ab24&X-Amz-SignedHeaders=host&x-id=GetObject
+        //   The thing we want is the last UUID before the ?
+        const thingToHash = (_b = findLastUuid(urlBeforeQuery)) !== null && _b !== void 0 ? _b : urlBeforeQuery;
+        const hash = hashOfString(thingToHash);
+        imageSet.outputFileName = `${hash}.${imageFileExtension}`;
+    }
+    else if (options.imageFileNameFormat === "content-hash") {
+        // This was requested by a user: https://github.com/sillsdev/docu-notion/issues/76.
+        // We chose not to include it in the default file name because we want to maintain
+        // as much stability in the file name as feasible for an image localization workflow.
+        // However, particularly in a workflow which is not concerned with localization,
+        // this could be a good option. One benefit is that the image only needs to exist once
+        // in the file system regardless of how many times it is used in the site.
+        const imageHash = hashOfBufferContent(imageSet.primaryBuffer);
+        imageSet.outputFileName = `${imageHash}.${imageFileExtension}`;
+    }
+    else {
+        // We decided not to do this for the default format because it means
+        // instability for the file name in Crowdin, which causes loss of localizations.
+        // If we decide to include it in the future, we should add a unit test.
+        // const imageFileName = Path.basename(urlBeforeQuery);
+        // const imageFileNameWithoutExtension = Path.parse(imageFileName).name;
+        // const originalFileNamePart = ["untitled", "unnamed"].includes(
+        //   imageFileNameWithoutExtension.toLocaleLowerCase()
+        // )
+        //   ? ""
+        //   : `${imageFileNameWithoutExtension.substring(0, 50)}.`;
+        // Format is page slug (if there is one) followed by the image block ID from Notion.
+        // The image block ID will remain stable as long as any changes to the image are done
+        // using the Replace feature. Also, image blocks can be moved using the Move To feature.
+        // We decided to include the page slug for easier workflow during localization, particularly in Crowdin.
+        // The block ID is a unique GUID and thus provides a unique file name.
+        const pageSlugPart = ((_c = imageSet.pageInfo) === null || _c === void 0 ? void 0 : _c.slug)
+            ? `${imageSet.pageInfo.slug.replace(/^\//, "")}.`
+            : "";
+        imageSet.outputFileName = `${pageSlugPart}${imageBlockId}.${imageFileExtension}`;
+    }
     imageSet.primaryFileOutputPath = Path.posix.join((imageOutputRootPath === null || imageOutputRootPath === void 0 ? void 0 : imageOutputRootPath.length) > 0
         ? imageOutputRootPath
-        : imageSet.pathToParentDocument, imageSet.outputFileName);
+        : imageSet.pageInfo.directoryContainingMarkdown, decodeURI(imageSet.outputFileName));
     if (imageOutputRootPath && imageSet.localizedUrls.length) {
         (0, log_1.error)("imageOutputPath was declared, but one or more localizedUrls were found too. If you are going to localize screenshots, then you can't declare an imageOutputPath.");
         (0, process_1.exit)(1);
@@ -76,3 +114,7 @@ function hashOfString(s) {
     return Math.abs(hash);
 }
 exports.hashOfString = hashOfString;
+function hashOfBufferContent(buffer) {
+    const hash = crypto_1.default.createHash("sha256").update(buffer).digest("hex");
+    return hash.slice(0, 20);
+}

package/dist/images.d.ts

@@ -1,7 +1,7 @@
 /// <reference types="node" />
 import { FileTypeResult } from "file-type";
 import { ListBlockChildrenResponseResult } from "notion-to-md/build/types";
-import { IPlugin } from "./plugins/pluginTypes";
+import { IDocuNotionContext, IDocuNotionContextPageInfo, IPlugin } from "./plugins/pluginTypes";
 export type ImageSet = {
     primaryUrl: string;
     caption?: string;
@@ -9,8 +9,7 @@ export type ImageSet = {
         iso632Code: string;
         url: string;
     }>;
-    pathToParentDocument?: string;
-    relativePathToParentDocument?: string;
+    pageInfo?: IDocuNotionContextPageInfo;
     primaryBuffer?: Buffer;
     fileType?: FileTypeResult;
     primaryFileOutputPath?: string;
@@ -19,6 +18,6 @@ export type ImageSet = {
 };
 export declare function initImageHandling(prefix: string, outputPath: string, incomingLocales: string[]): Promise<void>;
 export declare const standardImageTransformer: IPlugin;
-export declare function markdownToMDImageTransformer(block: ListBlockChildrenResponseResult, fullPathToDirectoryContainingMarkdown: string, relativePathToThisPage: string): Promise<string>;
+export declare function markdownToMDImageTransformer(block: ListBlockChildrenResponseResult, context: IDocuNotionContext): Promise<string>;
 export declare function parseImageBlock(image: any): ImageSet;
 export declare function cleanupOldImages(): Promise<void>;

package/dist/images.js

@@ -42,7 +42,7 @@ const axios_1 = __importDefault(require("axios"));
 const Path = __importStar(require("path"));
 const MakeImagePersistencePlan_1 = require("./MakeImagePersistencePlan");
 const log_1 = require("./log");
-// We several things here:
+// We handle several things here:
 // 1) copy images locally instead of leaving them in Notion
 // 2) change the links to point here
 // 3) read the caption and if there are localized images, get those too
@@ -75,16 +75,16 @@ exports.standardImageTransformer = {
             type: "image",
             // we have to set this one up for each page because we need to
             // give it two extra parameters that are context for each page
-            getStringFromBlock: (context, block) => markdownToMDImageTransformer(block, context.directoryContainingMarkdown, context.relativeFilePathToFolderContainingPage),
+            getStringFromBlock: (context, block) => markdownToMDImageTransformer(block, context),
         },
     ],
 };
 // This is a "custom transformer" function passed to notion-to-markdown
 // eslint-disable-next-line @typescript-eslint/require-await
-function markdownToMDImageTransformer(block, fullPathToDirectoryContainingMarkdown, relativePathToThisPage) {
+function markdownToMDImageTransformer(block, context) {
     return __awaiter(this, void 0, void 0, function* () {
         const image = block.image;
-        yield processImageBlock(image, fullPathToDirectoryContainingMarkdown, relativePathToThisPage);
+        yield processImageBlock(block, context);
         // just concatenate the caption text parts together
         const altText = image.caption
             // eslint-disable-next-line @typescript-eslint/no-unsafe-return
@@ -95,17 +95,17 @@ function markdownToMDImageTransformer(block, fullPathToDirectoryContainingMarkdo
     });
 }
 exports.markdownToMDImageTransformer = markdownToMDImageTransformer;
-function processImageBlock(imageBlock, pathToParentDocument, relativePathToThisPage) {
+function processImageBlock(block, context) {
     return __awaiter(this, void 0, void 0, function* () {
+        const imageBlock = block.image;
         (0, log_1.logDebug)("processImageBlock", JSON.stringify(imageBlock));
         const imageSet = parseImageBlock(imageBlock);
-        imageSet.pathToParentDocument = pathToParentDocument;
-        imageSet.relativePathToParentDocument = relativePathToThisPage;
+        imageSet.pageInfo = context.pageInfo;
         // enhance: it would much better if we could split the changes to markdown separately from actual reading/writing,
         // so that this wasn't part of the markdown-creation loop. It's already almost there; we just need to
         // save the imageSets somewhere and then do the actual reading/writing later.
         yield readPrimaryImage(imageSet);
-        (0, MakeImagePersistencePlan_1.makeImagePersistencePlan)(imageSet, imageOutputPath, imagePrefix);
+        (0, MakeImagePersistencePlan_1.makeImagePersistencePlan)(context.options, imageSet, block.id, imageOutputPath, imagePrefix);
         yield saveImage(imageSet);
         // change the src to point to our copy of the image
         if ("file" in imageBlock) {
@@ -156,7 +156,7 @@ function saveImage(imageSet) {
                 (0, log_1.verbose)(`No localized image specified for ${localizedImage.iso632Code}, will use primary image.`);
                 // otherwise, we're going to fall back to outputting the primary image here
             }
-            const directory = `./i18n/${localizedImage.iso632Code}/docusaurus-plugin-content-docs/current/${imageSet.relativePathToParentDocument}`;
+            const directory = `./i18n/${localizedImage.iso632Code}/docusaurus-plugin-content-docs/current/${imageSet.pageInfo.relativeFilePathToFolderContainingPage}`;
             writeImageIfNew((directory + "/" + imageSet.outputFileName).replaceAll("//", "/"), buffer);
         }
     });

package/dist/latex.spec.js

@@ -33,8 +33,12 @@ test("Latex Rendering", () => __awaiter(void 0, void 0, void 0, function* () {
         getBlockChildren: (id) => {
             return new Promise(resolve => resolve(new Array()));
         },
-        directoryContainingMarkdown: "",
-        relativeFilePathToFolderContainingPage: "",
+        // this changes with each page
+        pageInfo: {
+            directoryContainingMarkdown: "",
+            relativeFilePathToFolderContainingPage: "",
+            slug: "",
+        },
         layoutStrategy: layoutStrategy,
         notionToMarkdown: new notion_to_md_1.NotionToMarkdown({ notionClient }),
         options: {

package/dist/makeImagePersistencePlan.spec.js

@@ -1,60 +1,79 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 const MakeImagePersistencePlan_1 = require("./MakeImagePersistencePlan");
+const optionsUsingDefaultNaming = {
+    notionToken: "",
+    rootPage: "",
+    locales: [],
+    markdownOutputPath: "",
+    imgOutputPath: "",
+    imgPrefixInMarkdown: "",
+    statusTag: "",
+};
+const testImageSet = {
+    primaryUrl: "https://s3.us-west-2.amazonaws.com/primaryImage?Blah=foo",
+    localizedUrls: [],
+    pageInfo: {
+        directoryContainingMarkdown: "/pathToParentSomewhere/",
+        relativeFilePathToFolderContainingPage: "",
+        slug: "my-page",
+    },
+    fileType: { ext: "png", mime: "image/png" },
+    primaryBuffer: Buffer.from("some fake image content"),
+};
 test("primary file with explicit file output path and prefix", () => {
-    const imageSet = {
-        primaryUrl: "https://s3.us-west-2.amazonaws.com/primaryImage?Blah=foo",
-        localizedUrls: [],
-        pathToParentDocument: "/pathToParentSomewhere/",
-        fileType: { ext: "png", mime: "image/png" },
-    };
-    (0, MakeImagePersistencePlan_1.makeImagePersistencePlan)(imageSet, "./static/notion_imgs", "/notion_imgs");
-    const expectedHash = (0, MakeImagePersistencePlan_1.hashOfString)("https://s3.us-west-2.amazonaws.com/primaryImage");
-    expect(imageSet.outputFileName).toBe(`${expectedHash}.png`);
-    expect(imageSet.primaryFileOutputPath).toBe(`static/notion_imgs/${expectedHash}.png`);
-    expect(imageSet.filePathToUseInMarkdown).toBe(`/notion_imgs/${expectedHash}.png`);
+    (0, MakeImagePersistencePlan_1.makeImagePersistencePlan)(optionsUsingDefaultNaming, testImageSet, "ABC-123", "./static/notion_imgs", "/notion_imgs");
+    const expectedFileName = "my-page.ABC-123.png";
+    expect(testImageSet.outputFileName).toBe(`${expectedFileName}`);
+    expect(testImageSet.primaryFileOutputPath).toBe(`static/notion_imgs/${expectedFileName}`);
+    expect(testImageSet.filePathToUseInMarkdown).toBe(`/notion_imgs/${expectedFileName}`);
 });
 test("primary file with defaults for image output path and prefix", () => {
-    const imageSet = {
-        primaryUrl: "https://s3.us-west-2.amazonaws.com/primaryImage?Blah=foo",
-        localizedUrls: [],
-        pathToParentDocument: "/pathToParentSomewhere/",
-        fileType: { ext: "png", mime: "image/png" },
-    };
-    (0, MakeImagePersistencePlan_1.makeImagePersistencePlan)(imageSet, "", "");
-    const expectedHash = (0, MakeImagePersistencePlan_1.hashOfString)("https://s3.us-west-2.amazonaws.com/primaryImage");
-    expect(imageSet.outputFileName).toBe(`${expectedHash}.png`);
+    (0, MakeImagePersistencePlan_1.makeImagePersistencePlan)(optionsUsingDefaultNaming, testImageSet, "ABC-123", "", "");
+    const expectedFileName = "my-page.ABC-123.png";
+    expect(testImageSet.outputFileName).toBe(`${expectedFileName}`);
     // the default behavior is to put the image next to the markdown file
-    expect(imageSet.primaryFileOutputPath).toBe(`/pathToParentSomewhere/${expectedHash}.png`);
-    expect(imageSet.filePathToUseInMarkdown).toBe(`./${expectedHash}.png`);
+    expect(testImageSet.primaryFileOutputPath).toBe(`/pathToParentSomewhere/${expectedFileName}`);
+    expect(testImageSet.filePathToUseInMarkdown).toBe(`./${expectedFileName}`);
 });
 test("falls back to getting file extension from url if not in fileType", () => {
-    const imageSet = {
-        primaryUrl: "https://s3.us-west-2.amazonaws.com/primaryImage.png",
-        localizedUrls: [],
-        pathToParentDocument: "/pathToParentSomewhere/",
-    };
-    (0, MakeImagePersistencePlan_1.makeImagePersistencePlan)(imageSet, "", "");
-    const expectedHash = (0, MakeImagePersistencePlan_1.hashOfString)("https://s3.us-west-2.amazonaws.com/primaryImage.png");
-    expect(imageSet.outputFileName).toBe(`${expectedHash}.png`);
+    (0, MakeImagePersistencePlan_1.makeImagePersistencePlan)(optionsUsingDefaultNaming, testImageSet, "ABC-123", "", "");
+    expect(testImageSet.outputFileName).toBe("my-page.ABC-123.png");
+});
+// I'm not sure it is even possible to have encoded characters in the slug, but this proves
+// we are properly encoding them in the markdown file but not in the file system.
+// (This test originally was for including original file names, but we decided not to do that.)
+test("handles encoded characters", () => {
+    const imageSet = Object.assign(Object.assign({}, testImageSet), { pageInfo: {
+            directoryContainingMarkdown: "/pathToParentSomewhere/",
+            relativeFilePathToFolderContainingPage: "",
+            slug: "my-page%281%29",
+        } });
+    (0, MakeImagePersistencePlan_1.makeImagePersistencePlan)(optionsUsingDefaultNaming, imageSet, "ABC-123", "", "");
+    expect(imageSet.primaryFileOutputPath).toBe(`/pathToParentSomewhere/my-page(1).ABC-123.png`);
+    expect(imageSet.filePathToUseInMarkdown).toBe(`./my-page%281%29.ABC-123.png`);
+});
+const optionsUsingHashNaming = Object.assign(Object.assign({}, optionsUsingDefaultNaming), { imageFileNameFormat: "content-hash" });
+test("hash naming", () => {
+    (0, MakeImagePersistencePlan_1.makeImagePersistencePlan)(optionsUsingHashNaming, testImageSet, "ABC-123", "", "");
+    const expectedFileName = "fe3f26fd515b3cf299ac.png";
+    expect(testImageSet.outputFileName).toBe(`${expectedFileName}`);
+});
+const optionsUsingLegacyNaming = Object.assign(Object.assign({}, optionsUsingDefaultNaming), { imageFileNameFormat: "legacy" });
+test("Legacy naming", () => {
+    (0, MakeImagePersistencePlan_1.makeImagePersistencePlan)(optionsUsingLegacyNaming, testImageSet, "ABC-123", "./static/notion_imgs", "/notion_imgs");
+    const expectedHash = (0, MakeImagePersistencePlan_1.hashOfString)("https://s3.us-west-2.amazonaws.com/primaryImage");
+    expect(testImageSet.outputFileName).toBe(`${expectedHash}.png`);
 });
-test("properly extract UUID from old-style notion image url", () => {
-    const imageSet = {
-        primaryUrl: "https://s3.us-west-2.amazonaws.com/secure.notion-static.com/e1058f46-4d2f-4292-8388-4ad393383439/Untitled.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIAT73L2G45EIPT3X45%2F20220516%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20220516T233630Z&X-Amz-Expires=3600&X-Amz-Signature=f215704094fcc884d37073b0b108cf6d1c9da9b7d57a898da38bc30c30b4c4b5&X-Amz-SignedHeaders=host&x-id=GetObject",
-        localizedUrls: [],
-        fileType: { ext: "png", mime: "image/png" },
-    };
-    (0, MakeImagePersistencePlan_1.makeImagePersistencePlan)(imageSet, "./static/notion_imgs", "/notion_imgs");
+test("Legacy naming - properly extract UUID from old-style notion image url", () => {
+    const imageSet = Object.assign(Object.assign({}, testImageSet), { primaryUrl: "https://s3.us-west-2.amazonaws.com/secure.notion-static.com/e1058f46-4d2f-4292-8388-4ad393383439/Untitled.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIAT73L2G45EIPT3X45%2F20220516%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20220516T233630Z&X-Amz-Expires=3600&X-Amz-Signature=f215704094fcc884d37073b0b108cf6d1c9da9b7d57a898da38bc30c30b4c4b5&X-Amz-SignedHeaders=host&x-id=GetObject" });
+    (0, MakeImagePersistencePlan_1.makeImagePersistencePlan)(optionsUsingLegacyNaming, imageSet, "ABC-123", "./static/notion_imgs", "/notion_imgs");
     const expectedHash = (0, MakeImagePersistencePlan_1.hashOfString)("e1058f46-4d2f-4292-8388-4ad393383439");
     expect(imageSet.outputFileName).toBe(`${expectedHash}.png`);
 });
-test("properly extract UUID from new-style (Sept 2023) notion image url", () => {
-    const imageSet = {
-        primaryUrl: "https://prod-files-secure.s3.us-west-2.amazonaws.com/d9a2b712-cf69-4bd6-9d65-87a4ceeacca2/d1bcdc8c-b065-4e40-9a11-392aabeb220e/Untitled.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIAT73L2G45EIPT3X45%2F20230915%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20230915T161258Z&X-Amz-Expires=3600&X-Amz-Signature=28fca48e65fba86d539c3c4b7676fce1fa0857aa194f7b33dd4a468ecca6ab24&X-Amz-SignedHeaders=host&x-id=GetObject",
-        localizedUrls: [],
-        fileType: { ext: "png", mime: "image/png" },
-    };
-    (0, MakeImagePersistencePlan_1.makeImagePersistencePlan)(imageSet, "./static/notion_imgs", "/notion_imgs");
+test("Legacy naming - properly extract UUID from new-style (Sept 2023) notion image url", () => {
+    const imageSet = Object.assign(Object.assign({}, testImageSet), { primaryUrl: "https://prod-files-secure.s3.us-west-2.amazonaws.com/d9a2b712-cf69-4bd6-9d65-87a4ceeacca2/d1bcdc8c-b065-4e40-9a11-392aabeb220e/Untitled.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIAT73L2G45EIPT3X45%2F20230915%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20230915T161258Z&X-Amz-Expires=3600&X-Amz-Signature=28fca48e65fba86d539c3c4b7676fce1fa0857aa194f7b33dd4a468ecca6ab24&X-Amz-SignedHeaders=host&x-id=GetObject" });
+    (0, MakeImagePersistencePlan_1.makeImagePersistencePlan)(optionsUsingLegacyNaming, imageSet, "ABC-123", "./static/notion_imgs", "/notion_imgs");
     const expectedHash = (0, MakeImagePersistencePlan_1.hashOfString)("d1bcdc8c-b065-4e40-9a11-392aabeb220e");
     expect(imageSet.outputFileName).toBe(`${expectedHash}.png`);
 });

package/dist/plugins/pluginTestRun.d.ts

@@ -7,4 +7,4 @@ export declare function makeSamplePageObject(options: {
     name?: string;
     id?: string;
 }): NotionPage;
-export declare function oneBlockToMarkdown(config: IDocuNotionConfig, block: object, targetPage?: NotionPage): Promise<string>;
+export declare function oneBlockToMarkdown(config: IDocuNotionConfig, block: Record<string, unknown>, targetPage?: NotionPage): Promise<string>;

package/dist/plugins/pluginTestRun.js

@@ -49,8 +49,11 @@ children, validApiKey) {
             },
             imports: [],
             //TODO might be needed for some tests, e.g. the image transformer...
-            directoryContainingMarkdown: "not yet",
-            relativeFilePathToFolderContainingPage: "not yet",
+            pageInfo: {
+                directoryContainingMarkdown: "not yet",
+                relativeFilePathToFolderContainingPage: "not yet",
+                slug: "not yet",
+            },
             layoutStrategy: new HierarchicalNamedLayoutStrategy_1.HierarchicalNamedLayoutStrategy(),
             options: {
                 notionToken: "",
@@ -90,8 +93,8 @@ children, validApiKey) {
             // },
         };
         if (pages && pages.length) {
-            console.log(pages[0].matchesLinkId);
-            console.log(docunotionContext.pages[0].matchesLinkId);
+            // console.log(pages[0].matchesLinkId);
+            // console.log(docunotionContext.pages[0].matchesLinkId);
         }
         const r = yield (0, transform_1.getMarkdownFromNotionBlocks)(docunotionContext, config, blocks);
         //console.log("blocksToMarkdown", r);
@@ -219,7 +222,7 @@ function makeSamplePageObject(options) {
         metadata: m,
         foundDirectlyInOutline: false,
     });
-    console.log(p.matchesLinkId);
+    // console.log(p.matchesLinkId);
     return p;
 }
 exports.makeSamplePageObject = makeSamplePageObject;

package/dist/plugins/pluginTypes.d.ts

@@ -35,11 +35,15 @@ export type IDocuNotionContext = {
     options: DocuNotionOptions;
     getBlockChildren: IGetBlockChildrenFn;
     notionToMarkdown: NotionToMarkdown;
-    directoryContainingMarkdown: string;
-    relativeFilePathToFolderContainingPage: string;
+    pageInfo: IDocuNotionContextPageInfo;
     convertNotionLinkToLocalDocusaurusLink: (url: string) => string | undefined;
     pages: NotionPage[];
     counts: ICounts;
     imports: string[];
 };
+export type IDocuNotionContextPageInfo = {
+    directoryContainingMarkdown: string;
+    relativeFilePathToFolderContainingPage: string;
+    slug: string;
+};
 export {};

package/dist/pull.d.ts

@@ -1,5 +1,6 @@
 import { Client } from "@notionhq/client";
 import { ListBlockChildrenResponseResults } from "notion-to-md/build/types";
+type ImageFileNameFormat = "default" | "content-hash" | "legacy";
 export type DocuNotionOptions = {
     notionToken: string;
     rootPage: string;
@@ -8,8 +9,11 @@ export type DocuNotionOptions = {
     imgOutputPath: string;
     imgPrefixInMarkdown: string;
     statusTag: string;
+    requireSlugs?: boolean;
+    imageFileNameFormat?: ImageFileNameFormat;
 };
 export declare function notionPull(options: DocuNotionOptions): Promise<void>;
 export declare function executeWithRateLimitAndRetries<T>(label: string, asyncFunction: () => Promise<T>): Promise<T>;
 export declare function initNotionClient(notionToken: string): Client;
 export declare function numberChildrenIfNumberedList(blocks: ListBlockChildrenResponseResults): void;
+export {};

package/dist/pull.js

@@ -54,6 +54,7 @@ const counts = {
     skipped_because_empty: 0,
     skipped_because_status: 0,
     skipped_because_level_cannot_have_content: 0,
+    error_because_no_slug: 0,
 };
 function notionPull(options) {
     return __awaiter(this, void 0, void 0, function* () {
@@ -102,8 +103,12 @@ function outputPages(options, config, pages) {
     return __awaiter(this, void 0, void 0, function* () {
         const context = {
             getBlockChildren: getBlockChildren,
-            directoryContainingMarkdown: "",
-            relativeFilePathToFolderContainingPage: "",
+            // this changes with each page
+            pageInfo: {
+                directoryContainingMarkdown: "",
+                relativeFilePathToFolderContainingPage: "",
+                slug: "",
+            },
             layoutStrategy: layoutStrategy,
             notionToMarkdown: notionToMarkdown,
             options: options,
@@ -116,10 +121,11 @@ function outputPages(options, config, pages) {
             layoutStrategy.pageWasSeen(page);
             const mdPath = layoutStrategy.getPathForPage(page, ".md");
             // most plugins should not write to disk, but those handling image files need these paths
-            context.directoryContainingMarkdown = Path.dirname(mdPath);
+            context.pageInfo.directoryContainingMarkdown = Path.dirname(mdPath);
             // TODO: This needs clarifying: getLinkPathForPage() is about urls, but
             // downstream images.ts is using it as a file system path
-            context.relativeFilePathToFolderContainingPage = Path.dirname(layoutStrategy.getLinkPathForPage(page));
+            context.pageInfo.relativeFilePathToFolderContainingPage = Path.dirname(layoutStrategy.getLinkPathForPage(page));
+            context.pageInfo.slug = page.slug;
             if (page.type === NotionPage_1.PageType.DatabasePage &&
                 context.options.statusTag != "*" &&
                 page.status !== context.options.statusTag) {
@@ -127,10 +133,16 @@ function outputPages(options, config, pages) {
                 ++context.counts.skipped_because_status;
             }
             else {
+                if (options.requireSlugs && !page.hasExplicitSlug) {
+                    (0, log_1.error)(`Page "${page.nameOrTitle}" is missing a required slug. (--require-slugs is set.)`);
+                    ++counts.error_because_no_slug;
+                }
                 const markdown = yield (0, transform_1.getMarkdownForPage)(config, context, page);
                 writePage(page, markdown);
             }
         }
+        if (counts.error_because_no_slug > 0)
+            (0, process_1.exit)(1);
         (0, log_1.info)(`Finished processing ${pages.length} pages`);
         (0, log_1.info)(JSON.stringify(counts));
     });

package/dist/run.js

@@ -61,7 +61,11 @@ function run() {
             "debug",
         ]))
             .option("-i, --img-output-path  <string>", "Path to directory where images will be stored. If this is not included, images will be placed in the same directory as the document that uses them, which then allows for localization of screenshots.")
-            .option("-p, --img-prefix-in-markdown <string>", "When referencing an image from markdown, prefix with this path instead of the full img-output-path. Should be used only in conjunction with --img-output-path.");
+            .option("-p, --img-prefix-in-markdown <string>", "When referencing an image from markdown, prefix with this path instead of the full img-output-path. Should be used only in conjunction with --img-output-path.")
+            .option("--require-slugs", "If set, docu-notion will fail if any pages it would otherwise publish are missing a slug in Notion.", false)
+            .addOption(new commander_1.Option("--image-file-name-format <format>", "format:\n- default: {page slug (if any)}.{image block ID}\n- content-hash: Use a hash of the image content.\n- legacy: Use the legacy (before v0.16) method of determining file names. Set this to maintain backward compatibility.\nAll formats will use the original file extension.")
+            .choices(["default", "content-hash", "legacy"])
+            .default("default"));
         commander_1.program.showHelpAfterError();
         commander_1.program.parse();
         (0, log_1.setLogLevel)(commander_1.program.opts().logLevel);

package/package.json

@@ -17,7 +17,8 @@
     "pull-sample-site": "npm run ts -- -n $DOCU_NOTION_INTEGRATION_TOKEN -r $DOCU_NOTION_SAMPLE_ROOT_PAGE  --log-level debug",
     "// test with a semi-stable/public site:": "",
     "pull-sample": "npm run ts -- -n $DOCU_NOTION_INTEGRATION_TOKEN -r $DOCU_NOTION_SAMPLE_ROOT_PAGE -m ./sample --locales en,es,fr,de --log-level verbose",
-    "pull-sample-with-paths": "npm run ts -- -n $DOCU_NOTION_INTEGRATION_TOKEN -r $DOCU_NOTION_SAMPLE_ROOT_PAGE -m ./sample --img-output-path ./sample_img"
+    "pull-sample-with-paths": "npm run ts -- -n $DOCU_NOTION_INTEGRATION_TOKEN -r $DOCU_NOTION_SAMPLE_ROOT_PAGE -m ./sample --img-output-path ./sample_img",
+    "lint": "eslint . --ext .ts"
   },
   "//file-type": "have to use this version before they switched to ESM, which gives a compile error related to require()",
   "//chalk@4": "also ESM related problem",
@@ -90,5 +91,5 @@
   "volta": {
     "node": "18.16.0"
   },
-  "version": "0.14.0-alpha.13"
+  "version": "0.14.0-alpha.15"
 }