apostrophe 3.38.1 → 3.39.1

package/CHANGELOG.md

@@ -1,5 +1,27 @@
 # Changelog
 
+## 3.39.1 (2023-02-02)
+
+### Fixes
+
+* Rescaling cropped images with the `@apostrophecms/attachment:rescale` task now works correctly. Thanks to [Waldemar Pankratz](https://github.com/waldemar-p) for this contribution.
+
+## 3.39.0 (2023-02-01)
+
+### Adds
+
+* Basic support for editing tables by adding `table` to the rich text toolbar. Enabling `table` allows you to create tables, including `td` and `th` tags, with the ability to merge and split cells. For now the table editing UI is basic, all of the functionality is there but we plan to add more conveniences for easy table editing soon. See the "Table" dropdown for actions that are permitted based on the current selection.
+* `superscript` and `subscript` may now be added to the rich text widget's `toolbar` option.
+* Early beta-quality support for adding inline images to rich text, by adding `image` to the rich text toolbar. This feature works reliably, however the UI is not mature yet. In particular you must search for images by typing part of the title. We will support a proper "browse" experience here soon. For good results you should also configure the `imageStyles` option. You will also want to style the `figure` tags produced. See the documentation for more information.
+* Support for `div` tags in the rich text toolbar, if you choose to include them in `styles`. This is often necessary for A2 content migration and can potentially be useful in new work when combined with a `class` if there is no suitable semantic block tag.
+* The new `@apostrophecms/attachment:download-all --to=folder` command line task is useful to download all of your attachments from an uploadfs backend other than local storage, especially if you do not have a more powerful "sync" utility for that particular storage backend.
+* A new `loadingType` option can now be set for `image-widget` when configuring an `area` field. This sets the `loading` attribute of the `img` tag, which can be used to enable lazy loading in most browsers. Thanks to [Waldemar Pankratz](https://github.com/waldemar-p) for this contribution.
+* Two new module-level options have been added to the `image-widget` module: `loadingType` and `size`. These act as fallbacks for the same options at the area level. Thanks to [Waldemar Pankratz](https://github.com/waldemar-p) for this contribution.
+
+### Fixes
+
+* Adding missing require (`bluebird`) and fallback (`file.crops || []`) to `@apostrophecms/attachment:rescale`-task
+
 ## 3.38.1 (2023-01-23)
 
 ### Fixes
@@ -29,9 +51,12 @@ node app @apostrophecms/i18n:rename-locale --old=de-DE --new=de-de --keep=de-de
 * Emit a `beforeSaveSafe` event from the `@apostrophecms:user` module, with `req`, `safeUser` and `user` as arguments, in order to give the possibility to override properties of the `safeUser` object which contains password hashes and other information too sensitive to be stored in the aposDocs collection.
 * Automatically convert failed uppercase URLs to their lowercase version - can be disabled with `redirectFailedUpperCaseUrls: false` in `@apostrophecms/page/index.js` options. This only comes into play if a 404 is about to happen.
 * Automatically convert country codes in locales like `xx-yy` to `xx-YY` before passing them to `i18next`, which is strict about uppercase country codes.
+* Keyboard shortcuts conflicts are detected and logged on to the terminal.
 
 ### Fixes
 
+* Invalid locales passed to the i18n locale switching middleware are politely mapped to 400 errors.
+* Any other exceptions thrown in the i18n locale switching middleware can no longer crash the process.
 * Documents kept as the `previous` version for undo purposes were not properly marked as such, breaking the public language switcher in some cases. This was fixed and a migration was added for existing data.
 * Uploading an image in an apostrophe area with `minSize` requirements will not trigger an unexpected error anymore. If the image is too small, a notification will be displayed with the minimum size requirements. The `Edit Image` modal will now display the minimum size requirements, if any, above the `Browse Images` field.
 * Some browsers saw the empty `POST` response for new notifications as invalid XML. It will now return an empty JSON object with the `Content-Type` set to `application/json`.

package/modules/@apostrophecms/area/index.js

@@ -324,15 +324,20 @@ module.exports = {
       // to update a widget on the page after it is saved, or for
       // preview when editing.
       async renderWidget(req, type, data, options) {
-        const manager = self.getWidgetManager(type);
-        if (!manager) {
-          // No manager available - possibly a stale widget in the database
-          // of a type no longer in the project
-          self.warnMissingWidgetType(type);
-          return '';
+        try {
+          const manager = self.getWidgetManager(type);
+          if (!manager) {
+            // No manager available - possibly a stale widget in the database
+            // of a type no longer in the project
+            self.warnMissingWidgetType(type);
+            return '';
+          }
+          data.type = type;
+          return manager.output(req, data, options);
+        } catch (e) {
+          console.error(e);
+          throw e;
         }
-        data.type = type;
-        return manager.output(req, data, options);
       },
       // Update or create an area at the specified
       // dot path in the document with the specified

package/modules/@apostrophecms/asset/lib/globalIcons.js

@@ -56,6 +56,8 @@ module.exports = {
   'format-list-numbered-icon': 'FormatListNumbered',
   'format-quote-close-icon': 'FormatQuoteClose',
   'format-strikethrough-variant-icon': 'FormatStrikethroughVariant',
+  'format-superscript-icon': 'FormatSuperscript',
+  'format-subscript-icon': 'FormatSubscript',
   'format-underline-icon': 'FormatUnderline',
   'help-circle-icon': 'HelpCircle',
   'image-edit-outline': 'ImageEditOutline',

package/modules/@apostrophecms/attachment/index.js

@@ -112,6 +112,7 @@ module.exports = {
     self.sizeAvailableInArchive = self.options.sizeAvailableInArchive || 'one-sixth';
 
     self.rescaleTask = require('./lib/tasks/rescale.js')(self);
+    self.downloadAllTask = require('./lib/tasks/download-all.js')(self);
     self.addFieldType();
     self.enableBrowserData();
 
@@ -128,6 +129,10 @@ module.exports = {
         usage: 'Usage: node app @apostrophecms/attachment:rescale\n\nRegenerate all sizes of all image attachments. Useful after a new size\nis added to the configuration. Takes a long time!',
         task: self.rescaleTask
       },
+      'download-all': {
+        usage: 'Usage: node app @apostrophecms/attachment:download-all --to=public/uploads/attachments [--resume] [--parallel=3]\n\nDownload all attachments to a local folder, usually to sync\nfrom a non-local uploadfs backend. Takes a long time!',
+        task: self.downloadAllTask
+      },
       'migrate-to-disabled-file-key': {
         usage: 'Usage: node app @apostrophecms/attachment:migrate-to-disabled-file-key\n\nThis task should be run after adding the disabledFileKey option to uploadfs\nfor the first time. It should only be relevant for storage backends where\nthat option is not mandatory, i.e. only local storage as of this writing.',
         task: self.migrateToDisabledFileKeyTask

package/modules/@apostrophecms/attachment/lib/tasks/download-all.js

@@ -0,0 +1,77 @@
+// Direct use of console is appropriate in tasks. -Tom
+/* eslint-disable no-console */
+
+const fs = require('fs');
+const sep = require('path').sep;
+
+// Download everything Apostrophe believes to be in uploadfs.
+// Useful when uploadfs is not using a storage backend we
+// can conveniently sync from in some other way
+
+module.exports = function(self) {
+  return async function(argv) {
+    const copyOut = require('util').promisify(self.uploadfs.copyOut);
+    if (!argv.to) {
+      throw 'You must specify a --to=directory argument';
+    }
+    console.log(`Downloading all attachments to ${argv.to} (this takes time)`);
+    const files = fs.readdirSync(argv.to);
+    if ((files.length > 0) && (!argv.resume)) {
+      throw `The directory ${argv.to} is not empty and --resume not specified, exiting`;
+    }
+    if (!argv.to.endsWith(sep)) {
+      argv.to += sep;
+    }
+    const total = await self.db.count();
+    let n = 0;
+    const parallel = argv.parallel ? parseInt(argv.parallel) : 1;
+    await self.each({}, parallel, async function(file) {
+      const isImage = [ 'jpg', 'png', 'gif', 'webp' ].includes(file.extension);
+      const originalFile = filename(file);
+      n++;
+      console.log(n + ' of ' + total + ': ' + originalFile);
+      const files = [
+        originalFile
+      ];
+      if (isImage) {
+        files.push(...self.imageSizes.map(size => {
+          return filename(file, size);
+        }));
+        for (const crop of (file.crops || [])) {
+          files.push(filename(file, false, crop));
+          files.push(...self.imageSizes.map(size => filename(file, size, crop)));
+        }
+      }
+      for (const file of files) {
+        const to = argv.to + file;
+        const tmp = to + '.tmp';
+        if (fs.existsSync(to)) {
+          console.log(`${to} already exists, skipping`);
+        } else {
+          try {
+            console.log(`about to copy out: ${file}`);
+            await copyOut(`/attachments/${file}`, tmp);
+            fs.renameSync(tmp, argv.to + file);
+          } catch (e) {
+            if (e.code === 'ENOSPC') {
+              throw e;
+            } else {
+              console.log(`${e.code}: ${file} was probably never uploaded to uploadfs, skipping`);
+            }
+          }
+        }
+      }
+      function filename(file, size, crop) {
+        let name = file._id + '-' + file.name;
+        if (crop) {
+          name += '.' + crop.left + '.' + crop.top + '.' + crop.width + '.' + crop.height;
+        }
+        if (size) {
+          name += '.' + size.name;
+        }
+        name += '.' + file.extension;
+        return name;
+      }
+    });
+  };
+};

package/modules/@apostrophecms/attachment/lib/tasks/rescale.js

@@ -3,6 +3,7 @@
 
 const _ = require('lodash');
 const fs = require('fs');
+const Promise = require('bluebird');
 
 // Regenerate all scaled images. Useful after changing the configured sizes
 
@@ -61,12 +62,12 @@ module.exports = function(self) {
           return;
         }
       }
-      for (const crop of file.crops) {
+      for (const crop of file.crops || []) {
         console.log('RECROPPING');
         const originalFile = '/attachments/' + file._id + '-' + file.name + '.' + crop.left + '.' + crop.top + '.' + crop.width + '.' + crop.height + '.' + file.extension;
         console.log('Cropping ' + tempFile + ' to ' + originalFile);
         try {
-          Promise.promisify(self.uploadfs.copyImageIn)(tempFile, originalFile, {
+          await Promise.promisify(self.uploadfs.copyImageIn)(tempFile, originalFile, {
             crop: crop,
             sizes: self.imageSizes
           });

package/modules/@apostrophecms/command-menu/index.js

@@ -363,12 +363,58 @@ module.exports = {
         }
 
         const { groups, modals } = self.getVisible(req);
+        self.notifyConflicts(req, modals);
 
         return {
           components: { the: self.options.components.the || 'TheAposCommandMenu' },
           groups,
           modals
         };
+      },
+      notifyConflicts(req, modals = self.modals) {
+        const shortcuts = {
+          modal: {},
+          list: {},
+          conflict: {}
+        };
+
+        Object.entries(modals)
+          .forEach(([ modal, groups ]) => Object.values(groups)
+            .forEach(group => Object.entries(group.commands)
+              .forEach(([ name, field ]) => {
+                self.detectShortcutConflict({
+                  shortcuts,
+                  shortcut: field.shortcut.toUpperCase(),
+                  modal: modal === 'default' ? 'admin-bar' : modal,
+                  moduleName: name
+                });
+              })
+            )
+          );
+
+        self.apos.util.warnDev(
+          req.t('apostrophe:shortcutConflictNotification'),
+          shortcuts.conflict
+        );
+      },
+      detectShortcutConflict({
+        shortcuts, shortcut, modal, moduleName
+      }) {
+        shortcuts.modal[modal] = shortcuts.modal[modal] || [];
+        shortcuts.list[modal] = shortcuts.list[modal] || {};
+        shortcuts.list[modal][shortcut] = shortcuts.list[modal][shortcut] || [];
+
+        const existingShortcut = shortcuts.modal[modal].includes(shortcut);
+
+        if (existingShortcut) {
+          shortcuts.conflict[modal] = shortcuts.conflict[modal] || {};
+          shortcuts.conflict[modal][shortcut] = shortcuts.list[modal][shortcut];
+        } else {
+          shortcuts.modal[modal].push(shortcut);
+        }
+        shortcuts.list[modal][shortcut].push(moduleName);
+
+        return shortcuts;
       }
     };
   }

package/modules/@apostrophecms/i18n/i18n/en.json

@@ -1,6 +1,10 @@
 {
+  "addColumnBefore": "Add Column Before",
+  "addColumnAfter": "Add Column After",
   "addContent": "Add Content",
   "addItem": "Add Item",
+  "addRowBefore": "Add Row Before",
+  "addRowAfter": "Add Row After",
   "addWidgetType": "Add {{ label }}",
   "admin": "Admin",
   "affirmativeLabel": "Yes, continue.",
@@ -45,6 +49,7 @@
   "browseDocType": "Browse {{ type }}",
   "cancel": "Cancel",
   "cannotMoveArchive": "You cannot move the Archive",
+  "caption": "Caption",
   "changed": "Changed",
   "changesAwaitingApproval": "Changes to this document are awaiting approval by an admin or editor.",
   "changesDiscarded": "Changes discarded",
@@ -93,9 +98,12 @@
   "dayjsMediaCreatedDateFormat": "MMM Do, YYYY",
   "deduplicateSlugReserved": "The deduplicate- slug is reserved.",
   "delete": "Delete",
+  "deleteColumn": "Delete Column",
   "deleteDraft": "Delete Draft",
   "deleteDraftAffirmativeLabel": "Yes, delete document",
   "deleteDraftDescription": "Since {{ title }} has never been published, this will completely delete the document.",
+  "deleteRow": "Delete Row",
+  "deleteTable": "Delete Table",
   "description": "Description",
   "disabled": "Disabled",
   "discardChanges": "Discard Changes",
@@ -188,6 +196,7 @@
   "insertAndRedirect": "{{ saveLabel }} {{ typeLabel }} and be redirected to the {{ typeLabel }}.",
   "insertAndReturn": "{{ saveLabel }} and return to the {{ typeLabel }} listing.",
   "insertAndNew": "{{ saveLabel }} {{ typeLabel }} and create a new one.",
+  "insertTable": "Insert Table",
   "lastEdited": "Last Edited",
   "leavePageDescription": "The content you're trying to edit belongs to another document and must be edited there.\nChanges made to {{ oldTitle }} are saved automatically.",
   "leavePageHeading": "Leave {{ oldTitle }} to edit {{ newTitle }}?",
@@ -241,6 +250,7 @@
   "mediaMB": "{{ size }}MB",
   "mediaUploadViaDrop": "Drop ’em when you’re ready",
   "mediaUploadViaExplorer": "Or click to open the file explorer",
+  "mergeCells": "Merge Cells",
   "minLabel": "Min:",
   "minUi": "Min: {{ number }}",
   "modify": "Modify",
@@ -403,6 +413,14 @@
   "shareDraftEnable": "Enable draft sharing",
   "shareDraftHeader": "Share this page",
   "shareDraftError": "This document cannot be shared at this time",
+  "shortcutConflictNotification": "Shortcut conflicts detected:",
+  "subscript": "Subscript",
+  "superscript": "Superscript",
+  "splitCell": "Split Cell",
+  "style": "Style",
+  "toggleHeaderCell": "Toggle Header (Cell)",
+  "toggleHeaderColumn": "Toggle Header (Column)",
+  "toggleHeaderRow": "Toggle Header (Row)",
   "visibilityHelp": "Select whether this content is public or private",
   "slug": "Slug",
   "slugInUse": "Slug already in use",
@@ -414,6 +432,7 @@
   "submitUpdate": "Submit Update",
   "suggestionsHeader": "Try one of these suggestions:",
   "switchLocalesAndLocalizePage": "Switch locales and localize page to {{ label }}?",
+  "table": "Table",
   "tags": "Tags",
   "tagYourImages": "Tag your images to make searching and filtering the media manager easier",
   "takeActionAndCreateNew": "{{ saveLabel }} and Create New",

package/modules/@apostrophecms/i18n/index.js

@@ -287,6 +287,9 @@ module.exports = {
       post: {
         async locale(req) {
           const sanitizedLocale = self.sanitizeLocaleName(req.body.locale);
+          if (!sanitizedLocale) {
+            throw self.apos.error('invalid', 'invalid locale');
+          }
           // Clipboards transferring between locales needs to jump
           // from LocalStorage to the cross-domain session cache
           let clipboard = req.body.clipboard;
@@ -626,21 +629,29 @@ module.exports = {
       // if possible, to the corresponding version in toLocale.
       toLocaleRouteFactory(module) {
         return async (req, res) => {
-          const _id = module.inferIdLocaleAndMode(req, req.params._id);
-          const toLocale = req.params.toLocale;
-          const localeReq = req.clone({
-            locale: toLocale
-          });
-          const corresponding = await module.find(localeReq, {
-            _id: `${_id.split(':')[0]}:${localeReq.locale}:${localeReq.mode}`
-          }).toObject();
-          if (!corresponding) {
-            return res.status(404).send('not found');
-          }
-          if (!corresponding._url) {
-            return res.status(400).send('invalid (has no URL)');
+          try {
+            const _id = module.inferIdLocaleAndMode(req, req.params._id);
+            const toLocale = self.sanitizeLocaleName(req.params.toLocale);
+            if (!toLocale) {
+              return res.status(400).send('invalid locale name');
+            }
+            const localeReq = req.clone({
+              locale: toLocale
+            });
+            const corresponding = await module.find(localeReq, {
+              _id: `${_id.split(':')[0]}:${localeReq.locale}:${localeReq.mode}`
+            }).toObject();
+            if (!corresponding) {
+              return res.status(404).send('not found');
+            }
+            if (!corresponding._url) {
+              return res.status(400).send('invalid (has no URL)');
+            }
+            return res.redirect(corresponding._url);
+          } catch (e) {
+            self.apos.util.error(e);
+            return res.status(500).send('error');
           }
-          return res.redirect(corresponding._url);
         };
       },
       // Exclude private locales when logged out

package/modules/@apostrophecms/image/index.js

@@ -290,6 +290,34 @@ module.exports = {
       }
     }
   }),
+  routes(self, options) {
+    return {
+      get: {
+        // Convenience route to get the URL of the image
+        // knowing only the image id. Useful in the rich text editor.
+        // Not performant for frontend use
+        ':imageId/src': async (req, res) => {
+          const size = req.query.size || self.getLargestSize();
+          try {
+            const image = await self.find(req, {
+              aposDocId: req.params.imageId
+            }).toObject();
+            if (!image) {
+              return res.status(404).send('notfound');
+            }
+            const url = image.attachment && image.attachment._urls && image.attachment._urls[size];
+            if (url) {
+              return res.redirect(image.attachment._urls[size]);
+            }
+            return res.status(404).send('notfound');
+          } catch (e) {
+            self.apos.util.error(e);
+            return res.status(500).send('error');
+          }
+        }
+      }
+    };
+  },
   methods(self) {
     return {
       // This method is available as a template helper: apos.image.first
@@ -417,6 +445,14 @@ module.exports = {
           self.getComponentName('managerModal', 'AposMediaManager'),
           { moduleName: self.__meta.name }
         );
+      },
+      getLargestSize() {
+        return self.apos.attachment.imageSizes.reduce((a, size) => {
+          return size.width > a.width ? size : a;
+        }, {
+          name: 'dummy',
+          width: 0
+        }).name;
       }
     };
   },

package/modules/@apostrophecms/image-widget/views/widget.html

@@ -5,24 +5,18 @@
     class="image-widget-placeholder"
   />
 {% else %}
-  {% if data.options.className %}
-    {% set className = data.options.className %}
-  {% elif data.manager.options.className %}
-    {% set className = data.manager.options.className %}
-  {% endif %}
-
-  {% if data.options.dimensionAttrs %}
-    {% set dimensionAttrs = data.options.dimensionAttrs %}
-  {% elif data.manager.options.dimensionAttrs %}
-    {% set dimensionAttrs = data.manager.options.dimensionAttrs  %}
-  {% endif %}
+  {% set className = data.options.className or data.manager.options.className %}
+  {% set dimensionAttrs = data.options.dimensionAttrs or data.manager.options.dimensionAttrs %}
+  {% set loadingType = data.options.loadingType or data.manager.options.loadingType %}
+  {% set size = data.options.size or data.manager.options.size or 'full' %}
 
   {% set attachment = apos.image.first(data.widget._image) %}
 
   {% if attachment %}
     <img {% if className %} class="{{ className }}"{% endif %}
+      {% if loadingType %} loading="{{ loadingType }}"{% endif %}
       srcset="{{ apos.image.srcset(attachment) }}"
-      src="{{ apos.attachment.url(attachment, { size: data.options.size or 'full' }) }}"
+      src="{{ apos.attachment.url(attachment, { size: size }) }}"
       alt="{{ attachment._alt or '' }}"
       {% if dimensionAttrs %}
         {% if attachment.width %} width="{{ apos.attachment.getWidth(attachment) }}" {% endif %}