apostrophe 3.9.0 → 3.12.0

package/CHANGELOG.md

@@ -1,5 +1,67 @@
 # Changelog
 
+## 3.12.0 - 2022-01-21
+
+### Adds
+
+* It is now best practice to deliver namespaced i18n strings as JSON files in module-level subdirectories of `i18n/` named to match the namespace, e.g. `i18n/ourTeam` if the namespace is `ourTeam`. This allows base class modules to deliver phrases to any namespace without conflicting with those introduced at project level. The `i18n` option is now deprecated in favor of the new `i18n` module format section, which is only needed if `browser: true` must be specified for a namespace.
+* Brought back the `nestedModuleSubdirs` feature from A2, which allows modules to be nested in subdirectories if `nestedModuleSubdirs: true` is set in `app.js`. As in A2, module configuration (including activation) can also be grouped in a `modules.js` file in such subdirectories.
+
+### Fixes
+
+* Fixes minor inline documentation comments.
+* UI strings that are not registered localization keys will now display properly when they contain a colon (`:`). These were previously interpreted as i18next namespace/key pairs and the "namespace" portion was left out.
+* Fixes a bug where changing the page type immediately after clicking "New Page" would produce a console error. In general, areas and checkboxes now correctly handle their value being changed to `null` by the parent schema after initial startup of the `AposInputArea` or `AposInputCheckboxes` component.
+* It is now best practice to deliver namespaced i18n strings as JSON files in module-level subdirectories of `i18n/` named to match the namespace, e.g. `i18n/ourTeam` if the namespace is `ourTeam`. This allows base class modules to deliver phrases to any namespace without conflicting with those introduced at project level. The `i18n` option is now deprecated in favor of the new `i18n` module format section, which is only needed if `browser: true` must be specified for a namespace.
+* Removes the `@apostrophecms/util` module template helper `indexBy`, which was using a lodash method not included in lodash v4.
+* Removes an unimplemented `csrfExceptions` module section cascade. Use the `csrfExceptions` *option* of any module to set an array of URLs excluded from CSRF protection. More information is forthcoming in the documentation.
+* Fix `[Object Object]` in the console when warning `A permission.can() call was made with a type that has no manager` is printed.
+
+### Changes
+
+* Temporarily removes `npm audit` from our automated tests because of a sub-dependency of vue-loader that doesn't actually cause a security vulnerability for apostrophe.
+
+## 3.11.0 - 2022-01-06
+
+### Adds
+
+* Apostrophe now extends Passport's `req.login` to emit an `afterSessionLogin` event from the `@apostrophecms:login` module, with `req` as an argument. Note that this does not occur at all for login API calls that return a bearer token rather than establishing an Express session.
+
+### Fixes
+
+* Apostrophe's extension of `req.login` now accounts for the `req.logIn` alias and the skippable `options` parameter, which is relied upon in some `passport` strategies.
+* Apostrophe now warns if a nonexistent widget type is configured for an area field, with special attention to when `-widget` has been erroneously included in the name. For backwards compatibility this is a startup warning rather than a fatal error, as sites generally did operate successfully otherwise with this type of bug present.
+
+### Changes
+
+* Unpins `vue-click-outside-element` the packaging of which has been fixed upstream.
+* Adds deprecation note to `__testDefaults` option. It is not in use, but removing would be a minor BC break we don't need to make.
+* Allows test modules to use a custom port as an option on the `@apostrophecms/express` module.
+* Removes the code base pull request template to instead inherit the organization-level template.
+* Adds `npm audit` back to the test scripts.
+
+## 3.10.0 - 2021-12-22
+
+### Fixes
+
+* `slug` type fields can now have an empty string or `null` as their `def` value without the string `'none'` populating automatically.
+* The `underline` feature works properly in tiptap toolbar configuration.
+* Required checkbox fields now properly prevent editor submission when empty.
+* Pins `vue-click-outside-element` to a version that does not attempt to use `eval` in its distribution build, which is incompatible with a strict Content Security Policy.
+
+### Adds
+
+* Adds a `last` option to fields. Setting `last: true` on a field puts that field at the end of the field's widget order. If more than one field has that option active the true last item will depend on general field registration order. If the field is ordered with the `fields.order` array or field group ordering, those specified orders will take precedence.
+
+### Changes
+
+* Adds deprecation notes to the widget class methods `getWidgetWrapperClasses` and `getWidgetClasses` from A2.
+* Adds a deprecation note to the `reorganize` query builder for the next major version.
+* Uses the runtime build of Vue. This has major performance and bundle size benefits, however it does require changes to Apostrophe admin UI apps that use a `template` property (components should require no changes, just apps require an update). These apps must now use a `render` function instead. Since custom admin UI apps are not yet a documented feature we do not regard this as a bc break.
+* Compatible with the `@apostrophecms/security-headers` module, which supports a strict `Content-Security-Policy`.
+* Adds a deprecation note to the `addLateCriteria` query builder.
+* Updates the `toCount` doc type query method to use Math.ceil rather than Math.floor plus an additional step.
+
 ## 3.9.0 - 2021-12-08
 
 ### Adds

package/index.js

@@ -7,6 +7,7 @@ const cluster = require('cluster');
 const { cpus } = require('os');
 const process = require('process');
 const npmResolve = require('resolve');
+const glob = require('glob');
 
 let defaults = require('./defaults.js');
 
@@ -273,6 +274,33 @@ module.exports = async function(options) {
     return _module;
   }
 
+  function nestedModuleSubdirs() {
+    if (!options.nestedModuleSubdirs) {
+      return;
+    }
+    const configs = glob.sync(self.localModules + '/**/modules.js');
+    _.each(configs, function(config) {
+      try {
+        _.merge(self.options.modules, require(config));
+      } catch (e) {
+        console.error(stripIndent`
+          When nestedModuleSubdirs is active, any modules.js file beneath:
+
+          ${self.localModules}
+
+          must export an object containing configuration for Apostrophe modules.
+          
+          The file:
+          
+          ${config}
+          
+          did not parse.
+        `);
+        throw e;
+      }
+    });
+  }
+
   function autodetectBundles() {
     const modules = _.keys(self.options.modules);
     _.each(modules, function(name) {
@@ -406,7 +434,13 @@ module.exports = async function(options) {
       localModules: self.localModules,
       defaultBaseClass: '@apostrophecms/module',
       sections: [ 'helpers', 'handlers', 'routes', 'apiRoutes', 'restApiRoutes', 'renderRoutes', 'middleware', 'customTags', 'components', 'tasks' ],
-      unparsedSections: [ 'queries', 'extendQueries', 'icons' ]
+      nestedModuleSubdirs: self.options.nestedModuleSubdirs,
+      unparsedSections: [
+        'queries',
+        'extendQueries',
+        'icons',
+        'i18n'
+      ]
     });
 
     self.synth = synth;
@@ -417,6 +451,8 @@ module.exports = async function(options) {
     self.redefine = self.synth.redefine;
     self.create = self.synth.create;
 
+    nestedModuleSubdirs();
+
     _.each(self.options.modules, function(options, name) {
       synth.define(name, options);
     });

package/lib/moog.js

@@ -179,17 +179,42 @@ module.exports = function(options) {
               ...properties.add
             };
           }
+
+          const lastFields = Object.entries(that[cascade])
+            .filter(([ field, val ]) => val.last === true)
+            .map(([ field, val ]) => field);
+
           if (properties.remove) {
             for (const field of properties.remove) {
               delete that[cascade][field];
             }
           }
           if (properties.order) {
+            // 1. Ordered fields
+            // 2. Other fields not marked as last
+            // 3. Fields marked as last
             that[cascade] = Object.fromEntries([
               ...properties.order.map(field => [ field, that[cascade][field] ]),
-              ...Object.keys(that[cascade]).filter(field => !properties.order.includes[field]).map(field => [ field, that[cascade][field] ])
+              ...Object.keys(that[cascade])
+                .filter(field => {
+                  return !properties.order.includes(field) &&
+                    !lastFields.includes(field);
+                })
+                .map(field => [ field, that[cascade][field] ]),
+              ...lastFields.filter(field => !properties.order.includes(field))
+                .map(field => [ field, that[cascade][field] ])
+            ]);
+          } else if (lastFields.length > 0) {
+            // 1. Fields not marked as last
+            // 2. Fields marked as last
+            that[cascade] = Object.fromEntries([
+              ...Object.keys(that[cascade])
+                .filter(field => !lastFields.includes(field))
+                .map(field => [ field, that[cascade][field] ]),
+              ...lastFields.map(field => [ field, that[cascade][field] ])
             ]);
           }
+
           if (properties.group) {
             const groups = klona(that[`${cascade}Groups`]);
             for (const value of Object.values(properties.group)) {
@@ -344,9 +369,9 @@ module.exports = function(options) {
     }
 
     function capture(section) {
-      that[section] = {};
+      that.__meta[section] = {};
       for (const step of steps) {
-        that[section][step.__meta.name] = step[section];
+        that.__meta[section][step.__meta.name] = step[section];
       }
     }
 

package/modules/@apostrophecms/admin-bar/ui/apos/apps/AposAdminBar.js

@@ -10,7 +10,13 @@ export default function() {
           return window.apos;
         }
       },
-      template: '<component :is="`TheAposAdminBar`" :items="apos.adminBar.items" />'
+      render: function (h) {
+        return h('TheAposAdminBar', {
+          props: {
+            items: apos.adminBar.items
+          }
+        });
+      }
     });
   }
 };

package/modules/@apostrophecms/any-page-type/index.js

@@ -287,6 +287,8 @@ module.exports = {
         // are suitable for display in the reorganize view.
         // The only pages excluded are those with a `reorganize`
         // property explicitly set to `false`.
+        // NOTE: This query builder is deprecated and will be removed in the
+        // next major version.
         reorganize: {
           def: null,
           finalize() {

package/modules/@apostrophecms/area/ui/apos/apps/AposAreas.js

@@ -111,7 +111,19 @@ export default function() {
             renderings
           };
         },
-        template: `<${component} :options="options" :items="items" :choices="choices" :id="$data.id" :docId="$data.docId" :fieldId="fieldId" :renderings="renderings" />`
+        render(h) {
+          return h(component, {
+            props: {
+              options: this.options,
+              items: this.items,
+              choices: this.choices,
+              id: this.id,
+              docId: this.docId,
+              fieldId: this.fieldId,
+              renderings: this.renderings
+            }
+          });
+        }
       });
     }
   }

package/modules/@apostrophecms/asset/index.js

@@ -752,8 +752,7 @@ module.exports = {
         if (!self.shouldRefreshOnRestart()) {
           return '';
         }
-        const script = fs.readFileSync(path.join(__dirname, '/lib/refresh-on-restart.js'), 'utf8');
-        return self.apos.template.safe(`<script data-apos-refresh-on-restart="${self.action}/restart-id">\n${script}</script>`);
+        return self.apos.template.safe(`<script data-apos-refresh-on-restart="${self.action}/restart-id" src="${self.action}/refresh-on-restart"></script>`);
       },
       url(path) {
         return `${self.getAssetBaseUrl()}${path}`;
@@ -765,6 +764,12 @@ module.exports = {
       return;
     }
     return {
+      get: {
+        refreshOnRestart(req) {
+          req.res.setHeader('content-type', 'text/javascript');
+          return fs.readFileSync(path.join(__dirname, '/lib/refresh-on-restart.js'), 'utf8');
+        }
+      },
       // Use a POST route so IE11 doesn't cache it
       post: {
         async restartId(req) {

package/modules/@apostrophecms/asset/lib/webpack/apos/webpack.config.js

@@ -28,7 +28,7 @@ module.exports = ({
     optimization: {
       minimize: process.env.NODE_ENV === 'production'
     },
-    devtool: 'eval-source-map',
+    devtool: 'source-map',
     output: {
       path: outputPath,
       filename: outputFilename
@@ -42,7 +42,7 @@ module.exports = ({
     resolve: {
       extensions: [ '*', '.js', '.vue', '.json' ],
       alias: {
-        vue$: 'vue/dist/vue.esm.js',
+        vue$: 'vue/dist/vue.runtime.esm.js',
         // resolve apostrophe modules
         Modules: path.resolve(modulesDir)
       },

package/modules/@apostrophecms/asset/lib/webpack/src/webpack.config.js

@@ -34,7 +34,7 @@ module.exports = ({
     optimization: {
       minimize: process.env.NODE_ENV === 'production'
     },
-    devtool: 'eval-source-map',
+    devtool: 'source-map',
     output: {
       path: outputPath,
       filename: outputFilename

package/modules/@apostrophecms/busy/ui/apos/apps/AposBusy.js

@@ -3,6 +3,8 @@ import Vue from 'Modules/@apostrophecms/ui/lib/vue';
 export default function() {
   return new Vue({
     el: '#apos-busy',
-    template: '<component :is="`TheAposBusy`" />'
+    render: function (h) {
+      return h('TheAposBusy');
+    }
   });
 };

package/modules/@apostrophecms/doc/index.js

@@ -675,10 +675,10 @@ module.exports = {
       },
 
       // Insert the given document. Called by `.insert()`. You will usually want to
-      // call the update method of the appropriate doc type manager instead:
+      // call the insert method of the appropriate doc type manager instead:
       //
       // ```javascript
-      // self.apos.doc.getManager(doc.type).update(...)
+      // self.apos.doc.getManager(doc.type).insert(...)
       // ```
       //
       // However you can override this method to alter the

package/modules/@apostrophecms/doc-type/index.js

@@ -1152,8 +1152,10 @@ module.exports = {
 
         // `.addLateCriteria({...})` provides an object to be merged directly into the final
         // criteria object that will go to MongoDB. This is to be used only
-        // in cases where MongoDB forbids the use of an operator inside
-        // `$and`, such as the `$near` operator.
+        // in cases where MongoDB forbids the use of an operator inside `$and`.
+        //
+        // TODO: Since `$near` can now be used in `$and` operators, this query
+        // builder is deprecated and should be removed in the 4.x major version.
         addLateCriteria: {
           set(c) {
             let lateCriteria = query.get('lateCriteria');
@@ -1420,7 +1422,7 @@ module.exports = {
           }
         },
 
-        // `.permission('admin')` would limit the returned docs to those for which the
+        // `.permission('edit')` would limit the returned docs to those for which the
         // user associated with the query's `req` has the named permission.
         // By default, `view` is checked for. You might want to specify
         // `edit`.
@@ -1658,7 +1660,11 @@ module.exports = {
             const _req = query.req.clone({
               mode: 'published'
             });
-            const publishedDocs = await self.find(_req)._ids(results.map(result => result._id.replace(':draft', ':published'))).project(query.get('project')).toArray();
+            const publishedDocs = await self.find(_req)
+              ._ids(results.map(result => {
+                return result._id.replace(':draft', ':published');
+              })).project(query.get('project')).toArray();
+
             for (const doc of results) {
               const publishedDoc = publishedDocs.find(publishedDoc => doc.aposDocId === publishedDoc.aposDocId);
               doc._publishedDoc = publishedDoc;
@@ -2148,10 +2154,8 @@ module.exports = {
           const count = await mongo.count();
           if (query.get('perPage')) {
             const perPage = query.get('perPage');
-            let totalPages = Math.floor(count / perPage);
-            if (count % perPage) {
-              totalPages++;
-            }
+            const totalPages = Math.ceil(count / perPage);
+
             query.set('totalPages', totalPages);
           }
           return count;

package/modules/@apostrophecms/i18n/index.js

@@ -10,6 +10,7 @@ const fs = require('fs');
 const _ = require('lodash');
 const { stripIndent } = require('common-tags');
 const ExpressSessionCookie = require('express-session/session/cookie');
+const path = require('path');
 
 const apostropheI18nDebugPlugin = {
   type: 'postProcessor',
@@ -380,21 +381,65 @@ module.exports = {
       // Add the i18next resources provided by the specified module,
       // merging with any existing phrases for the same locales and namespaces
       addResourcesForModule(module) {
-        if (!module.options.i18n) {
-          return;
-        }
-        const ns = module.options.i18n.ns || 'default';
+        self.addDefaultResourcesForModule(module);
+        self.addNamespacedResourcesForModule(module);
+      },
+      // Automatically adds any localizations found in .json files in the main `i18n` subdirectory
+      // of a module.
+      //
+      // These are added to the `default` namespace, unless the legacy `i18n.ns` option is set
+      // for the module (not the preferred way, use namespace subdirectories in new projects).
+      addDefaultResourcesForModule(module) {
+        const ns = (module.options.i18n && module.options.i18n.ns) || 'default';
         self.namespaces[ns] = self.namespaces[ns] || {};
-        self.namespaces[ns].browser = self.namespaces[ns].browser || !!module.options.i18n.browser;
+        self.namespaces[ns].browser = self.namespaces[ns].browser || (module.options.i18n && module.options.i18n.browser);
         for (const entry of module.__meta.chain) {
-          const localizationsDir = `${entry.dirname}/i18n`;
-          if (!fs.existsSync(localizationsDir)) {
-            continue;
+          const localizationsDir = path.join(entry.dirname, 'i18n');
+          if (!self.defaultLocalizationsDirsAdded.has(localizationsDir)) {
+            self.defaultLocalizationsDirsAdded.add(localizationsDir);
+            if (!fs.existsSync(localizationsDir)) {
+              continue;
+            }
+            for (const localizationFile of fs.readdirSync(localizationsDir)) {
+              if (!localizationFile.endsWith('.json')) {
+                // Likely a namespace subdirectory
+                continue;
+              }
+              const data = JSON.parse(fs.readFileSync(path.join(localizationsDir, localizationFile)));
+              const locale = localizationFile.replace('.json', '');
+              self.i18next.addResourceBundle(locale, ns, data, true, true);
+            }
           }
-          for (const localizationFile of fs.readdirSync(localizationsDir)) {
-            const data = JSON.parse(fs.readFileSync(`${localizationsDir}/${localizationFile}`));
-            const locale = localizationFile.replace('.json', '');
-            self.i18next.addResourceBundle(locale, ns, data, true, true);
+        }
+      },
+      // Automatically adds any localizations found in subdirectories of the main `i18n`
+      // subdirectory of a module. The subdirectory's name is treated as an i18n namespace
+      // name.
+      addNamespacedResourcesForModule(module) {
+        for (const entry of module.__meta.chain) {
+          const metadata = module.__meta.i18n[entry.name] || {};
+          const localizationsDir = `${entry.dirname}/i18n`;
+          if (!self.namespacedLocalizationsDirsAdded.has(localizationsDir)) {
+            self.namespacedLocalizationsDirsAdded.add(localizationsDir);
+            if (!fs.existsSync(localizationsDir)) {
+              continue;
+            }
+            for (const ns of fs.readdirSync(localizationsDir)) {
+              if (ns.endsWith('.json')) {
+                // A JSON file for the default namespace, already handled
+                continue;
+              }
+              self.namespaces[ns] = self.namespaces[ns] || {};
+              self.namespaces[ns].browser = self.namespaces[ns].browser ||
+                (metadata[ns] && metadata[ns].browser);
+              const namespaceDir = path.join(localizationsDir, ns);
+              for (const localizationFile of fs.readdirSync(namespaceDir)) {
+                const fullLocalizationFile = path.join(namespaceDir, localizationFile);
+                const data = JSON.parse(fs.readFileSync(fullLocalizationFile));
+                const locale = localizationFile.replace('.json', '');
+                self.i18next.addResourceBundle(locale, ns, data, true, true);
+              }
+            }
           }
         }
       },
@@ -402,6 +447,8 @@ module.exports = {
       // itself, called by init. Later modules call addResourcesForModule(self),
       // making phrases available gradually as Apostrophe starts up
       addInitialResources() {
+        self.defaultLocalizationsDirsAdded = new Set();
+        self.namespacedLocalizationsDirsAdded = new Set();
         for (const module of Object.values(self.apos.modules)) {
           self.addResourcesForModule(module);
         }

package/modules/@apostrophecms/login/index.js

@@ -37,13 +37,6 @@
 //
 // Apostrophe's instance of the [passport](https://npmjs.org/package/passport) npm module.
 // You may access this object if you need to implement additional passport "strategies."
-//
-// ## callAll method: loginAfterLogin
-//
-// The method `loginAfterLogin` is invoked on **all modules that have one**. This method
-// is a good place to set `req.redirect` to the URL of your choice. If no module sets
-// `req.redirect`, the newly logged-in user is redirected to the home page. `loginAfterLogin`
-// is invoked with `req` and may be an async function.
 
 const Passport = require('passport').Passport;
 const LocalStrategy = require('passport-local');
@@ -405,15 +398,35 @@ module.exports = {
         before: '@apostrophecms/i18n',
         middleware(req, res, next) {
           const superLogin = req.login.bind(req);
-          req.login = (user, callback) => {
-            return superLogin(user, (err) => {
+          req.login = (user, ...args) => {
+            let options, callback;
+            // Support inconsistent calling conventions inside passport core
+            if (typeof args[0] === 'function') {
+              options = {};
+              callback = args[0];
+            } else {
+              options = args[0];
+              callback = args[1];
+            }
+            return superLogin(user, options, async (err) => {
               if (err) {
                 return callback(err);
               }
-              req.session.loginAt = Date.now();
+              await self.emit('afterSessionLogin', req);
+              // Make sure no handler removed req.user
+              if (req.user) {
+                // Mark the login timestamp. Middleware takes care of ensuring
+                // that logins cannot be used to carry out actions prior
+                // to this property being added
+                req.session.loginAt = Date.now();
+              }
               return callback(null);
             });
           };
+          // Passport itself maintains this bc alias, while refusing
+          // to actually decide which one is best in its own dev docs.
+          // Both have to exist to avoid bugs when passport calls itself
+          req.logIn = req.login;
           return next();
         }
       },

package/modules/@apostrophecms/login/ui/apos/apps/AposLogin.js

@@ -5,8 +5,9 @@ export default function() {
   if (el) {
     return new Vue({
       el,
-      // TODO check apos.login.browser.components.theAposLogin for alternate name
-      template: '<component :is="`TheAposLogin`" />'
+      render: function (h) {
+        return h('TheAposLogin');
+      }
     });
   }
   apos.bus.$on('admin-menu-click', async (item) => {

package/modules/@apostrophecms/modal/ui/apos/apps/AposModals.js

@@ -27,11 +27,14 @@ export default function() {
         return this.$refs.modals.execute(componentName, props);
       }
     },
-    template: `<component
-      ref="modals"
-      :is="apos.modal.components.the"
-      :modals="apos.modal.modals"
-    />`
+    render(h) {
+      return h(apos.modal.components.the, {
+        ref: 'modals',
+        props: {
+          modals: apos.modal.modals
+        }
+      });
+    }
   });
   apos.modal.execute = theAposModals.execute;
   apos.confirm = theAposModals.confirm;

package/modules/@apostrophecms/module/index.js

@@ -29,10 +29,20 @@ const _ = require('lodash');
 
 module.exports = {
 
-  cascades: [ 'csrfExceptions' ],
-
   init(self) {
     self.apos = self.options.apos;
+    const capturedSections = [
+      'queries',
+      'extendQueries',
+      'icons'
+    ];
+    for (const section of capturedSections) {
+      // Unparsed sections are now captured in __meta, promote
+      // these to the top level to maintain bc. For new unparsed
+      // sections we'll leave them in `__meta` to avoid bc breaks
+      // with project-level properties of the module
+      self[section] = self.__meta[section];
+    }
     // all apostrophe modules are properties of self.apos.modules.
     // Those with an alias are also properties of self.apos
     self.apos.modules[self.__meta.name] = self;

package/modules/@apostrophecms/notification/ui/apos/apps/AposNotification.js

@@ -9,6 +9,8 @@ export default function() {
 
   return new Vue({
     el: '#apos-notification',
-    template: '<TheAposNotifications />'
+    render: function (h) {
+      return h('TheAposNotifications');
+    }
   });
 };

package/modules/@apostrophecms/page/index.js

@@ -1977,14 +1977,13 @@ database.`);
         );
       },
       // Returns the effective base URL for the given request.
-      // If Apostrophe's top-level `baseUrl` option is set, it is returned,
-      // otherwise the empty string. This makes it easier to build absolute
+      // If Apostrophe's top-level `baseUrl` option is set, or a hostname is
+      // defined for the active locale, then that is consulted, otherwise the base URL
+      // is the empty string. This makes it easier to build absolute
       // URLs (when `baseUrl` is configured), or to harmlessly prepend
       // the empty string (when it is not configured). The
       // Apostrophe queries used to fetch Apostrophe pages
-      // consult this method, and it is extended by the optional
-      // `@apostrophecms/workflow` module to create correct absolute URLs
-      // for specific locales.
+      // consult this method.
       getBaseUrl(req) {
         const hostname = self.apos.i18n.locales[req.locale].hostname;
         if (hostname) {

package/modules/@apostrophecms/permission/index.js

@@ -63,7 +63,7 @@ module.exports = {
         const doc = (docOrType && docOrType._id) ? docOrType : null;
         const manager = type && self.apos.doc.getManager(type);
         if (type && !manager) {
-          self.apos.util.warn(`A permission.can() call was made with a type that has no manager: ${type}`);
+          self.apos.util.warn('A permission.can() call was made with a type that has no manager:', type);
           return false;
         }
         if (action === 'view') {

package/modules/@apostrophecms/piece-type/index.js

@@ -566,7 +566,7 @@ module.exports = {
       },
       //
       // Update a piece. Convenience wrapper for `apos.doc.insert`.
-      // Returns the piece. `beforeInsert`, `beforeSave`, `afterInsert`
+      // Returns the piece. `beforeUpdate`, `beforeSave`, `afterUpdate`
       // and `afterSave` async events are emitted by this module.
       async update(req, piece, options) {
         return self.apos.doc.update(req, piece, options);

package/modules/@apostrophecms/rich-text-widget/index.js

@@ -250,7 +250,8 @@ module.exports = {
           codeBlock: [
             'pre',
             'code'
-          ]
+          ],
+          underline: [ 'u' ]
         };
         for (const item of options.toolbar || []) {
           if (simple[item]) {

package/modules/@apostrophecms/rich-text-widget/ui/apos/components/AposRichTextWidgetEditor.vue

@@ -44,6 +44,7 @@ import StarterKit from '@tiptap/starter-kit';
 import TextAlign from '@tiptap/extension-text-align';
 import Highlight from '@tiptap/extension-highlight';
 import TextStyle from '@tiptap/extension-text-style';
+import Underline from '@tiptap/extension-underline';
 export default {
   name: 'AposRichTextWidgetEditor',
   components: {
@@ -181,7 +182,8 @@ export default {
           types: [ 'heading', 'paragraph' ]
         }),
         Highlight,
-        TextStyle
+        TextStyle,
+        Underline
       ].concat(this.aposTiptapExtensions)
     });
   },

package/modules/@apostrophecms/schema/index.js

@@ -18,7 +18,7 @@ const _ = require('lodash');
 const dayjs = require('dayjs');
 const tinycolor = require('tinycolor2');
 const { klona } = require('klona');
-const { stripIndent } = require('common-tags');
+const { stripIndents } = require('common-tags');
 
 module.exports = {
   options: {
@@ -74,6 +74,22 @@ module.exports = {
           return true;
         }
         return _.isEqual(one[field.name], two[field.name]);
+      },
+      validate: function (field, options, warn, fail) {
+        if (field.options && field.options.widgets) {
+          for (const name of Object.keys(field.options.widgets)) {
+            if (!self.apos.modules[`${name}-widget`]) {
+              if (name.match(/-widget$/)) {
+                warn(stripIndents`
+                  Do not include "-widget" in the name when configuring a widget in an area field.
+                  Apostrophe will automatically add "-widget" when looking for the right module.
+                `);
+              } else {
+                warn(`Nonexistent widget type name ${name} in area field.`);
+              }
+            }
+          }
+        }
       }
     });
 
@@ -142,11 +158,14 @@ module.exports = {
       // leading slash required). Otherwise, expect a object-style slug
       // (no slashes at all)
       convert: function (req, field, data, destination) {
-        const options = {};
+        const options = {
+          def: field.def
+        };
         if (field.page) {
           options.allow = '/';
         }
         destination[field.name] = self.apos.util.slugify(self.apos.launder.string(data[field.name], field.def), options);
+
         if (field.page) {
           if (!(destination[field.name].charAt(0) === '/')) {
             destination[field.name] = '/' + destination[field.name];
@@ -1217,12 +1236,14 @@ module.exports = {
 
         // all fields in the schema will end up in this variable
         let newSchema = [];
+
         // loop over any groups and orders we want to respect
         _.each(groups, function (group) {
 
           _.each(group.fields, function (field) {
             // find the field we are ordering
             let f = _.find(schema, { name: field });
+
             if (!f) {
               // May have already been migrated due to subclasses re-grouping fields
               f = _.find(newSchema, { name: field });
@@ -1242,6 +1263,7 @@ module.exports = {
               if (fIndex !== -1) {
                 newSchema.splice(fIndex, 1);
               }
+
               newSchema.push(f);
 
               // remove the field from the old schema, if that is where we got it from
@@ -2241,7 +2263,7 @@ module.exports = {
           self.apos.util.error(format(s));
         }
         function format(s) {
-          return stripIndent`
+          return stripIndents`
             ${options.type} ${options.subtype}, ${field.type} field "${field.name}":
 
             ${s}

package/modules/@apostrophecms/schema/ui/apos/components/AposInputArea.vue

@@ -39,11 +39,7 @@ export default {
   mixins: [ AposInputMixin ],
   data () {
     return {
-      next: this.value.data || {
-        metaType: 'area',
-        _id: cuid(),
-        items: []
-      },
+      next: this.value.data || this.getEmptyValue(),
       error: false,
       // This is just meant to be sufficient to prevent unintended collisions
       // in the UI between id attributes
@@ -66,6 +62,17 @@ export default {
     }
   },
   methods: {
+    getEmptyValue() {
+      return {
+        metaType: 'area',
+        _id: cuid(),
+        items: []
+      };
+    },
+    watchValue () {
+      this.error = this.value.error;
+      this.next = this.value.data || this.getEmptyValue();
+    },
     validate(value) {
       if (this.field.required) {
         if (!value.items.length) {

package/modules/@apostrophecms/schema/ui/apos/components/AposInputCheckboxes.vue

@@ -32,13 +32,17 @@ export default {
     getChoiceId(uid, value) {
       return uid + value.replace(/\s/g, '');
     },
+    watchValue () {
+      this.error = this.value.error;
+      this.next = this.value.data || [];
+    },
     validate(values) {
-      if (!Array.isArray(this.field.choices)) {
+      // The choices and values should always be arrays.
+      if (!Array.isArray(this.field.choices) || !Array.isArray(values)) {
         return 'malformed';
       }
 
-      if (this.field.required &&
-        !Array.isArray(values) && (!values || !values.length)) {
+      if (this.field.required && !values.length) {
         return 'required';
       }
 

package/modules/@apostrophecms/ui/ui/apos/lib/i18next.js

@@ -23,6 +23,23 @@ export default {
       debug: i18n.debug,
       interpolation: {
         escapeValue: false
+      },
+      appendNamespaceToMissingKey: true,
+      parseMissingKeyHandler (key) {
+        // We include namespaces with unrecognized l10n keys using
+        // `appendNamespaceToMissingKey: true`. This passes strings containing
+        // colons that were never meant to be localized through to the UI.
+        //
+        // Strings that do not include colons ("Content area") are given the
+        // default namespace by i18next ("translation," by default). Here we
+        // check if the key starts with that default namespace, meaning it
+        // belongs to no other registered namespace, then remove that default
+        // namespace before passing this through to be processed and displayed.
+        if (key.startsWith(`${this.defaultNS[0]}:`)) {
+          return key.slice(this.defaultNS[0].length + 1);
+        } else {
+          return key;
+        }
       }
     });
 

package/modules/@apostrophecms/util/index.js

@@ -936,15 +936,6 @@ module.exports = {
         });
       },
 
-      // If propertyName is _id, then the keys in the returned
-      // object will be the ids of each object in arr,
-      // and the values will be the corresponding objects.
-      // You may index by any property name.
-
-      indexBy: function(arr, propertyName) {
-        return _.indexBy(arr, propertyName);
-      },
-
       // Find all the array elements, if any, that have the specified value for
       // the specified property.
 
@@ -1061,11 +1052,14 @@ module.exports = {
 
         function groupByArray(items, arrayName) {
           const results = {};
+          // looping over each item in the original array
           _.each(items, function(item) {
+            // looping over each item in the array within the top level item
             _.each(item[arrayName] || [], function(inner) {
               if (!results[inner]) {
                 results[inner] = [];
               }
+              // grouping top level items on the sub properties
               results[inner].push(item);
             });
           });

package/modules/@apostrophecms/widget-type/index.js

@@ -112,6 +112,7 @@
 // you are debugging a change and need to test all of the different ways a widget has
 // been used, or are wondering if you can safely remove one.
 
+const { stripIndent } = require('common-tags');
 const _ = require('lodash');
 
 module.exports = {
@@ -310,12 +311,20 @@ module.exports = {
       },
 
       // override to add CSS classes to the outer wrapper div of the widget.
+      // TODO: Remove in the 4.x major version.
       getWidgetWrapperClasses(widget) {
+        self.apos.util.warnDev(stripIndent`
+          The getWidgetWrapperClasses method is deprecated and will be removed in the next
+          major version. The method in 3.x simply returns an empty array.`);
         return [];
       },
 
       // Override to add CSS classes to the div of the widget itself.
+      // TODO: Remove in the 4.x major version.
       getWidgetClasses(widget) {
+        self.apos.util.warnDev(stripIndent`
+          The getWidgetClasses method is deprecated and will be removed in the next major
+          version. The method in 3.x simply returns an empty array.`);
         return [];
       }
     };

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "apostrophe",
-  "version": "3.9.0",
+  "version": "3.12.0",
   "description": "The Apostrophe Content Management System.",
   "main": "index.js",
   "scripts": {
@@ -28,16 +28,16 @@
   "license": "MIT",
   "dependencies": {
     "@apostrophecms/vue-color": "^2.8.2",
-    "@babel/core": "^7.14.3",
-    "@babel/preset-env": "^7.14.4",
+    "@babel/core": "^7.16.7",
+    "@babel/preset-env": "^7.16.7",
     "@tiptap/extension-highlight": "^2.0.0-beta.13",
     "@tiptap/extension-link": "^2.0.0-beta.17",
     "@tiptap/extension-text-align": "^2.0.0-beta.17",
     "@tiptap/extension-text-style": "^2.0.0-beta.13",
-    "@tiptap/extension-underline": "^2.0.0-beta.14",
-    "@tiptap/starter-kit": "^2.0.0-beta.75",
-    "@tiptap/vue-2": "^2.0.0-beta.34",
-    "autoprefixer": "^10.2.4",
+    "@tiptap/extension-underline": "^2.0.0-beta.22",
+    "@tiptap/starter-kit": "^2.0.0-beta.164",
+    "@tiptap/vue-2": "^2.0.0-beta.73",
+    "autoprefixer": "^10.4.1",
     "babel-loader": "^8.2.2",
     "bluebird": "^3.7.2",
     "body-parser": "^1.18.2",
@@ -68,7 +68,7 @@
     "he": "^0.5.0",
     "html-to-text": "^5.1.1",
     "i18next": "^20.3.2",
-    "i18next-http-middleware": "^3.1.4",
+    "i18next-http-middleware": "^3.1.5",
     "import-fresh": "^3.3.0",
     "is-wsl": "^2.2.0",
     "jsdom": "^17.0.0",
@@ -95,7 +95,7 @@
     "resolve": "^1.19.0",
     "resolve-from": "^5.0.0",
     "sanitize-html": "^2.0.0",
-    "sass": "^1.34.1",
+    "sass": "^1.45.2",
     "sass-loader": "^10.1.1",
     "server-destroy": "^1.0.1",
     "sluggo": "^0.3.0",
@@ -110,7 +110,7 @@
     "uploadfs": "^1.17.1",
     "v-tooltip": "^2.0.3",
     "vue": "^2.6.14",
-    "vue-click-outside-element": "^1.0.13",
+    "vue-click-outside-element": "^1.0.15",
     "vue-loader": "^15.9.6",
     "vue-material-design-icons": "~4.12.1",
     "vue-style-loader": "^4.1.2",

package/test/modules/base-type/i18n/custom/en.json

@@ -0,0 +1,4 @@
+{
+  "customTestOne": "Custom Test One From Base Type",
+  "customTestTwo": "Custom Test Two From Base Type"
+}

package/test/modules/base-type/i18n/en.json

@@ -0,0 +1,3 @@
+{
+  "defaultTestOne": "Default Test One"
+}

package/test/modules/nested-module-subdirs/example1/index.js

@@ -0,0 +1,5 @@
+module.exports = {
+  init(self) {
+    self.initialized = true;
+  }
+};

package/test/modules/nested-module-subdirs/modules.js

@@ -0,0 +1,7 @@
+module.exports = {
+  example1: {
+    options: {
+      folderLevelOption: true
+    }
+  }
+};

package/test/modules/subtype/i18n/custom/en.json

@@ -0,0 +1,4 @@
+{
+  "customTestOne": "Custom Test One From Subtype",
+  "customTestThree": "Custom Test Three From Subtype"
+}

package/test/modules/subtype/index.js

@@ -0,0 +1,7 @@
+module.exports = {
+  i18n: {
+    custom: {
+      browser: true
+    }
+  }
+};

package/test/moog.js

@@ -284,6 +284,54 @@ describe('moog', function() {
       assert(myObject.fieldsGroups.basics.fields.includes('five'));
     });
 
+    it('should order fields with the last option unless the order array overrides', async function() {
+      const moog = require('../lib/moog.js')({});
+
+      moog.define('unorderedObject', {
+        cascades: [ 'fields' ],
+        fields: {
+          add: {
+            first: { type: 'string' },
+            last: {
+              type: 'string',
+              last: true
+            },
+            second: { type: 'string' },
+            third: { type: 'string' }
+          }
+        }
+      });
+
+      moog.define('orderedObject', {
+        cascades: [ 'fields' ],
+        fields: {
+          add: {
+            first: { type: 'string' },
+            last: {
+              type: 'string',
+              last: true
+            },
+            second: { type: 'string' },
+            third: { type: 'string' }
+          },
+          order: [ 'last', 'third', 'second', 'first' ]
+        }
+      });
+      const unordered = await moog.create('unorderedObject', {});
+      assert(unordered);
+      assert(Object.keys(unordered.fields)[0] === 'first');
+      assert(Object.keys(unordered.fields)[1] === 'second');
+      assert(Object.keys(unordered.fields)[2] === 'third');
+      assert(Object.keys(unordered.fields)[3] === 'last');
+
+      const ordered = await moog.create('orderedObject', {});
+      assert(ordered);
+      assert(Object.keys(ordered.fields)[0] === 'last');
+      assert(Object.keys(ordered.fields)[1] === 'third');
+      assert(Object.keys(ordered.fields)[2] === 'second');
+      assert(Object.keys(ordered.fields)[3] === 'first');
+    });
+
     // ==================================================
     // `redefine` AND `isDefined`
     // ==================================================

package/test/static-i18n.js

@@ -33,9 +33,19 @@ describe('static i18n', function() {
         'apos-fr': {
           options: {
             i18n: {
+              // Legacy technique must work
               ns: 'apostrophe'
             }
           }
+        },
+        // A base class that contributes some namespaced phrases in the new style way (subdirs)
+        'base-type': {
+          instantiate: false
+        },
+        // Also contributes namespaced phrases in the new style way (subdirs)
+        // plus default locale phrases in the root i18n folder
+        subtype: {
+          extend: 'base-type'
         }
       }
     });
@@ -60,4 +70,22 @@ describe('static i18n', function() {
     assert.strictEqual(apos.task.getReq({ locale: 'fr' }).t('apostrophe:richTextAlignCenter'), 'Aligner Le Centre');
   });
 
+  it('should fetch default locale phrases from main i18n dir with no i18n option necessary', function() {
+    assert.strictEqual(apos.task.getReq().t('defaultTestOne'), 'Default Test One');
+  });
+
+  it('should fetch custom locale phrases from corresponding subdir', function() {
+    assert.strictEqual(apos.task.getReq().t('custom:customTestTwo'), 'Custom Test Two From Base Type');
+    assert.strictEqual(apos.task.getReq().t('custom:customTestThree'), 'Custom Test Three From Subtype');
+  });
+
+  it('last appearance in inheritance + configuration order wins', function() {
+    assert.strictEqual(apos.task.getReq().t('custom:customTestOne'), 'Custom Test One From Subtype');
+  });
+
+  it('should honor the browser: true flag in the i18n section of an index.js file', function() {
+    const browserData = apos.i18n.getBrowserData(apos.task.getReq());
+    assert.strictEqual(browserData.i18n.en.custom.customTestOne, 'Custom Test One From Subtype');
+  });
+
 });

package/test/with-nested-module-subdirs.js

@@ -0,0 +1,32 @@
+const t = require('../test-lib/test.js');
+const assert = require('assert');
+
+describe('With Nested Module Subdirs', function() {
+  this.timeout(t.timeout);
+
+  let apos;
+
+  after(function () {
+    return t.destroy(apos);
+  });
+
+  /// ///
+  // EXISTENCE
+  /// ///
+
+  it('should initialize', async function() {
+    apos = await t.create({
+      root: module,
+      nestedModuleSubdirs: true,
+      modules: {
+        example1: {}
+      }
+    });
+    assert(apos.modules.example1);
+    // With nestedModuleSubdirs switched on, the index.js should be found,
+    // and modules.js should be loaded
+    assert(apos.modules.example1.options.folderLevelOption);
+    assert(apos.modules.example1.initialized);
+  });
+
+});

package/test/without-nested-module-subdirs.js

@@ -0,0 +1,31 @@
+const t = require('../test-lib/test.js');
+const assert = require('assert');
+
+describe('Without Nested Module Subdirs', function() {
+  this.timeout(t.timeout);
+
+  let apos;
+
+  after(function () {
+    return t.destroy(apos);
+  });
+
+  /// ///
+  // EXISTENCE
+  /// ///
+
+  it('should initialize', async function() {
+    apos = await t.create({
+      root: module,
+      modules: {
+        example1: {}
+      }
+    });
+    assert(apos.modules.example1);
+    // Should fail because we didn't turn on nestedModuleSubdirs,
+    // so the index.js was not found and modules.js was not loaded
+    assert(!apos.modules.example1.options.folderLevelOption);
+    assert(!apos.modules.example1.initialized);
+  });
+
+});

package/test-lib/util.js

@@ -43,12 +43,14 @@ async function create(options) {
   };
   // Automatically configure Express, but not if we're in a special
   // environment where the default apostrophe modules don't initialize
+  // TODO: Remove __testDefaults references in 4.x major version or formalize
+  // intended usage with documentation.
   if (!config.__testDefaults) {
     config.modules = config.modules || {};
     const express = config.modules['@apostrophecms/express'] || {};
     express.options = express.options || {};
-    // Allow OS to choose open port
-    express.options.port = null;
+    // Allow OS to choose open port if not explicitly set.
+    express.options.port = express.options.port || null;
     express.options.address = express.options.address || 'localhost';
     express.options.session = express.options.session || {};
     express.options.session.secret = express.options.session.secret || 'test';

package/.github/pull_request_template.md

@@ -1,8 +0,0 @@
-Please ensure your pull request follows these guidelines:
-
-- [ ] Link to the related issue with requirements and/or clearly describe 1) what problem this solves and 2) the requirements to review it against.
-- [ ] Update the `CHANGELOG.md` file with the added feature or fix. If there is not yet a heading for an upcoming release, add one following the established pattern.
-- [ ] Keep the pull request minimal! Break large updates into separate pull requests for individual features/fixes whenever possible. Small requests get reviewed more quickly.
-- [ ] All tests must pass, including the linters. Run `npm run lint` to test code linting independently.
-
-Thanks for contributing!