adapt-authoring-core 1.3.0 → 1.3.1

package/.github/workflows/new.yml

@@ -1,19 +1,15 @@
-name: Add to main project
+# Calls the org-level reusable workflow to add PRs to the TODO Board
+
+name: Add PR to Project
 
 on:
-  issues:
-    types:
-      - opened
   pull_request:
     types:
       - opened
+      - reopened
 
 jobs:
   add-to-project:
-    name: Add to main project
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/add-to-project@v0.1.0
-        with:
-          project-url: https://github.com/orgs/adapt-security/projects/5
-          github-token: ${{ secrets.PROJECTS_SECRET }}
+    uses: adapt-security/.github/.github/workflows/new.yml@main
+    secrets:
+      PROJECTS_SECRET: ${{ secrets.PROJECTS_SECRET }}

package/adapt-authoring.json

@@ -7,10 +7,10 @@
     "manualPages": {
       "binscripts.md": "reference",
       "coremodules.md": "reference",
-      "data-folder.md": "basics",
       "customising.md": "advanced",
+      "folder-structure.md": "getting-started",
+      "hooks.md": "basics",
       "licensing.md": "reference",
-      "temp-folder.md": "basics",
       "writing-a-module.md": "basics",
       "writing-core-code.md": "contributing"
     },

package/docs/folder-structure.md

@@ -0,0 +1,105 @@
+# Folder structure
+
+This guide explains the folder structure of an Adapt authoring tool installation.
+
+## Overview
+
+A typical installation has the following structure:
+
+```
+adapt-authoring/
+├── APP_DATA/            # Application runtime data
+│   ├── data/            # Persistent files
+│   └── temp/            # Temporary files
+├── bin/                 # CLI scripts
+├── conf/                # Environment configuration files
+└── node_modules/        # Installed modules and dependencies
+```
+
+## `APP_DATA`
+
+The `APP_DATA` folder contains all files required to run an Adapt authoring tool instance (with the exception of the source code, which are found in `node_modules`).
+
+### `data`
+
+The data directory stores persistent application data that must survive restarts and updates. By default this is `APP_DATA/data/`, but can be configured via the `dataDir` config option.
+
+> **Warning:** Never modify or delete the data directory while the application is running. Doing so will cause data loss and runtime errors.
+
+### Using the data directory
+
+Module developers should store any data that must persist between restarts in this directory. You can use the `$DATA` variable in your config schema to automatically populate this value at runtime:
+
+```json
+{
+  "myDataPath": {
+    "type": "string",
+    "isDirectory": true,
+    "default": "$DATA/mymodule"
+  }
+}
+```
+
+See the [configuration guide](configuration.md) for more information.
+
+
+### `temp`
+
+The temp directory stores temporary files used at runtime. By default this is `APP_DATA/temp/`, but can be configured via `tempDir`. Files in this directory may be removed at any time when the application is stopped.
+
+#### Using the temp directory
+
+Developers should use the temp directory to store any files which are not needed permanently. Please try to remove any temporary files once they're no longer needed to conserve disk space and reduce the need for manual housekeeping by the server admin.
+
+Examples of the kind of data found in the temp directory: generated asset thumbnails, documentation build output, compiled UI app code and file uploads.
+
+As with the data directory, there is a custom variable (`$TEMP`) which can be used in config schemas to populate the correct value at runtime:
+
+```json
+{
+  "myCachePath": {
+    "type": "string",
+    "isDirectory": true,
+    "default": "$TEMP/mymodule-cache"
+  }
+}
+```
+
+### Clearing the temp directory
+
+It is safe to delete the temp directory when the application is stopped. On restart, the application will recreate any required directories. This can be useful for:
+
+- Freeing disk space
+- Preparing for updates
+
+> **Warning:** Do not delete the temp directory while the application is running. Users will encounter errors and builds will fail.
+
+## `bin`
+
+Command-line tools are found in the `bin` folder. Modules can also provide their own CLIs which are available via `npx`. See the [CLI reference](binscripts.md) for available commands.
+
+## `docs`
+
+The `doc` folder contains documentation pages. These are picked up and compiled by the documentation generation tools when running `at-docgen`. See the [Building the docs](building-docs) for details.
+
+## `conf`
+
+The `conf/` directory contains configuration files. The application loads the file matching your `NODE_ENV` value (e.g., `production.config.js` when `NODE_ENV=production`). If `NODE_ENV` is not set, it defaults to `production`. 
+
+See the [configuration guide](configuration.md) for details.
+
+## `node_modules`
+
+The Adapt authoring tool is built from modular components, each published as an npm package. All modules are installed into the `node_modules/` directory as standard npm dependencies.
+
+Core modules follow the naming convention `adapt-authoring-*` and are listed as dependencies in `package.json`, e.g.
+
+```json
+{
+  "dependencies": {
+    "adapt-authoring-auth": "^1.0.0",
+    "adapt-authoring-content": "^1.0.0",
+    "adapt-authoring-mongodb": "^1.0.0"
+  }
+}
+```

package/docs/hooks.md

@@ -0,0 +1,214 @@
+# Hooks
+
+Hooks allow modules to react to events and extend functionality without modifying core code. They're the primary mechanism for inter-module communication in the Adapt authoring tool.
+
+## How hooks work
+
+A hook is a point in the code where external observers can run their own functions. When a hook is invoked, all registered observers are called with the same arguments.
+
+All hook observers must complete before the operation continues. For example, a document won't be inserted until all `preInsertHook` observers have finished executing.
+
+### Mutable vs. non-mutable
+
+Hooks can be either **mutable** or **immutable**:
+
+- **Immutable**: the _default_ behaviour, observers receive a deep copy of any arguments to ensure that the original data is read-only and prevent unintended modifications. By default, observers are run in parallel (at the same time).
+- **Mutable**: hooks allow modification of param data, and run observers in series (one after another) to ensure modifications are applied in order.
+
+## Basic usage
+
+### Creating hooks
+
+```javascript
+import { Hook } from 'adapt-authoring-core'
+
+class MyModule extends AbstractModule {
+  async init () {
+    // param data will be read only, observers will be run in parallel
+    this.myBasicHook = new Hook()
+
+    // allows param data to be modified, observers run in series
+    this.myMutableHook = new Hook({ mutable: true })
+
+    // force observers to run in series
+    this.mySeriesHook = new Hook({ type: Hook.Types.Series })
+  }
+
+  async doSomething () {
+    // Invoke the hook, passing any relevant data
+    await this.myBasicHook.invoke(someData)
+  }
+}
+```
+
+### Listening to a hook
+
+Use `tap()` to register an observer function. 
+
+Listeners can be `async`, and a second `scope` parameter can be passed to bind `this`:
+
+```javascript
+const content = await this.app.waitForModule('content')
+
+content.preInsertHook.tap(async data => {
+  data.createdAt = new Date()
+}, this)
+```
+
+### Removing an observer
+
+Use `untap()` to remove an observer:
+
+```javascript
+const observer = data => console.log(data)
+content.preInsertHook.tap(observer)
+
+// Later...
+content.preInsertHook.untap(observer)
+```
+
+### Waiting for a hook
+
+Use `onInvoke()` to get a promise that resolves when the hook is next invoked:
+
+```javascript
+await server.listeningHook.onInvoke()
+console.log('Server is now listening')
+```
+
+### Error handling
+
+If an observer throws an error, the hook stops executing and the error propagates to the caller. For mutable hooks running in series, any observers after the failing one won't be called.
+
+```javascript
+content.preInsertHook.tap(data => {
+  if (!data.title) {
+    throw new Error('Title is required')
+  }
+})
+
+try {
+  await content.insert({ body: 'No title here' })
+} catch (e) {
+  console.log(e.message) // 'Title is required'
+}
+```
+
+## Best practices
+
+1. **Keep observers focused** — Each observer should do one thing well
+2. **Handle errors gracefully** — Don't let one observer break the entire flow (unless intended)
+3. **Avoid side effects in non-mutable hooks** — They receive copies of data, so modifications won't persist
+4. **Use descriptive names** — Try to name your hooks clearly, and try to follow established patterns (see below for examples)
+6. **Consider execution order** — For mutable hooks, observers run in the order they were registered. Keep this in mind both as the hook creator, and as the hook observer.
+
+## Common hooks
+
+Below are some commonly used hooks, which you may find useful.
+
+| Module | Hook | Description | Parameters | Mutable |
+| ------ | ---- | ----------- | ---------- | :-----: |
+| AbstractModule | `readyHook` | Module has initialised | | No |
+| AbstractApiModule | `requestHook` | API request received | `(req)` | Yes |
+| AbstractApiModule | `preInsertHook` | Before document insert | `(data, options, mongoOptions)` | Yes |
+| AbstractApiModule | `postInsertHook` | After document insert | `(doc)` | No |
+| AbstractApiModule | `preUpdateHook` | Before document update | `(originalDoc, newData, options, mongoOptions)` | Yes |
+| AbstractApiModule | `postUpdateHook` | After document update | `(originalDoc, updatedDoc)` | No |
+| AbstractApiModule | `preDeleteHook` | Before document delete | `(doc, options, mongoOptions)` | No |
+| AbstractApiModule | `postDeleteHook` | After document delete | `(doc)` | No |
+| AbstractApiModule | `accessCheckHook` | Check document access | `(req, doc)` | No |
+| AdaptFrameworkBuild | `preBuildHook` | Before course build starts | | Yes |
+| AdaptFrameworkBuild | `postBuildHook` | After course build completes | | Yes |
+| AdaptFrameworkImport | `preImportHook` | Before course import starts | | No |
+| AdaptFrameworkImport | `postImportHook` | After course import completes | | No |
+
+## Practical examples
+
+### Adding timestamps
+
+```javascript
+async init () {
+  await super.init()
+  const content = await this.app.waitForModule('content')
+
+  content.preInsertHook.tap(data => {
+    data.createdAt = new Date()
+  })
+
+  content.preUpdateHook.tap((original, newData) => {
+    newData.updatedAt = new Date()
+  })
+}
+```
+
+### Enforcing data format
+
+```javascript
+async init () {
+  await super.init()
+
+  this.preInsertHook.tap(this.forceLowerCaseEmail)
+  this.preUpdateHook.tap(this.forceLowerCaseEmail)
+}
+
+forceLowerCaseEmail (data) {
+  if (data.email) {
+    data.email = data.email.toLowerCase()
+  }
+}
+```
+
+### Access control
+
+```javascript
+async init () {
+  await super.init()
+  const content = await this.app.waitForModule('content')
+
+  content.accessCheckHook.tap((req, doc) => {
+    // Only allow access to own documents
+    if (doc.createdBy !== req.auth.user._id.toString()) {
+      throw this.app.errors.UNAUTHORISED
+    }
+  })
+}
+```
+
+### Cascading deletes
+
+```javascript
+async init () {
+  await super.init()
+  const assets = await this.app.waitForModule('assets')
+
+  assets.preDeleteHook.tap(async doc => {
+    // Remove all references to this asset
+    await this.removeAssetReferences(doc._id)
+  })
+}
+```
+
+### Registering schemas
+
+```javascript
+async init () {
+  await super.init()
+  const jsonschema = await this.app.waitForModule('jsonschema')
+
+  jsonschema.registerSchemasHook.tap(async () => {
+    await jsonschema.registerSchema('/path/to/schema.json')
+  })
+}
+```
+
+### Waiting for server startup
+
+```javascript
+async init () {
+  await super.init()
+  const server = await this.app.waitForModule('server')
+
+  await server.listeningHook.onInvoke()
+  this.log('info', 'Server is ready, starting background tasks')
+}
+```

package/docs/plugins/binscripts.js

@@ -1,45 +1,48 @@
-import fs from 'fs-extra';
-import { parse } from 'comment-parser';
+import fs from 'fs-extra'
+import { parse } from 'comment-parser'
 
 export default class BinScripts {
-  async run() {
-    this.manualFile = 'binscripts.md';
+  async run () {
+    this.manualFile = 'binscripts.md'
     this.replace = { CONTENT: await this.generateMd() }
   }
-  async generateMd() {
-    const allDeps = await Promise.all(Object.values(this.app.dependencies).map(this.processDep));
+
+  async generateMd () {
+    const allDeps = await Promise.all(Object.values(this.app.dependencies).map(this.processDep))
     return allDeps
       .reduce((m, d) => d ? m.concat(d) : m, [])
-      .sort((a,b) => a.name.localeCompare(b.name))
+      .sort((a, b) => a.name.localeCompare(b.name))
       .map(d => this.dataToMd(d))
-      .join('\n');
+      .join('\n')
   }
-  async processDep({ name, bin, rootDir }) {
-    if(!bin || typeof bin === 'string') {
-      return;
+
+  async processDep ({ name, bin, rootDir }) {
+    if (!bin || typeof bin === 'string') {
+      return
     }
     return await Promise.all(Object.entries(bin).map(async ([scriptName, filePath]) => {
-      const data = { name: scriptName, description: 'No description provided.', moduleName: name };
-      const contents = (await fs.readFile(`${rootDir}/${filePath}`)).toString();
-      const match = contents.match(/^#!\/usr\/bin\/env node(\s*)?\/\*\*([\s\S]+?)\*\//);
-      if(match) {
-        const [{ description, tags }] = parse(match[0]);
-        const params = tags.reduce((m,t) => {
-          if(t.tag === 'param') m.push({ name: t.name, description: t.description });
-          return m;
-        }, []);
-        data.description = description;
-        if(params.length) data.params = params;
+      const data = { name: scriptName, description: 'No description provided.', moduleName: name }
+      const contents = (await fs.readFile(`${rootDir}/${filePath}`)).toString()
+      const match = contents.match(/^#!\/usr\/bin\/env node(\s*)?\/\*\*([\s\S]+?)\*\//)
+      if (match) {
+        const [{ description, tags }] = parse(match[0])
+        const params = tags.reduce((m, t) => {
+          if (t.tag === 'param') m.push({ name: t.name, description: t.description })
+          return m
+        }, [])
+        data.description = description
+        if (params.length) data.params = params
       }
-      return data;
-    }));
+      return data
+    }))
   }
-  dataToMd({ name, description, moduleName, params }) {
+
+  dataToMd ({ name, description, moduleName, params }) {
     let content = `<h2 class="script" id="${name}">${name} <span class="module">from ${moduleName}</span></h2>`
-    content += `<div class="details"><p class="description">${description}</p>`;
-    if(params) {
-      content += `<p><b>Params</b><ul>${params.reduce((s,p) => `${s}<li><code>${p.name}</code> ${p.description}</li>`, '')}</ul></p>`;
+    content += `<div class="details"><p class="description">${description}</p>`
+    if (params) {
+      content += `<p><b>Params</b><ul>${params.reduce((s, p) => `${s}<li><code>${p.name}</code> ${p.description}</li>`, '')}</ul></p>`
     }
-    return content;
+    return content
   }
-}
+}

package/docs/plugins/coremodules.js

@@ -1,15 +1,16 @@
 export default class CoreModules {
-  async run() {
-    this.manualFile = 'coremodules.md';
+  async run () {
+    this.manualFile = 'coremodules.md'
     this.replace = {
       VERSION: this.app.pkg.version,
       MODULES: this.generateMd()
-    };
+    }
   }
-  generateMd() {
+
+  generateMd () {
     return Object.keys(this.app.dependencies).sort().reduce((s, name) => {
-      const { version, description, homepage } = this.app.dependencies[name];
-      return s += `\n| ${homepage ? `[${name}](${homepage})` : name} | ${version} | ${description} |`;
-    }, '| Name | Version | Description |\n| - | :-: | - |');
+      const { version, description, homepage } = this.app.dependencies[name]
+      return `${s}\n| ${homepage ? `[${name}](${homepage})` : name} | ${version} | ${description} |`
+    }, '| Name | Version | Description |\n| - | :-: | - |')
   }
-}
+}

package/docs/plugins/licensing.js

@@ -83,17 +83,19 @@ export default class Licensing {
 
     Object.keys(this.licenses)
       .sort()
-      .forEach(l => md += `| ${l} | ${this.licenses[l].count} |\n`)
+      .forEach(l => {
+        md += `| ${l} | ${this.licenses[l].count} |\n`
+      })
 
     return md
   }
 
   async generateLicenseDetailsMd () {
     let md = ''
-    Object.entries(this.licenses).forEach(([key, { name, spdx_id, description, body, permissions }]) => {
+    Object.entries(this.licenses).forEach(([key, { name, spdxId, description, body, permissions }]) => {
       if (!name) return
       md += '<details>\n'
-      md += `<summary>${name} (${spdx_id})</summary>\n`
+      md += `<summary>${name} (${spdxId})</summary>\n`
       md += `<p>${description}</p>\n`
       md += `<p>This license allows the following:\n<ul>${permissions.map(p => `<li>${this.permissionsMap(p)}</li>`).join('\n')}</ul></p>\n`
       md += '<p>The original license text is as follows:</p>\n'
@@ -113,7 +115,9 @@ export default class Licensing {
 
   async generateMd () {
     let md = '<tr><th>Name</th><th>Version</th><th>License</th><th>Description</th></tr>\n'
-    this.dependencies.forEach(pkg => md += `<tr><td>${pkg.homepage ? `<a href="${pkg.homepage}" target="_blank">${pkg.name}</a>` : pkg.name}</td><td>${pkg.version}</td><td>${pkg.license}</td><td>${pkg.description}</tr>\n`)
+    this.dependencies.forEach(pkg => {
+      md += `<tr><td>${pkg.homepage ? `<a href="${pkg.homepage}" target="_blank">${pkg.name}</a>` : pkg.name}</td><td>${pkg.version}</td><td>${pkg.license}</td><td>${pkg.description}</tr>\n`
+    })
     return `<details>\n<summary>Module dependency list</summary>\n<table>${md}</table>\n</details>`
   }
 }

package/lib/App.js

@@ -1,6 +1,5 @@
 import AbstractModule from './AbstractModule.js'
 import DependencyLoader from './DependencyLoader.js'
-import { fileURLToPath } from 'url'
 import fs from 'fs'
 import path from 'path'
 import Utils from './Utils.js'
@@ -95,7 +94,7 @@ class App extends AbstractModule {
         branch: gitHead.split('/').pop(),
         commit: fs.readFileSync(path.join(gitRoot, gitHead.split(': ')[1]), 'utf8').trim()
       }
-    } catch(e) {
+    } catch (e) {
       return {}
     }
   }

package/lib/Utils.js

@@ -63,7 +63,6 @@ class Utils {
         error = e
       })
       task.on('close', exitCode => {
-        console.log(output);
         exitCode !== 0 ? reject(App.instance.errors.SPAWN.setData({ error: error ?? output })) : resolve(output)
       })
     })

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "adapt-authoring-core",
-  "version": "1.3.0",
+  "version": "1.3.1",
   "description": "A bundle of reusable 'core' functionality",
   "homepage": "https://github.com/adapt-security/adapt-authoring-core",
   "license": "GPL-3.0",
@@ -8,8 +8,8 @@
   "main": "index.js",
   "repository": "github:adapt-security/adapt-authoring-core",
   "dependencies": {
-    "fs-extra": "11.3.1",
-    "glob": "^11.0.0",
+    "fs-extra": "11.3.3",
+    "glob": "^13.0.0",
     "lodash": "^4.17.21",
     "minimist": "^1.2.8"
   },

package/.github/workflows/labelled_prs.yml

@@ -1,16 +0,0 @@
-name: Add labelled PRs to project
-
-on:
-  pull_request:
-    types: [ labeled ]
-
-jobs:
-  add-to-project:
-    if: ${{ github.event.label.name == 'dependencies' }}
-    name: Add to main project
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/add-to-project@v0.1.0
-        with:
-          project-url: https://github.com/orgs/adapt-security/projects/5
-          github-token: ${{ secrets.PROJECTS_SECRET }}

package/docs/data-folder.md

@@ -1,4 +0,0 @@
-# The data folder
-The data folder is used to store persistant application data and **should not in any circumstances be modified or removed**. Doing so will result in data loss and generally cause unexpected runtime issues.
-
-As a module developer, you should store any data in here which should persist between app restarts. You can make use of the `isDirectory` schema keyword to make configuration simpler for end-users. See the [schemas page](schemas-introduction#isdirectory) for more information.

package/docs/temp-folder.md

@@ -1,12 +0,0 @@
-# The temp folder
-The `temp` folder (note: this path will vary depending on your local configuration) is, as the name suggests, a temporary store for files used at runtime by the application. These files can be related to any number of things such as file uploads, course builds, documentation builds and so on.
-
-## Using the temp folder
-As a module developer, it is recommended that you store any temporary files needed by your module in here. Please be aware however, that due to its non-permanent nature, any data stored in the temp folder has the potential to be removed at any time. We therefore recommend running relevant checks during the startup of your module, and reinitialise any missing files at that point.
-
-It is safe to assume that the temp folder will *not* be removed while the app is running; anyone choosing to do so should expect fatal errors.
-
-## Removing the temp folder
-It is perfectly safe to remove the temp folder. You may wish to do this in the event of low disk space, before an update, or even as a regular housekeeping task.
-
-> If you do remove the temp folder, this must be done while the app is stopped, or your users may encounter errors while using the app. Restarting the application will ensure that any vital files are reinstated.