@defra/forms-engine-plugin 0.0.5 → 0.1.0

package/README.md

@@ -1,21 +1,242 @@
-# forms-engine
+# @defra/forms-engine-plugin
 
-Form hapi-plugin
+The `@defra/forms-engine-plugin` is a [plugin](https://hapi.dev/tutorials/plugins/?lang=en_US) for [hapi](https://hapi.dev/) used to serve GOV.UK-based form journeys.
 
-...
+It is designed to be embedded in the frontend of a digital service and provide a convenient, configuration driven approach to building forms that are aligned to [GDS Design System](https://design-system.service.gov.uk/) guidelines.
+
+## Table of Contents
+
+- [Installation](#installation)
+- [Dependencies](#dependencies)
+- [Setup](#setup)
+  - [Form Config](#form-config)
+  - [Static Assets and Styles](#static-assets-and-styles)
+- [Example](#example)
+- [Environment Variables](#environment-variables)
+- [Options](#options)
+  - [Services](#services)
+  - [Custom Controllers](#custom-controllers)
+  - [Custom Filters](#custom-filters)
+  - [Custom Cache](#custom-cache)
+- [Exemplar](#exemplar)
+- [Templates](#templates)
+  - [Template Data](#template-data)
+  - [Liquid Filters](#liquid-filters)
+  - [Examples](#examples)
+- [Templates and Views: Extending the Default Layout](#templates-and-views-extending-the-default-layout)
+- [Publishing the Package](#publishing-the-package)
+  - [Semantic Versioning Control](#semantic-versioning-control)
+  - [Major-Version Release Branches](#major-version-release-branches)
+  - [Manual Workflow Triggers](#manual-workflow-triggers)
+  - [Workflow Triggers](#workflow-triggers)
+  - [Safety and Consistency](#safety-and-consistency)
+
+## Installation
+
+`npm install @defra/forms-engine-plugin --save`
+
+## Dependencies
+
+The following are [plugin dependencies](<https://hapi.dev/api/?v=21.4.0#server.dependency()>) that are required to be registered with hapi:
+
+`npm install hapi-pino @hapi/crumb @hapi/yar @hapi/vision --save`
+
+- [hapi-pino](https://github.com/hapijs/hapi-pino) - [Pino](https://github.com/pinojs/pino) logger for hapi
+- [@hapi/crumb](https://github.com/hapijs/crumb) - CSRF crumb generation and validation
+- [@hapi/yar](https://github.com/hapijs/yar) - Session manager
+- [@hapi/vision](https://github.com/hapijs/vision) - Template rendering support
+
+Additional npm dependencies that you will need are:
+
+`npm install nunjucks govuk-frontend --save`
+
+- [nunjucks](https://www.npmjs.com/package/nunjucks) - [templating engine](https://mozilla.github.io/nunjucks/) used by GOV.UK design system
+- [govuk-frontend](https://www.npmjs.com/package/govuk-frontend) - [code](https://github.com/alphagov/govuk-frontend) you need to build a user interface for government platforms and services
+
+Optional dependencies
+
+`npm install @hapi/inert --save`
+
+- [@hapi/inert](https://www.npmjs.com/package/@hapi/inert) - static file and directory handlers for serving GOV.UK assets and styles
+
+## Setup
+
+### Form config
+
+The `form-engine-plugin` uses JSON configuration files to serve form journeys.
+These files are called `Form definitions` and are built up of:
+
+- `pages` - includes a `path`, `title`
+- `components` - one or more questions on a page
+- `conditions` - used to conditionally show and hide pages and
+- `lists` - data used to in selection fields like [Select](https://design-system.service.gov.uk/components/select/), [Checkboxes](https://design-system.service.gov.uk/components/checkboxes/) and [Radios](https://design-system.service.gov.uk/components/radios/)
+
+The [types](https://github.com/DEFRA/forms-designer/blob/main/model/src/form/form-definition/types.ts), `joi` [schema](https://github.com/DEFRA/forms-designer/blob/main/model/src/form/form-definition/index.ts) and the [examples](test/form/definitions) folder are a good place to learn about the structure of these files.
+
+TODO - Link to wiki for `Form metadata`
+TODO - Link to wiki for `Form definition`
+
+#### Providing form config to the engine
+
+The engine plugin registers several [routes](https://hapi.dev/tutorials/routing/?lang=en_US) on the hapi server.
+
+They look like this:
+
+```
+GET     /{slug}/{path}
+POST    /{slug}/{path}
+```
+
+A unique `slug` is used to route the user to the correct form, and the `path` used to identify the correct page within the form to show.
+The [plugin registration options](#options) have a `services` setting to provide a `formsService` that is responsible for returning `form definition` data.
+
+WARNING: This below is subject to change
+
+A `formsService` has two methods, one for returning `formMetadata` and another to return `formDefinition`s.
+
+```
+const formsService = {
+  getFormMetadata: async function (slug) {
+    // Returns the metadata for the slug
+  },
+  getFormDefinition: async function (id, state) {
+    // Returns the form definition for the given id
+  }
+}
+```
+
+The reason for the two separate methods is caching.
+`formMetadata` is a lightweight record designed to give top level information about a form.
+This method is invoked for every page request.
+
+Only when the `formMetadata` indicates that the definition has changed is a call to `getFormDefinition` is made.
+The response from this can be quite big as it contains the entire form definition.
+
+See [example](#example) below for more detail
+
+### Static assets and styles
+
+TODO
+
+## Example
+
+```
+import hapi from '@hapi/hapi'
+import yar from '@hapi/yar'
+import crumb from '@hapi/crumb'
+import inert from '@hapi/inert'
+import pino from 'hapi-pino'
+import plugin from '@defra/forms-engine-plugin'
+
+const server = hapi.server({
+  port: 3000
+})
+
+// Register the dependent plugins
+await server.register(pino)
+await server.register(inert)
+await server.register(crumb)
+await server.register({
+  plugin: yar,
+  options: {
+    cookieOptions: {
+      password: 'ENTER_YOUR_SESSION_COOKIE_PASSWORD_HERE' // Must be > 32 chars
+    }
+  }
+})
+
+// Register the `forms-engine-plugin`
+await server.register({
+  plugin
+})
+
+await server.start()
+```
+
+## Environment variables
+
+## Options
+
+The forms plugin is configured with [registration options](https://hapi.dev/api/?v=21.4.0#plugins)
+
+- `services` (optional) - object containing `formsService`, `formSubmissionService` and `outputService`
+  - `formsService` - used to load `formMetadata` and `formDefinition`
+  - `formSubmissionService` - used prepare the form during submission (ignore - subject to change)
+  - `outputService` - used to save the submission
+- `controllers` (optional) - Object map of custom page controllers used to override the default. See [custom controllers](#custom-controllers)
+- `filters` (optional) - A map of custom template filters to include
+- `cacheName` (optional) - The cache name to use. Defaults to hapi's [default server cache]. Recommended for production. See [here]
+  (#custom-cache) for more details
+- `pluginPath` (optional) - The location of the plugin (defaults to `node_modules/@defra/forms-engine-plugin`)
+
+### Services
+
+TODO
+
+### Custom controllers
+
+TODO
+
+### Custom filters
+
+Use the `filter` plugin option to provide custom template filters.
+Filters are available in both [nunjucks](https://mozilla.github.io/nunjucks/templating.html#filters) and [liquid](https://liquidjs.com/filters/overview.html) templates.
+
+```
+const formatter = new Intl.NumberFormat('en-GB')
+
+await server.register({
+  plugin,
+  options: {
+    filters: {
+      money: value => formatter.format(value),
+      upper: value => typeof value === 'string' ? value.toUpperCase() : value
+    }
+  }
+})
+```
+
+### Custom cache
+
+The plugin will use the [default server cache](https://hapi.dev/api/?v=21.4.0#-serveroptionscache) to store form answers on the server.
+This is just an in-memory cache which is fine for development.
+
+In production you should create a custom cache one of the available `@hapi/catbox` adapters.
+
+E.g. [Redis](https://github.com/hapijs/catbox-redis)
+
+```
+import { Engine as CatboxRedis } from '@hapi/catbox-redis'
+
+const server = new Hapi.Server({
+  cache : [
+    {
+      name: 'my_cache',
+      provider: {
+        constructor: CatboxRedis,
+        options: {}
+      }
+    }
+  ]
+})
+```
+
+## Exemplar
+
+TODO: Link to CDP exemplar
 
 ## Templates
 
 The following elements support [LiquidJS templates](https://liquidjs.com/):
 
 - Page **title**
-- Form component **titles**
+- Form component **title**
   - Support for fieldset legend text or label text
   - This includes when the title is used in **error messages**
 - Html (guidance) component **content**
 - Summary component **row** key title (check answers and repeater summary)
 
-### Template data
+### Template Data
 
 The data the templates are evaluated against is the raw answers the user has provided up to the page they're currently on.
 For example, given a YesNoField component called `TKsWbP`, the template `{{ TKsWbP }}` would render "true" or "false" depending on how the user answered the question.
@@ -85,3 +306,60 @@ There are a number of `LiquidJS` filters available to you from within the templa
   }
 ]
 ```
+
+## Templates and views: Extending the default layout
+
+TODO
+
+To override the default page template, vision and nunjucks both need to be configured to search in the `forms-engine-plugin` views directory when looking for template files.
+
+For vision this is done through the `path` [plugin option](https://github.com/hapijs/vision/blob/master/API.md#options)
+For nunjucks it is configured through the environment [configure options](https://mozilla.github.io/nunjucks/api.html#configure).
+
+The `forms-engine-plugin` path to add can be imported from:
+
+`import { VIEW_PATH } from '@defra/forms-engine-plugin'`
+
+Which can then be appended to the `node_modules` path `node_modules/@defra/forms-engine`.
+
+The main template layout is `govuk-frontend`'s `template.njk` file, this also needs to be added to the `path`s that nunjucks can look in.
+
+## Publishing the Package
+
+Our GitHub Actions workflow (`publish.yml`) is set up to make publishing a breeze, using semantic versioning and a variety of release strategies. Here's how you can make the most of it:
+
+### Semantic Versioning Control
+
+- **Patch Versioning**: This happens automatically whenever you merge code changes into `main` or any release branch.
+- **Minor and Major Bumps**: You can control these by making empty commits with specific hashtags:
+  - Use `#minor` for a minor version bump.
+  - Use `#major` for a major version bump.
+
+**Example Commands**:
+
+```bash
+git commit --allow-empty -m "chore(release): #minor" # Minor bump
+git commit --allow-empty -m "chore(release): #major" # Major bump
+```
+
+### Major-Version Release Branches
+
+- **Branch Naming**: Stick to `release/vX` (like `release/v1`, `release/v2`).
+- **Independent Versioning**: Each branch has its own versioning and publishes to npm with a unique dist-tag (like `2x` for `release/v2`).
+
+### Manual Workflow Triggers
+
+- **Customizable Options**: You can choose the type of version bump, specify custom npm tags, and use dry run mode for testing. Dry-run is enabled by default.
+- **Special Releases**: Perfect for beta releases or when you need to publish outside the usual process.
+
+### Workflow Triggers
+
+1. **Standard Development Flow**: Merging PRs to `main` automatically triggers a patch bump and publishes with the `beta` tag.
+2. **Backporting**: Cherry-pick fixes to release branches for patch bumps with specific tags (like `2x`).
+3. **Version Bumps**: Use empty commits for minor or major bumps.
+4. **Manual Releases**: Trigger these manually for special cases like beta or release candidates.
+
+### Safety and Consistency
+
+- **Build Process**: Every publishing scenario includes a full build to ensure everything is in top shape, except for files like tests and lint rules.
+- **Tagging Safety**: We prevent overwriting the `beta` tag by enforcing custom tags for non-standard branches. The default is set to `beta`. For release branches, the tag will be picked up from the release branch itself.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@defra/forms-engine-plugin",
-  "version": "0.0.5",
+  "version": "0.1.0",
   "description": "Defra forms engine",
   "type": "module",
   "main": ".server/server/plugins/engine/index.js",
@@ -60,7 +60,6 @@
     "@hapi/vision": "^7.0.3",
     "@hapi/wreck": "^18.1.0",
     "@hapi/yar": "^11.0.2",
-    "@hapipal/schmervice": "^3.0.0",
     "@types/humanize-duration": "^3.27.4",
     "accessible-autocomplete": "^3.0.1",
     "atob": "^2.1.2",

package/src/server/index.ts

@@ -8,7 +8,6 @@ import hapi, {
 import inert from '@hapi/inert'
 import Scooter from '@hapi/scooter'
 import Wreck from '@hapi/wreck'
-import Schmervice from '@hapipal/schmervice'
 import blipp from 'blipp'
 import { ProxyAgent } from 'proxy-agent'
 
@@ -25,7 +24,6 @@ import pluginPulse from '~/src/server/plugins/pulse.js'
 import pluginRouter from '~/src/server/plugins/router.js'
 import pluginSession from '~/src/server/plugins/session.js'
 import { prepareSecureContext } from '~/src/server/secure-context.js'
-import { CacheService } from '~/src/server/services/index.js'
 import { type RouteConfig } from '~/src/server/types.js'
 
 const proxyAgent = new ProxyAgent()
@@ -94,9 +92,6 @@ export async function createServer(routeConfig?: RouteConfig) {
   await server.register(Scooter)
   await server.register(pluginBlankie)
   await server.register(pluginCrumb)
-  await server.register(Schmervice)
-
-  server.registerService(CacheService)
 
   server.ext('onPreResponse', (request: Request, h: ResponseToolkit) => {
     const { response } = request

package/src/server/plugins/engine/helpers.ts

@@ -5,7 +5,7 @@ import {
   type Page
 } from '@defra/forms-model'
 import Boom from '@hapi/boom'
-import { type ResponseToolkit } from '@hapi/hapi'
+import { type ResponseToolkit, type Server } from '@hapi/hapi'
 import { format, parseISO } from 'date-fns'
 import { StatusCodes } from 'http-status-codes'
 import { type Schema, type ValidationErrorItem } from 'joi'
@@ -362,6 +362,7 @@ export function getExponentialBackoffDelay(depth: number): number {
   const delay = BASE_DELAY_MS * 2 ** (depth - 1)
   return Math.min(delay, CAP_DELAY_MS)
 }
+
 export function evaluateTemplate(
   template: string,
   context: FormContext
@@ -377,3 +378,7 @@ export function evaluateTemplate(
     globals
   })
 }
+
+export function getCacheService(server: Server) {
+  return server.plugins['forms-engine-plugin'].cacheService
+}

package/src/server/plugins/engine/index.ts

@@ -1,7 +1,47 @@
+import { type Environment } from 'nunjucks'
+
+import { engine } from '~/src/server/plugins/engine/helpers.js'
 import { plugin } from '~/src/server/plugins/engine/plugin.js'
+import { type FilterFunction } from '~/src/server/plugins/engine/types.js'
+import {
+  checkComponentTemplates,
+  checkErrorTemplates,
+  evaluate
+} from '~/src/server/plugins/nunjucks/environment.js'
+import * as filters from '~/src/server/plugins/nunjucks/filters/index.js'
 
 export { getPageHref } from '~/src/server/plugins/engine/helpers.js'
 export { configureEnginePlugin } from '~/src/server/plugins/engine/configureEnginePlugin.js'
-export { CacheService } from '~/src/server/services/index.js'
+export { context } from '~/src/server/plugins/nunjucks/context.js'
+
+const globals = {
+  checkComponentTemplates,
+  checkErrorTemplates,
+  evaluate
+}
+
+export const VIEW_PATH = 'src/server/plugins/engine/views'
+export const PLUGIN_PATH = 'node_modules/@defra/forms-engine-plugin'
+
+export const prepareNunjucksEnvironment = function (
+  env: Environment,
+  additionalFilters?: Record<string, FilterFunction>
+) {
+  for (const [name, nunjucksFilter] of Object.entries(filters)) {
+    env.addFilter(name, nunjucksFilter)
+  }
+
+  for (const [name, nunjucksGlobal] of Object.entries(globals)) {
+    env.addGlobal(name, nunjucksGlobal)
+  }
+
+  // Apply any additional filters to both the liquid and nunjucks engines
+  if (additionalFilters) {
+    for (const [name, filter] of Object.entries(additionalFilters)) {
+      env.addFilter(name, filter)
+      engine.registerFilter(name, filter)
+    }
+  }
+}
 
 export default plugin

package/src/server/plugins/engine/pageControllers/FileUploadPageController.test.ts

@@ -4,7 +4,10 @@ import { type ResponseToolkit } from '@hapi/hapi'
 import { type ValidationErrorItem, type ValidationResult } from 'joi'
 
 import { tempItemSchema } from '~/src/server/plugins/engine/components/FileUploadField.js'
-import { getError } from '~/src/server/plugins/engine/helpers.js'
+import {
+  getCacheService,
+  getError
+} from '~/src/server/plugins/engine/helpers.js'
 import { FormModel } from '~/src/server/plugins/engine/models/FormModel.js'
 import {
   FileUploadPageController,
@@ -71,14 +74,18 @@ describe('FileUploadPageController', () => {
         trace: jest.fn(),
         level: 'info'
       },
-      services: jest.fn().mockReturnValue({
-        cacheService: {
-          setFlash: jest.fn(),
-          setState: jest
-            .fn()
-            .mockImplementation((req, updated) => Promise.resolve(updated))
+      server: {
+        plugins: {
+          'forms-engine-plugin': {
+            cacheService: {
+              setFlash: jest.fn(),
+              setState: jest
+                .fn()
+                .mockImplementation((req, updated) => Promise.resolve(updated))
+            }
+          }
         }
-      }),
+      },
       query: {}
     } as unknown as FormRequest
   })
@@ -316,7 +323,7 @@ describe('FileUploadPageController', () => {
         )
         initiateSpy.mockResolvedValue(state as never)
 
-        const { cacheService } = request.services([])
+        const cacheService = getCacheService(request.server)
         await controller['checkUploadStatus'](request, state, 1)
 
         expect(cacheService.setFlash).toHaveBeenCalledWith(request, {
@@ -608,7 +615,8 @@ describe('FileUploadPageController', () => {
               Promise.resolve(Object.assign({}, s, { newUpload: true }))
           )
 
-          const { cacheService } = request.services([])
+          const cacheService = getCacheService(request.server)
+
           await controller['checkUploadStatus'](request, state, 1)
 
           expect(cacheService.setFlash).toHaveBeenCalledWith(request, {
@@ -670,7 +678,7 @@ describe('FileUploadPageController', () => {
 
           initiateSpy.mockResolvedValue(state)
 
-          const { cacheService } = request.services([])
+          const cacheService = getCacheService(request.server)
           await controller['checkUploadStatus'](request, state, 1)
 
           expect(cacheService.setFlash).toHaveBeenCalledWith(request, {
@@ -729,7 +737,7 @@ describe('FileUploadPageController', () => {
 
           initiateSpy.mockResolvedValue(state)
 
-          const { cacheService } = request.services([])
+          const cacheService = getCacheService(request.server)
 
           await controller['checkUploadStatus'](request, state, 1)
 

package/src/server/plugins/engine/pageControllers/FileUploadPageController.ts

@@ -9,6 +9,7 @@ import {
   type FileUploadField
 } from '~/src/server/plugins/engine/components/FileUploadField.js'
 import {
+  getCacheService,
   getError,
   getExponentialBackoffDelay
 } from '~/src/server/plugins/engine/helpers.js'
@@ -364,7 +365,7 @@ export class FileUploadPageController extends QuestionPageController {
     } else {
       // Flash the error message.
       const { fileUpload } = this
-      const { cacheService } = request.services([])
+      const cacheService = getCacheService(request.server)
       const name = fileUpload.name
       const text = file.errorMessage ?? 'Unknown error'
       const errors: FormSubmissionError[] = [

package/src/server/plugins/engine/pageControllers/QuestionPageController.ts

@@ -15,6 +15,7 @@ import { ComponentCollection } from '~/src/server/plugins/engine/components/Comp
 import { optionalText } from '~/src/server/plugins/engine/components/constants.js'
 import { type BackLink } from '~/src/server/plugins/engine/components/types.js'
 import {
+  getCacheService,
   getErrors,
   normalisePath,
   proceed
@@ -298,7 +299,8 @@ export class QuestionPageController extends PageController {
       return {}
     }
 
-    const { cacheService } = request.services([])
+    const cacheService = getCacheService(request.server)
+
     return cacheService.getState(request)
   }
 
@@ -313,7 +315,8 @@ export class QuestionPageController extends PageController {
       return state
     }
 
-    const { cacheService } = request.services([])
+    const cacheService = getCacheService(request.server)
+
     return cacheService.setState(request, state)
   }
 
@@ -332,7 +335,8 @@ export class QuestionPageController extends PageController {
       return updated
     }
 
-    const { cacheService } = request.services([])
+    const cacheService = getCacheService(request.server)
+
     return cacheService.setState(request, updated)
   }
 

package/src/server/plugins/engine/pageControllers/StatusPageController.ts

@@ -1,6 +1,7 @@
 import { type PageStatus } from '@defra/forms-model'
 import { type ResponseToolkit } from '@hapi/hapi'
 
+import { getCacheService } from '~/src/server/plugins/engine/helpers.js'
 import { type FormModel } from '~/src/server/plugins/engine/models/index.js'
 import { QuestionPageController } from '~/src/server/plugins/engine/pageControllers/QuestionPageController.js'
 import { type FormContext } from '~/src/server/plugins/engine/types.js'
@@ -26,7 +27,7 @@ export class StatusPageController extends QuestionPageController {
     ) => {
       const { viewModel, viewName } = this
 
-      const { cacheService } = request.services([])
+      const cacheService = getCacheService(request.server)
       const confirmationState = await cacheService.getConfirmationState(request)
 
       // If there's no confirmation state, then

package/src/server/plugins/engine/pageControllers/SummaryPageController.ts

@@ -6,7 +6,8 @@ import { FileUploadField } from '~/src/server/plugins/engine/components/FileUplo
 import { getAnswer } from '~/src/server/plugins/engine/components/helpers.js'
 import {
   checkEmailAddressForLiveFormSubmission,
-  checkFormStatus
+  checkFormStatus,
+  getCacheService
 } from '~/src/server/plugins/engine/helpers.js'
 import {
   SummaryViewModel,
@@ -88,8 +89,8 @@ export class SummaryPageController extends QuestionPageController {
       const { model } = this
       const { params } = request
       const { state } = context
+      const cacheService = getCacheService(request.server)
 
-      const { cacheService } = request.services([])
       const { formsService } = this.model.services
       const { getFormMetadata } = formsService