@toa.io/extensions.configuration 1.0.0-alpha.291 → 1.0.0-alpha.293

package/features/ui.feature

@@ -1,93 +0,0 @@
-@ui
-Feature: Publishing the UI
-
-  The values service serves the UI on its own port, out of the directory `ui` builds. The
-  mount path is baked into the bundle and forwarded by the ingress, so the server routes
-  relative to it.
-
-  Scenario: The root is the way in
-    Given the UI is published
-    When "/" is requested
-    Then the status is 302
-    And the "location" header is "/.configuration/"
-
-  Scenario: The UI answers at its mount path
-    Given the UI is published
-    When "/.configuration" is requested
-    Then the status is 200
-    And the body contains "the page"
-
-  Scenario: A trailing slash is the same resource
-    Given the UI is published
-    When "/.configuration/" is requested
-    Then the status is 200
-    And the body contains "the page"
-
-  Scenario: A query string does not change the resource
-    Given the UI is published
-    When "/.configuration?epoch=e1" is requested
-    Then the status is 200
-    And the body contains "the page"
-
-  Scenario: An unknown route is the page
-    The client router knows what routes there are; this server does not.
-
-    Given the UI is published
-    When "/.configuration/whatever" is requested
-    Then the status is 200
-    And the body contains "the page"
-
-  Scenario: A route that looks like a file is still a route
-    A component is named `namespace.component`, and the address of its configuration
-    carries that name — so an extension is no way to tell an asset from a route.
-
-    Given the UI is published
-    When "/.configuration/dummies.dummy/" is requested
-    Then the status is 200
-    And the body contains "the page"
-
-  Scenario: And with no trailing slash to give it away
-    Given the UI is published
-    When "/.configuration/dummies.dummy" is requested
-    Then the status is 200
-    And the body contains "the page"
-
-  Scenario: A hashed asset is served forever
-    Given the UI is published
-    When "/.configuration/_app/immutable/asset.js" is requested
-    Then the status is 200
-    And the "content-type" header is "text/javascript; charset=utf-8"
-    And the "cache-control" header is "public, max-age=31536000, immutable"
-
-  Scenario: An icon is held, but not forever
-    It is named for what it is rather than for its content, so it is asked about again.
-
-    Given the UI is published
-    When "/.configuration/favicon.ico" is requested
-    Then the status is 200
-    And the "cache-control" header is "public, max-age=86400"
-
-  Scenario: The page itself is never cached
-    Given the UI is published
-    When "/.configuration/" is requested
-    Then the "cache-control" header is "no-cache"
-
-  Scenario: A missing asset is missing
-    Given the UI is published
-    When "/.configuration/_app/immutable/absent.js" is requested
-    Then the status is 404
-
-  Scenario: The path of the values API is not served by the UI
-    Given the UI is published
-    When "/configuration/values/" is requested
-    Then the status is 404
-
-  Scenario: A path that only looks like the mount path is absent
-    Given the UI is published
-    When "/.configurationable" is requested
-    Then the status is 404
-
-  Scenario: Nothing outside the site is reachable
-    Given the UI is published
-    When "/.configuration/%2e%2e/%2e%2e/package.json" is requested
-    Then the status is 404

package/notes/consistency.md

@@ -1,70 +0,0 @@
-# Configuration Consistency
-
-![Not Implemented](https://img.shields.io/badge/Not_Implemented-red)
-
-## Problem Definition
-
-At arbitrary moments when the configuration has changed, some distributed operations have started
-but haven't yet
-finished. This leads to operations that start with certain configuration values but will finish with
-another, which may
-result in logical problems.
-
-Given that distributed operations may have arbitrary participants and last for an arbitrary period,
-there is no such
-single moment when configuration can be safely changed. Thus, it must be a process.
-
-## ~~Not a~~ Solution
-
-1. Configuration is a single versioned object.
-2. Configuration version value is included in the Uniform Communication Interface.
-3. The first distributed operation participant (which is the one who has received a UCI without the
-   configuration
-   version value) must include in the outgoing UCI the configuration version value that it considers
-   as current.
-4. Non-first distributed operations participants (which is those who has received a UCI with the
-   configuration version
-   value) must use the configuration version specified in the UCI.
-5. Non-first distributed operation participants must include in the outgoing UCI the same
-   configuration version value as
-   they received in the incoming.
-
-## Restrictions
-
-1. Configuration updates are always backward compatible.
-2. Configuration updates are always being delivered before algorithm updates.
-   1. Including federated deployment, that is: first deliver configuration to all facilities (data
-      centers, zones,
-      whatever), then deliver algorithm updates.
-3. Configuration storage and access solution must provide the transactional updates, that is if any
-   arbitrary
-   participant observed a certain configuration version, then any other participant is guaranteed to
-   be able to
-   subsequently observe that version.
-   1. Including federated deployment[^1].
-
-## Solution
-
-The configuration compatibility problem described above is a particular case of a common
-compatibility problem for
-distributed operation participants. It is not the only configuration that may change while a
-distributed operation is
-running, but participant algorithms themselves. This leads to the conclusion that this particular
-solution must be
-propagated to algorithm versions, that is: which system (as a set of algorithms and configuration)
-version was used to
-start the operation, and that system version must be used to finish the operation.
-
-This kind of solution results in the need to run all versions of the system with appropriate message
-routing. The
-implementation complexity of this solution and its operations costs are considered unreasonable.
-However, an attempt
-will be made to implement this solution with a certain
-constraints [#147](https://github.com/toa-io/toa/issues/147).
-
-[^1]:
-    Since it looks like there is no reasonable way to provide this kind of guarantee without
-    significant performance
-    and/or availability impact, it may be implemented as a mechanism with read retries and with a
-    timeout considered as
-    “enough for the most of fail-over scenarios”.

package/schemas/annotation.cos.yaml

@@ -1,29 +0,0 @@
-# context.toa.yaml configuration annotation
-#
-# Every key names a component and holds that component's values, except `resources`, which
-# is the values service's own: it deploys like any other and has to state what it may take.
-# A component actually named `resources` is written with its namespace, `default.resources`
-# — the bare form is only shorthand for that.
-properties:
-  resources:
-    # `null` deploys the service without any, which every deployment has to state one way
-    # or the other — see `migrations/262.md`.
-    type: object
-    nullable: true
-    properties:
-      cpu:
-        type: array
-        items:
-          type: string
-        minItems: 2
-        maxItems: 2
-      memory:
-        type: array
-        items:
-          type: string
-        minItems: 2
-        maxItems: 2
-patternProperties:
-  ^(?!resources$).+$:
-    type: object
-type: object

package/schemas/manifest.cos.yaml

@@ -1,9 +0,0 @@
-properties:
-  schema:
-    type: object
-  defaults:
-    type: object
-additionalProperties: false
-type: object
-required:
-  - schema

package/source/Aspect.test.ts

@@ -1,119 +0,0 @@
-import { it, beforeEach, afterEach, mock } from 'node:test'
-import assert from 'node:assert/strict'
-import { isDeepStrictEqual } from 'node:util'
-
-import { Connector, Locator } from '@toa.io/core'
-import { generate } from 'randomstring'
-import { Aspect } from './Aspect.js'
-import type { Client, Listener } from './Client.js'
-import type { Manifest } from './manifest.js'
-
-class Fake extends Connector {
-  public readonly fetch = mock.fn(async () => ({
-    configuration: { foo: 'served' },
-    created: 5
-  }))
-  public readonly subscribe = mock.fn()
-  public readonly unsubscribe = mock.fn()
-}
-
-const manifest: Manifest = {
-  schema: {
-    type: 'object',
-    properties: {
-      foo: { type: 'string' },
-      bar: { type: 'object', properties: { baz: { type: 'string' } } }
-    }
-  },
-  defaults: { bar: { baz: 'quux' } }
-}
-
-let locator: Locator
-
-beforeEach(() => {
-  locator = new Locator(generate(), generate())
-})
-
-afterEach(() => {
-  delete process.env['TOA_CONFIGURATION_' + locator.uppercase]
-})
-
-it('should be named', async () => {
-  assert.deepStrictEqual(new Aspect(locator, manifest, null).name, 'configuration')
-})
-
-it('should resolve locally without a client', async () => {
-  process.env['TOA_CONFIGURATION_' + locator.uppercase] = JSON.stringify({ foo: 'local' })
-
-  const aspect = new Aspect(locator, manifest, null)
-
-  await aspect.connect()
-
-  assert.deepStrictEqual(aspect.invoke(), { foo: 'local', bar: { baz: 'quux' } })
-  assert.deepStrictEqual(aspect.invoke(['foo']), 'local')
-  assert.deepStrictEqual(aspect.invoke(['bar', 'baz']), 'quux')
-})
-
-it('should fetch from the client and follow it', async () => {
-  const client = new Fake()
-  const aspect = new Aspect(locator, manifest, client as unknown as Client)
-
-  await aspect.connect()
-
-  assert.deepStrictEqual(client.connected, true)
-  assert.strictEqual(client.fetch.mock.callCount(), 1)
-
-  const [component, epoch] = client.fetch.mock.calls[0].arguments as unknown as [
-    string,
-    string
-  ]
-
-  assert.deepStrictEqual(component, locator.id)
-  assert.match(epoch, /^[a-f0-9]{64}$/)
-  // what the service serves is what was stored, and the manifest defaults are not applied to it
-  assert.deepStrictEqual(aspect.invoke(), { foo: 'served' })
-
-  assert.ok(
-    client.subscribe.mock.calls.some(
-      (call: any) =>
-        call.arguments.length === 3 &&
-        isDeepStrictEqual(call.arguments[0], component) &&
-        isDeepStrictEqual(call.arguments[1], epoch) &&
-        typeof call.arguments[2] === 'function'
-    )
-  )
-
-  const listener = client.subscribe.mock.calls[0].arguments[2] as Listener
-
-  listener({ configuration: { foo: 'updated' }, created: 6 })
-
-  assert.deepStrictEqual(aspect.invoke(['foo']), 'updated')
-
-  // what is not newer than the held value is left alone
-  listener({ configuration: { foo: 'stale' }, created: 6 })
-  listener({ configuration: { foo: 'older' }, created: 4 })
-
-  assert.deepStrictEqual(aspect.invoke(['foo']), 'updated')
-
-  // a value that does not fit keeps the previous one
-  listener({ configuration: { foo: { nested: true } }, created: 7 })
-
-  assert.deepStrictEqual(aspect.invoke(['foo']), 'updated')
-
-  // and the one after it still applies
-  listener({ configuration: { foo: 'latest' }, created: 8 })
-
-  assert.deepStrictEqual(aspect.invoke(['foo']), 'latest')
-
-  await aspect.disconnect()
-
-  assert.ok(
-    client.unsubscribe.mock.calls.some(
-      (call: any) =>
-        call.arguments.length === 3 &&
-        isDeepStrictEqual(call.arguments[0], component) &&
-        isDeepStrictEqual(call.arguments[1], epoch) &&
-        isDeepStrictEqual(call.arguments[2], listener)
-    )
-  )
-})

package/source/Aspect.ts

@@ -1,91 +0,0 @@
-import { console } from 'openspan'
-import { Connector, type Locator } from '@toa.io/core'
-import type { extensions } from '@toa.io/core/types'
-import { fit, local, type Node } from './configuration.js'
-import { epoch } from './epoch.js'
-import type { Client, Value } from './Client.js'
-import type { Manifest } from './manifest.js'
-
-export class Aspect extends Connector implements extensions.Aspect {
-  public readonly name = 'configuration'
-
-  private readonly locator: Locator
-  private readonly manifest: Manifest
-  private readonly client: Client | null
-  private readonly epoch: string
-  private value: Node = {}
-  private created = 0
-
-  /**
-   * Without a client the value is local: the variable, the defaults and the schema.
-   * With one, the value is what the service holds, and it follows the service.
-   */
-  public constructor(locator: Locator, manifest: Manifest, client: Client | null) {
-    super()
-
-    this.locator = locator
-    this.manifest = manifest
-    this.client = client
-    this.epoch = epoch(manifest.schema)
-
-    if (client !== null) this.depends(client)
-  }
-
-  public invoke(path?: string[]): any {
-    let cursor: any = this.value
-
-    if (path !== undefined) for (const segment of path) cursor = cursor[segment]
-
-    return cursor
-  }
-
-  protected override async open(): Promise<void> {
-    if (this.client === null) {
-      this.value = local(this.locator, this.manifest)
-
-      return
-    }
-
-    const { configuration, created } = await this.client.fetch(
-      this.locator.id,
-      this.epoch
-    )
-
-    this.value = fit(configuration, this.manifest)
-    this.created = created
-    this.client.subscribe(this.locator.id, this.epoch, this.listener)
-
-    console.info('Configuration resolved', {
-      component: this.locator.id,
-      epoch: this.epoch,
-      created
-    })
-  }
-
-  protected override async close(): Promise<void> {
-    this.client?.unsubscribe(this.locator.id, this.epoch, this.listener)
-  }
-
-  private readonly listener = ({ configuration, created }: Value): void => {
-    // deliveries may repeat or cross: only what is newer than the held value replaces it
-    if (created <= this.created) return
-
-    try {
-      this.value = fit(configuration, this.manifest)
-      this.created = created
-
-      console.info('Configuration updated', {
-        component: this.locator.id,
-        epoch: this.epoch,
-        created
-      })
-    } catch (error) {
-      // the service validated it against the schema of its epoch, so the two schemas differ
-      console.error('Configuration value does not match the schema', {
-        component: this.locator.id,
-        epoch: this.epoch,
-        error
-      })
-    }
-  }
-}

package/source/Client.test.ts

@@ -1,125 +0,0 @@
-import { it, beforeEach, afterEach, mock } from 'node:test'
-import assert from 'node:assert/strict'
-import { isDeepStrictEqual } from 'node:util'
-
-import { Connector, type Locator, type Receiver } from '@toa.io/core'
-import type { Message } from '@toa.io/core/types'
-import { timeout } from '@toa.io/generic'
-import { Client, type Fetched } from './Client.js'
-import { EVENT } from './const.js'
-import type { Host } from './Factory.js'
-
-class Remote extends Connector {
-  public readonly invoke = mock.fn(
-    async (
-      _endpoint: string,
-      request: { input: Array<{ component: string; epoch: string }> }
-    ) =>
-      request.input.map((pair): Fetched => ({
-        ...pair,
-        configuration: this.values[pair.component] ?? null,
-        created: this.values[pair.component] === undefined ? 0 : 7
-      }))
-  )
-
-  public values: Record<string, object | null> = {}
-}
-
-let remote: Remote
-let receiver: Receiver | null
-let host: Host
-let client: Client
-
-beforeEach(() => {
-  remote = new Remote()
-  receiver = null
-
-  host = {
-    remote: mock.fn(async (locator: Locator) => {
-      assert.deepStrictEqual(locator.id, 'configuration.values')
-
-      return remote
-    }),
-    receive: mock.fn(async (label: string, consumer: Receiver) => {
-      assert.deepStrictEqual(label, EVENT)
-
-      receiver = consumer
-
-      return new Connector()
-    })
-  } as unknown as Host
-
-  client = new Client(host, { base: 10, max: 20, warn: 2 })
-})
-
-afterEach(async () => {
-  await client.disconnect()
-})
-
-it('should send the requests of one tick as one call', async () => {
-  remote.values = { 'a.one': { foo: 1 }, 'a.two': { foo: 2 } }
-
-  await client.connect()
-
-  const [one, two] = await Promise.all([
-    client.fetch('a.one', 'e1'),
-    client.fetch('a.two', 'e2')
-  ])
-
-  assert.deepStrictEqual(one, { configuration: { foo: 1 }, created: 7 })
-  assert.deepStrictEqual(two, { configuration: { foo: 2 }, created: 7 })
-  assert.strictEqual(remote.invoke.mock.callCount(), 1)
-  assert.deepStrictEqual(remote.invoke.mock.calls[0].arguments[1], {
-    input: [
-      { component: 'a.one', epoch: 'e1' },
-      { component: 'a.two', epoch: 'e2' }
-    ]
-  })
-})
-
-it('should keep asking until served', async () => {
-  await client.connect()
-
-  const fetching = client.fetch('a.one', 'e1')
-  const deadline = Date.now() + 1000
-
-  while (remote.invoke.mock.calls.length < 2 && Date.now() < deadline) await timeout(5)
-
-  assert.ok(remote.invoke.mock.calls.length >= 2)
-
-  remote.values = { 'a.one': { foo: 1 } }
-
-  assert.deepStrictEqual((await fetching).configuration, { foo: 1 })
-})
-
-it('should hand a created object to its subscribers', async () => {
-  await client.connect()
-
-  const listener = mock.fn()
-  const other = mock.fn()
-
-  client.subscribe('a.one', 'e1', listener)
-  client.subscribe('a.one', 'e0', other)
-
-  await receiver!.receive({
-    payload: { component: 'a.one', epoch: 'e1', configuration: { foo: 2 }, CREATED: 12 }
-  } satisfies Message)
-
-  assert.ok(
-    listener.mock.calls.some(
-      (call: any) =>
-        call.arguments.length === 1 &&
-        isDeepStrictEqual(call.arguments[0], { configuration: { foo: 2 }, created: 12 })
-    )
-  )
-  assert.strictEqual(other.mock.callCount(), 0)
-  assert.strictEqual(remote.invoke.mock.callCount(), 0)
-
-  client.unsubscribe('a.one', 'e1', listener)
-
-  await receiver!.receive({
-    payload: { component: 'a.one', epoch: 'e1', configuration: { foo: 3 }, CREATED: 13 }
-  } satisfies Message)
-
-  assert.strictEqual(listener.mock.callCount(), 1)
-})