@toa.io/extensions.exposition 1.0.0-alpha.29 → 1.0.0-alpha.291

package/documentation/query.md

@@ -30,6 +30,7 @@ exposition:
         query:
           criteria: state==hot
           sort: rank:desc
+          text: true
           limit: 10
   /latest:
     GET:
@@ -47,9 +48,13 @@ Undefined `query` denies any query arguments in requests.
 
 Search criteria in [RSQL](https://github.com/jirutka/rsql-parser) format.
 
-The `criteria` property is considered as *open* when it ends with a `;`, allowing the combination of
+A value is read as what the property it selects on holds, and one that cannot be read as that is
+refused with `400 Bad Request` — `volume>abc` where `volume` is a number, or `booked==yes` where
+`booked` is a boolean. A string property takes whatever is written.
+
+The `criteria` property is considered as _open_ when it ends with a `;`, allowing the combination of
 request query criteria using `and` logic.
-Otherwise, criteria property is *closed*, that is, doesn't allow `criteria` in a request query.
+Otherwise, criteria property is _closed_, that is, doesn't allow `criteria` in a request query.
 
 ```yaml
 # manifest.toa.yaml
@@ -148,6 +153,29 @@ rank: 5
 
 > In case of conflict, path variables override input properties.
 
+## Text search
+
+For entities with `text` indexes — declared in a
+[migration](/documentation/component/declaration.md#migrations) — search queries can be enabled
+using the `search` property.
+
+```yaml
+# manifest.toa.yaml
+
+name: dummies
+
+exposition:
+  /:type:
+    GET:
+      endpoint: observe
+      query:
+        search: true
+```
+
+```yaml
+GET /dummies/?search=some+text+query
+```
+
 ## Omit, limit
 
 `omit` and `limit` properties can declare their default values and allowed boundaries:
@@ -177,6 +205,10 @@ Constant values can be declared using the shortcut:
 limit: 10
 ```
 
+```http
+GET /dummies/?omit=100&limit=10
+```
+
 ## Sort
 
 The `sort` query property defines the result order of Observations within an `objects` scope
@@ -197,7 +229,7 @@ sort: rank:asc
 sort: rank:desc;timestamp:asc
 ```
 
-If `sort` value ends with a semicolon `;` then the sorting is considered *open*
+If `sort` value ends with a semicolon `;` then the sorting is considered _open_
 and can be extended using request query `sort` argument.
 
 ```yaml
@@ -213,6 +245,8 @@ GET /dummies/?sort=timestamp:asc
 
 ## Selectors
 
+![Not implemented](https://img.shields.io/badge/Not_implemented-red)
+
 The `selectors` query property contains a list of Entity properties allowed for a client to use in
 the `criteria` and `sort` query parameters.
 If no value is provided, then no selectors are allowed.
@@ -229,12 +263,27 @@ A list of Entity properties to be included in the Observation result.
 projection: [id, title, timestamp]
 ```
 
+## Parameters
+
+By default, the only query parameters allowed are described above. Arbitrary query parameters
+can be allowed by specifying them in the `parameters` property.
+
+```yaml
+parameters: [foo, bar]
+```
+
+These parameters are embedded in the operation call input, which must be an object.
+
+```http
+GET /dummies/?foo=0&bar=baz
+```
+
 ## Optimistic concurrency control
 
-If an operation returns an object with `_version` property,
-then its value is passed as the value of
-the [`etag` header](https://datatracker.ietf.org/doc/html/rfc7232#section-2.3) in the response
-(and removed from the object).
+If an operation answers a safe request (`GET`, `HEAD`) with an object that has a `VERSION`
+property, its value is passed as the value of
+the [`etag` header](https://datatracker.ietf.org/doc/html/rfc7232#section-2.3) in the response,
+see [validators](cache.md#validators).
 
 Client can use the `if-match` request header to perform an operation only if the corresponding
 object has not been modified since the last retrieval.

package/documentation/rpc.md

@@ -0,0 +1,154 @@
+# JSON-RPC
+
+An application annotates it, and every resource it exposes answers as a procedure as well.
+
+```yaml
+exposition:
+  rpc:
+    batch: 32
+```
+
+Without it `/.rpc` is a path like any other, and nothing answers there. The path is fixed:
+one an application could choose is one it could collide with a route of its own.
+
+## The name
+
+A procedure is an RTD method — a node and a verb — and its name is the route template with a
+variable marked by a leading `_` and the verb as the last segment, joined by `.`.
+
+The separator is `.` and not `/` because a name is a tool's name too, and
+[MCP](mcp.md) names the characters one may hold: letters, digits, `_`, `-` and `.`. A client that
+receives one outside that set may refuse the tool, and one that accepts it rewrites what it shows
+a model — so what is published has to be spellable as it stands.
+
+| route                        | verb   | method                           |
+| ---------------------------- | ------ | -------------------------------- |
+| `/pots`                      | `GET`  | `pots.GET`                       |
+| `/pots`                      | `POST` | `pots.POST`                      |
+| `/pots/:id`                  | `GET`  | `pots._id.GET`                   |
+| `/identity/tokens/:identity` | `POST` | `identity.tokens._identity.POST` |
+| `/files/**`                  | `GET`  | `files.__.GET`                   |
+| `/`                          | `GET`  | `GET`                            |
+
+The verb is always the last segment, so `/pots/GET` answering `POST` is `pots.GET.POST` and
+`/pots` answering `GET` is `pots.GET`. Nothing is declared to make a name: a re-mounted route is
+a renamed procedure, because the name is the address.
+
+## What has a name
+
+A segment is addressed where it holds only `A-Z`, `a-z`, `0-9` and `-`. `_` is the name's own,
+marking a variable, and `__` the rest of a path; `.` is what divides one segment from the next.
+
+A route with a segment holding anything else — a `.`, a `_`, a `*`, a variable that is not one
+word — has no name. It is served over HTTP as it always was; what it loses is the address, so
+`/.rpc` cannot call it. Each is said once at startup, naming the route and the segment.
+
+```
+Route cannot be addressed as a procedure  route=/pots/v1.0 segment=v1.0
+```
+
+## The parameters
+
+`params` is by name, and carries what a request carries.
+
+A key the template names is a route variable, and is taken by the path. `query` is the
+querystring. Whatever is left is the body.
+
+```json
+{ "jsonrpc": "2.0", "id": 1, "method": "pots._id.GET", "params": { "id": "a1b2" } }
+```
+
+```json
+{
+  "jsonrpc": "2.0",
+  "id": 2,
+  "method": "pots.POST",
+  "params": { "title": "Kettle", "volume": 1.7 }
+}
+```
+
+```json
+{
+  "jsonrpc": "2.0",
+  "id": 3,
+  "method": "pots.GET",
+  "params": { "query": { "criteria": "volume=gt=1", "limit": 10 } }
+}
+```
+
+The querystring has a name of its own because an operation's input is free to have an `id` or
+a `limit`, and the two would otherwise be one object. An operation whose input has a property
+named `query` cannot be called with one.
+
+A variable stands for a single segment, so its value may not contain `/`, `?` or `#`. `__`
+stands for the rest of the path and may contain `/`; its value is read under the name `**`,
+which is what the RTD calls it wherever else it is read.
+
+## What answers
+
+The reply is the operation's, restricted by the same [`io:output`](io.md) as the resource's,
+and encoded as the request's `accept` asks. A call that returned nothing answers `result: null`
+— JSON-RPC has no 404, and nothing went wrong.
+
+A call with no `id` is a notification: it runs and answers nothing, and a request carrying only
+notifications answers `204`.
+
+## Several at once
+
+A request may carry an array of calls instead of one, and answers an array of what they
+answered — shorter than what it was given, where some were notifications.
+
+```json
+[
+  { "jsonrpc": "2.0", "id": 1, "method": "pots._id.GET", "params": { "id": "a1b2" } },
+  { "jsonrpc": "2.0", "method": "pots._id.DELETE", "params": { "id": "c3d4" } }
+]
+```
+
+They run one after another, in the order given. `batch` is how many one request may carry, 32
+where the annotation does not say: a request is one authentication and one reply however many
+calls it makes, so what a caller may ask for at once is bounded.
+
+## What refuses
+
+The whole request is refused, with a status, when it is not a request this endpoint serves: a
+verb other than `POST` is `405`, an unreadable body is `400`, and an array that is empty or
+longer than `batch` is `400`.
+
+A call that is not one — a version it does not state, a method it does not name — is answered
+to nobody: `id` is `null`, because nothing about it is trustworthy enough to answer to.
+
+A credential is the request's, so a call cannot be refused for one: a request that presents
+none where the procedure requires one is `401`, and carries the challenge that says where to
+authenticate.
+
+Everything else a call runs into is a value, at `200`, because the request itself succeeded.
+
+|          |                                                    |
+| -------- | -------------------------------------------------- |
+| `-32601` | no route of that name, or no such verb on it       |
+| `-32602` | a variable is missing, or a query is not one       |
+| `-32000` | the identity is not authorized to make this call   |
+| `-32001` | the operation refused, its own code in `data.code` |
+| `-32603` | anything the gateway did not mean to answer        |
+| `-32002` | the request carries more calls than `batch`        |
+
+The first five codes JSON-RPC states mean the same to every client. `-32000` and `-32001` are
+this gateway's, from the block the specification reserves and leaves empty, so
+`-32001` says only that an operation refused — `data.code` is the code that operation declares
+in its manifest, and is what a caller reads.
+
+## Authorization
+
+A procedure is authorized as the resource is, against the path and the verb its name states.
+A token restricted to `/pots/:id/` for `GET` authorizes `pots._id.GET` and nothing else,
+whichever way the call arrives.
+
+[`anonymous`](access.md#anonymous) is the one directive that reads a call differently from a
+request: it admits one whatever the request presented, because what refuses a credentialed request
+is that the reply would not be cacheable, and what a call answers is not a reply.
+
+## What has no procedure
+
+`map:buffer` reads the request, and a call is not the request. A resource that declares it
+answers over HTTP alone.

package/documentation/tracing.md

@@ -0,0 +1,70 @@
+# Tracing
+
+Each incoming request is processed within a [trace](../../telemetry/readme.md#tracing) —
+a server span is created, and its trace context is propagated to component invocations and
+emitted events, so all log entries and spans across the application caused by the request
+share the same `trace_id`.
+
+## The `ray` response header
+
+Each response contains a `ray` header with the trace ID of the request.
+It can be used to find related log entries, or be attached to a support ticket.
+
+Log entries are stamped with the trace ID regardless of the sampling decision,
+while the trace itself is only recorded if the trace is sampled (see [Sampling](#sampling)).
+
+```
+HTTP/1.1 201 Created
+ray: 0af7651916cd43dd8448eb211c80319c
+```
+
+## Continuing a trace
+
+By default, the trace starts at the gateway. If the request carries a trace context in one of the
+following headers, the trace is continued, that is, the incoming trace ID is used.
+
+### `traceparent`
+
+The [W3C Trace Context](https://www.w3.org/TR/trace-context/#traceparent-header) header, sent by
+clients instrumented with OpenTelemetry-compatible tooling, or by upstream proxies.
+
+```
+traceparent: 00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01
+```
+
+In addition to the trace ID, it carries the ID of the client span, which becomes the _parent_ of
+the gateway's server span, linking client-side and server-side spans into a single trace tree.
+
+### `ray`
+
+A bare trace ID (32 lowercase hex digits), for clients without tracing instrumentation willing to
+correlate their requests with server-side traces.
+
+```
+ray: 4bf92f3577b34da6a3ce929d0e0e4736
+```
+
+Unlike `traceparent`, it carries no parent span, so the gateway's server span becomes the root of
+the trace.
+
+If both headers are present, `traceparent` takes precedence as it carries more information.
+Headers with invalid values are ignored, and a new trace is started.
+
+## Sampling
+
+The trace _context_ (trace ID) is always created and propagated, so the `ray` header and
+`trace_id` in log entries are always present. The sampling decision only determines whether
+the _spans_ of the trace are recorded.
+
+The decision is made once, by the process that starts the trace, and is propagated along with
+the trace context (the flags byte of `traceparent`), so a trace is either recorded as a whole
+or not at all. Sampling is configured with the
+[`telemetry.traces` annotation](../../telemetry/readme.md#sampling).
+
+Sending a trace context does not bypass sampling: an incoming `ray` header adopts the trace ID,
+while the recording decision is still made by the gateway. The `sampled` flag of an incoming
+`traceparent` is respected, as the decision belongs to the trace root.
+
+## See also
+
+- [Telemetry extension](../../telemetry/readme.md) — spans, structured logs, `context.span`

package/documentation/tree.md

@@ -56,8 +56,24 @@ as it provides a more specific match compared to the generic `/users/:id` route.
 
 The priority of Routes with the same specificity is determined by the order of declaration.
 
+## Route forwarding
+
+A Route can be forwarded to another Route by specifying the destination Route as the value of the
+Route.
+
+```yaml
+/destination/:var: ...
+/static: /destination/hello
+/variables/:bar: /destination/:bar
+```
+
+Forwarding Route variables are mapped to the forwarded Route variables if they have the same name.
+
 ## Methods
 
+Every Method is also addressable as a procedure, where the context annotates
+[JSON-RPC](rpc.md).
+
 Methods are mappings of the HTTP methods to the corresponding operations.
 
 A Method is a key named after the corresponding HTTP method, with a value following the schema
@@ -96,7 +112,7 @@ key.
 HTTP methods can only be mapped to operations of the corresponding types.
 
 | HTTP method | Operation type                                |
-|-------------|-----------------------------------------------|
+| ----------- | --------------------------------------------- |
 | `POST`      | **Transition** (without Query)<br/>**Effect** |
 | `PUT`       | **Transition** (with Query)<br/>**Effect**    |
 | `GET`       | **Observation**<br/>**Computation**           |

package/features/access.feature

@@ -52,6 +52,56 @@ Feature: Access authorization
       access: granted!
       """
 
+  Scenario: Allow access to anyone
+    Given the annotation:
+      """yaml
+      /:
+        io:output: true
+        auth:anyone: true
+        GET:
+          dev:stub:
+            access: granted!
+      """
+    When the following request is received:
+      """
+      GET / HTTP/1.1
+      host: nex.toa.io
+      authorization: Basic ZGV2ZWxvcGVyOnNlY3JldA==
+      accept: application/yaml
+      """
+    Then the following reply is sent:
+      """
+      200 OK
+      content-type: application/yaml
+
+      access: granted!
+      """
+
+    # request without credentials
+    When the following request is received:
+      """
+      GET / HTTP/1.1
+      host: nex.toa.io
+      accept: application/yaml
+      """
+    Then the following reply is sent:
+      """
+      401 Unauthorized
+      """
+
+    # request with invalid credentials
+    When the following request is received:
+      """
+      GET / HTTP/1.1
+      host: nex.toa.io
+      authorization: Basic bm9uOmV4aXN0ZW50
+      accept: application/yaml
+      """
+    Then the following reply is sent:
+      """
+      401 Unauthorized
+      """
+
   Scenario: Deny access with credentials to a resource with anonymous access
     Given the annotation:
       """yaml
@@ -364,8 +414,8 @@ Feature: Access authorization
     Then the following reply is sent:
       """
       200 OK
-      content-type: application/yaml
       authorization: Token ${{ token }}
+      content-type: application/yaml
 
       access: granted!
       """
@@ -421,12 +471,12 @@ Feature: Access authorization
       """
       403 Forbidden
 
-      Basic authentication scheme is required to access this resource.
+      Basic authentication scheme is required to access this resource
       """
 
   Scenario: Adding a role without required permissions
 
-  Trunk directives should not be applied to the Identity management resources.
+  Trunk directives should not be applied to the Identity management resources
 
     Given the annotation:
       """yaml
@@ -468,9 +518,79 @@ Feature: Access authorization
     Then the following reply is sent:
       """
       200 OK
+      authorization: Token ${{ token }}
 
       identity:
         id: efe3a65ebbee47ed95a73edd911ea328
         roles:
           - developer
       """
+    When the following request is received:
+      """
+      GET /echo/ HTTP/1.1
+      host: nex.toa.io
+      authorization: Token ${{ token }}
+      """
+    Then the following reply is sent:
+      """
+      200 OK
+      """
+    And the reply does not contain:
+      """
+      authorization: Token
+      """
+
+  Scenario: Delegation does not violate input restrictions
+    Given the `identity.roles` database contains:
+      | _id                              | identity                         | role      |
+      | 775a648d054e4ce1a65f8f17e5b51803 | efe3a65ebbee47ed95a73edd911ea328 | developer |
+    # `auth:` is declared before `io:`, and the identity it embeds is the server's
+    # own addition — what `io:input` restricts is the body the client sent
+    And the `echo` is running with the following manifest:
+      """yaml
+      exposition:
+        /:
+          POST:
+            auth:rule:
+              role: developer
+              delegate: identity
+            io:input: [foo]
+            io:output: true
+            endpoint: echo
+      """
+    When the following request is received:
+      """
+      POST /echo/ HTTP/1.1
+      host: nex.toa.io
+      authorization: Basic ZGV2ZWxvcGVyOnNlY3JldA==
+      accept: application/yaml
+      content-type: application/yaml
+
+      foo: bar
+      """
+    Then the following reply is sent:
+      """
+      201 Created
+
+      foo: bar
+      identity:
+        id: efe3a65ebbee47ed95a73edd911ea328
+      """
+    # and a property the client is not allowed to send is still refused
+    When the following request is received:
+      """
+      POST /echo/ HTTP/1.1
+      host: nex.toa.io
+      authorization: Basic ZGV2ZWxvcGVyOnNlY3JldA==
+      accept: application/yaml
+      content-type: application/yaml
+
+      foo: bar
+      baz: qux
+      """
+    Then the following reply is sent:
+      """
+      400 Bad Request
+
+      'Unexpected input: baz'
+      """

package/features/auth.assert.feature

@@ -0,0 +1,55 @@
+Feature: Identity assertion
+
+  Scenario: Assert Identity
+    Given the `identity.basic` database contains:
+      | _id                              | authority | username  | password                                                     | VERSION |
+      | efe3a65ebbee47ed95a73edd911ea328 | nex       | developer | $2b$10$ZRSKkgZoGnrcTNA5w5eCcu3pxDzdTduhteVYXcp56AaNcilNkwJ.O | 1       |
+    And the `echo` is running with the following manifest:
+      """yaml
+      exposition:
+        /:
+          GET:
+            auth:assert: true
+            anyone: true
+            io:output: true
+            endpoint: ping
+      """
+
+    # existent identity
+    When the following request is received:
+      """
+      GET /echo/ HTTP/1.1
+      host: nex.toa.io
+      authorization: Basic ZGV2ZWxvcGVyOnNlY3JldA==
+      """
+    Then the following reply is sent:
+      """
+      200 OK
+      authorization: Token ${{ developer.token }}
+      """
+
+    # new identity
+    When the following request is received:
+      """
+      GET /echo/ HTTP/1.1
+      host: nex.toa.io
+      authorization: Basic dXNlcjpwYSQkdzByZCE=
+      """
+    Then the following reply is sent:
+      """
+      201 Created
+      authorization: Token ${{ user.token }}
+      """
+
+    # wrong password for the credentials just created
+    When the following request is received:
+      """
+      GET /echo/ HTTP/1.1
+      host: nex.toa.io
+      authorization: Basic dXNlcjpwYXNz
+      accept: application/yaml
+      """
+    Then the following reply is sent:
+      """
+      401 Unauthorized
+      """