davinci_pas_test_kit 0.12.1 → 0.13.0

data/lib/davinci_pas_test_kit/docs/client_suite_description_v201.md

@@ -26,6 +26,13 @@ Inferno will seek to observe conformant handling of PAS requirements, including
     - The ability to handle responses that contain all PAS profiles and all must support elements on those
     profiles (not included in the current version of these tests)
 
+All requests and responses will be checked for conformance to the PAS
+IG requirements individually and used in aggregate to determine whether
+required features and functionality are present. HL7® FHIR® resources are
+validated with the Java validator using `tx.fhir.org` as the terminology server.
+
+### Responses
+
 Inferno contains basic logic to generate approval, denial, and pended responses, along with a
 notification that a final decision was made, as a part of the above workflows.
 These responses are based on examples available in the PAS Implementation Guide
@@ -34,80 +41,103 @@ testers may provide Inferno with specific responses for Inferno to echo. If resp
 are provided, Inferno will check them for conformance to ensure that they demonstrate
 a fully conformant exchange.
 
-All requests and responses will be checked for conformance to the PAS
-IG requirements individually and used in aggregate to determine whether
-required features and functionality are present. HL7® FHIR® resources are
-validated with the Java validator using `tx.fhir.org` as the terminology server.
+### Authentication
+
+The [Privacy and Security section](https://hl7.org/fhir/us/davinci-pas/STU2/privacy.html) of the PAS
+Implementation Guide states that payers must "require that the provider system authenticates"
+itself when making PAS requests against the payer system. However, the specific method of authentication
+is left to the Da Vinci HRex IG, which [provides recommendations and potential 
+approaches](https://hl7.org/fhir/us/davinci-hrex/STU1/security.html#exchange-security) for
+authentication, but does not require a specific one to be used. Inferno requires some
+authentication approach to be used in order for it to be able to identify which incoming
+requests are from the client under test.
+
+Inferno's simulated payer server includes a simulation of two standard authentication approaches:
+- SMART Backend Services
+- UDAP B2B client credentials flow, including dynamic registration
+
+Clients under test can register with the authorization server and request tokens for use
+when making PAS requests. In this case, Inferno will verify that the client's interactions with
+the simulated authorization server are conformant and that the provided tokens are used.
+
+If the client under test does not support either of these standards-based methods of authentication, the tester
+may instead attest to other authentication capabilities. In this case, the client will authenticate
+by sending requests to dedicated PAS endpoints created by Inferno for use during the testing session.
+To reduce configuration burden, the dedicated endpoints can be reused in subsequent sessions.
 
 ## Running the Tests
 
 ### Quick Start
 
-For Inferno to simulate a server that returns mocked conformant responses, it needs
-only to know the bearer token that the client will send on requests, for which there are two options.
-
-1. If you want to choose your own bearer token, then:
-    1. Select the **2.** PAS Client Validation test from the list on the left.
-    2. Click the '*Run All Tests*' button on the right.
-    3. In the "Access Token" input field, enter the bearer token that will be sent by the client under test
-       in the Authorization HTTP header (format: `Bearer <provided value>`) for all requests to Inferno.
-    4. Click the '*Submit*' button at the bottom of the dialog.
-2. If you want to use a Client ID to obtain an access token, then:
-    1. Click the '*Run All Tests*' button on the right.
-    2. Provide the client's registered id "Client ID" input field (NOTE: Inferno doesn't support the
-        registration API, so this must be obtained from another system or configured manually).
-    3. Click the '*Submit*' button at the bottom of the dialog.
-    4. Make a token request that includes the specified Client ID to the
-       `<inferno host>/custom/davinci_pas_client_suite_v201/mock_auth/token` endpoint to get
-       an access token from Inferno which the client will need to provide in the 
-       Authorization HTTP header (format: `Bearer <provided value>`) for all subsequent 
-       requests to Inferno for this test session. See the documentation in the 
-       **1.** Demonstration Authorization test for details on the supported access token request.
-
-In either case, the tests will continue from that point, requesting the tester to
-direct the client to make certain requests to demonstrate PAS client capabilities.
-
-Note: authentication options for these tests have not been finalized and are subject to change.
-
-### Postman-based Demo
+To execute a simple set of tests with minimal setup and input, perform an approval workflow using
+inferno-generated responses and dedicated session-specific endpoints with the following steps:
+
+1. Create a Da Vinci PAS Client Suite v2.0.1 session using the "Other Authentication" option
+   for the Client Security Type.
+1. Select the "Client Registration" group from the list at the left and and click
+   the "RUN TESTS" button in the upper right.
+1. Optionally provide a value for the **Session-specific URL path extension** input to
+   specify the extra path for the dedicated session endpoint or leave blank to let
+   Inferno generate a value for you. Then click the "SUBMIT" button at the bottom right.
+1. Attest to an alternate authentication approach in the wait dialog that appears and
+   then configure your client to connect to the Inferno FHIR server subsequently displayed
+   and click the link continue.
+1. Select the "Approval Workflow" group from the list at the left and click
+   the "RUN TESTS" button in the upper right.
+1. Click the "SUBMIT" button at the bottom right of the input dialog that appears.
+1. Submit a PAS prior authorization request to the endpoint shown in the wait
+   dialog that appears.
+1. When another wait dialog appears, check your system to see whether Inferno's response
+   was interpreted as an approval or not and click the appropriate link in the dialog.
+1. Review the results including any errors or warnings found when checking the conformance
+   of the request or the generated response.
+
+Group "Denial Workflow" can be run in the same manner. To run the "Pended Workflow" group,
+first run the "Subscription Setup" group, during which the client system will submit a
+Subscription so that Inferno knows how and where to send a notification that a decision has
+been rendered on a pended prior authorization request. Then proceed to execute the 
+"Pended Workflow" group and follow the instructions in the dialogs that appear.
+
+### Postman-based Demonstration
 
 If you do not have a PAS client but would like to try the tests out, you can use
-[this postman collection](https://github.com/inferno-framework/davinci-pas-test-kit/blob/main/config/PAS%20Test%20Kit%20Client%20Test%20Demo.postman_collection.json)
+[this postman collection](https://github.com/inferno-framework/davinci-pas-test-kit/blob/main/lib/davinci_pas_test_kit/docs/demo/PAS%20Client%20Suite%20Demonstration.postman_collection.json)
 to make requests against Inferno and see the mocked responses provided by Inferno. To use, load
 the collection into the [Postman app](https://www.postman.com/downloads/) and follow these steps:
 
-1. Select the *PAS Client Test Suite Demo* Collection in postman and go to the "Variables" tab 
-   (see the Overview tab for more details on what the variables control).
-1. Note the "Current value" of the **client_id** variable for use in configuring Inferno. Update it
-   to another value to use instead, if desired.
 1. Start a Da Vinci PAS Client Suite v2.0.1 session from the [PAS Test Kit page on 
-   inferno.healthit.gov](https://inferno.healthit.gov/test-kits/davinci-pas/). 
-1. Click the *Run All Tests* button in the upper right hand corner of the suite.
-1. In the **Client ID** input, enter the value from the **client_id** Postman variable and click the 
-   *Submit* button.
-1. When the "User Action" dialog appears, return to Postman and change to the Authorization tab. Scroll down 
-   to find the *Get New Access Token* button at the bottom and click it.
-1. When a success message appears, click the *Proceed* button and then the *Use Token* button.
-1. Back in Inferno, a new "User Action" dialog will appear requesting a Subscription. When it does, return to Postman,
-   open the "Create Subscription Request" entry under the "Subscription Setup" folder in the collection,
-   and click the *Send* button.
-1. Back in Inferno, an **Approval Workflow Test** "User Action" will appear. When it does, return to Postman,
-   open the "Prior Auth Request For Approval" entry under the "Approval Workflow Requests" folder in the
-   collection, and click the *Send* button.
-1. Back in Inferno, an attestation "User Action" will appear asking you to confirm that the prior auth
-   request is listed as approved in the client app based on Inferno's response to the request. Search
-   in the response returned to Postman for the string "Certified in total" which indicates the prior
-   auth request was approved and click the link in Inferno indicating the attestation statement is true.
-1. Next, a **Denial Workflow Test** "User Action" will appear. When it does, return to Postman,
-   open the "Prior Auth Request For Denial" entry under the "Denial Workflow Requests" folder in the
-   collection, and click the *Send* button.
-1. Back in Inferno, an attestation "User Action" will appear asking you to confirm that the prior auth
-   request is listed as denied in the client app based on Inferno's response to the request. Search
-   in the response returned to Postman for the string "Not Certified" which indicates the prior
-   auth request was denied and click the link in Inferno indicating the attestation statement is true.
-1. Next, a **Pended Workflow Test** "User Action" will appear. When it does, return to Postman,
-   open the "Prior Auth Request For Pended" entry under the "Pended Workflow Requests" folder in the
-   collection, and click the *Send* button.
+   inferno.healthit.gov](https://inferno.healthit.gov/test-kits/davinci-pas/),
+   choosing the "Other Authentication" option for the Client Security Type.
+2. Click the *Run All Tests* button in the upper right hand corner of the suite.
+3. Client Registration
+   - In the **Session-specific URL path extension** input, put a short alpha string, such as `demo`
+   - In Postman, select the *PAS Client Suite Demonstration* Collection in postman and go to the "Variables" tab 
+     (see the collection's Overview tab for more details on what the variables control).
+   - In the current value for the **session_url_path** variable, put the same value as in the
+     **Session-specific URL path extension** input, surrounded by `/`, e.g., `/demo/` and
+     save the collection.
+   - Back in Inferno, click the "SUBMIT" button and click the links to continue the tests
+     in the next two wait dialogs until a **Subscription Creation Test** wait dialog appears.
+1. In Postman, select the *Create Subscription Request* in the *Subscription Setup* folder
+   and click the "Send" button in the upper right.
+1. Back in Inferno, the wait dialog should disappear and a new **Approval Workflow Test** wait
+   dialog will appear.
+1. In Postman, select the *Prior Auth Request For Approval* in the *Approval Workflow* folder
+   and click the "Send" button in the upper right.
+1. Back in Inferno, the wait dialog should disappear and a new attestation wait dialog will
+   appear asking to confirm the system's interpretation of the "Approved" response. Check that
+   the response from the last step in Postman contains the string "Certified in total" and respond
+   to the attestation. The wait dialog should disappear and a new **Denial Workflow Test** wait
+   dialog will appear.
+1. In Postman, select the *Prior Auth Request For Denial* in the *Denial Workflow* folder
+   and click the "Send" button in the upper right.
+1. Back in Inferno, the wait dialog should disappear and a new attestation wait dialog will
+   appear asking to confirm the system's interpretation of the "Denied" response. Check that
+   the response from the last step in Postman contains the string "Not Certified" and respond
+   to the attestation. The wait dialog should disappear and a new **Pended Workflow Test** wait
+   dialog will appear.
+1. In Postman, select the *Prior Auth Request For Pended* entry under the *Pended Workflow* folder in the
+   and click the "Send" button in the upper right.
 1. Search in the response returned to Postman for the string "Pending" which indicates the prior
    auth request was pended and a final decision will be made later. You'll use this information in
    a later attestation.
@@ -119,31 +149,125 @@ the collection into the [Postman app](https://www.postman.com/downloads/) and fo
    hosts a notification endpoint that is used to receive Subscription notifications for this demo.
    Note that when looking for recent notifications, **Received** timestamps are in UTC which is
    5 hours ahead of Eastern Standard Time (4 hours ahead of Eastern Daylight Time).
-1. Return to Postman, open the "Prior Auth Inquiry For Pended" entry under the "Pended Workflow Requests"
-   folder in the collection, and click the *Send* button.
+1. In Postman, select the *Prior Auth Inquiry for Pended* entry under the *Pended Workflow* folder in the
+   and click the "Send" button in the upper right.
 1. Search in the response returned to Postman for the string "Certified in total" which indicates the prior
    auth request was approved. You'll use this information in a later attestation.
-1. Return to Inferno, scroll down in the "User Action" dialog and click the "click here to complete the test"
+1. Back in Inferno, scroll down in wait dialog and click the "click here to complete the test"
    link to allow Inferno to evaluate the pended workflow.
-1. Two attestations will appear, the first stating that the prior auth request was registered in the
-   client as pended and that it was subsequently finalized. You checked these above and can use the
-   true link for both.
+1. The next two attestations ask whether the system displayed the claim as pended and approved at the
+   appropriate points in the workflow. Attest based on whether the correct strings were found in the
+   responses in the previous steps.
 1. Two additional "User Action" dialogs will appear requesting additional `$submit` and `$inquire`
    requests to demonstrate must support elements. This demo does not have any additional requests
    and does not attempt to demonstrate all must support elements, so click the link to indicate
    you are done submitting requests for each. Note that requests submitted during the workflow section
-   will be evaluated and you can inspect the results under test **3.2** *Demonstrate Element Support*
+   will be evaluated and you can inspect the results under the Demonstrate Element Support test
    to see both passing and failing tests.
 1. Once Inferno finishes evaluating the requests, the test will complete allowing you to review the
    results, including warning and error messages as well as requests associated with each test.
 
-#### Optional Demo Modifications
+The tests are expected to pass with the exception of the Must Support tests.
+
+#### Optional Demo Modification: full-resource Subscription
 
 This demo uses `id-only` notifications for Pended workflow. To see a demonstration of `full-resource`
 notifications, replace the string `id-only` in the "Create Subscription Request" entry under the 
 "Subscription Setup" folder in the collection with the string `full-resource` (found in an extension
 under the `_payload` element).
 
+#### Optional Demo Modification: SMART Backend Services Auth
+
+To use SMART Backend Services with the demo, choose the "SMART Backend Services" Client Security Type
+option and replace the 3. Client Registration steps above with the following:
+
+- In the **SMART JSON Web Key Set (JWKS)** input, put `https://inferno.healthit.gov/suites/custom/smart_stu2_2/.well-known/jwks.json`
+- In the **Client Id** input, put `pas_demo_smart`
+- Click the **SUBMIT** button
+- A wait dialog will display asking the tester to confirm configuration of the client. Note the
+  FHIR endpoint and client id details
+- Start an instance of the SMART App Launch STU2.2 test suite.
+- Select the **3** Backend Services group from the list at the left and the click the "RUN TESTS"
+  button in the upper right.
+- Fill in the following input values and then click "SUBMIT":
+  - **FHIR Endpoint**: from the wait dialog in the PAS Client suite
+  - **Scopes**: any scope string, e.g., `system/*.rs`
+  - **Client Id**: same value as in the corresponding input to the PAS Client tests, also displayed
+    in the wait dialog
+- Find the access token to use for the data access request by opening test **3.2.05** Authorization
+  request succeeds when supplied correct information, click on the "REQUESTS" tab, clicking on the "DETAILS"
+  button, and expanding the "Response Body". Copy the "access_token" value, which will be a ~100 character
+  string of letters and numbers (e.g., eyJjbGllbnRfaWQiOiJzbWFydF9jbGllbnRfdGVzdF9kZW1vIiwiZXhwaXJhdGlvbiI6MTc0MzUxNDk4Mywibm9uY2UiOiJlZDI5MWIwNmZhMTE4OTc4In0)
+- In Postman, select the *PAS Client Suite Demonstration* Collection in postman and go to the "Variables" tab 
+  (see the collection's Overview tab for more details on what the variables control).
+- In the current value for the **access_token** variable, put access token value copied from the SMART tests.
+  Make sure that the **session_url_path** variable has a current value of `/`.
+- Back in Inferno, click link in the wait dialog confirming the configuration to continue the tests.
+- A **Subscription Creation Test** wait dialog will appear.
+
+Continue the tests according to step 4 around Subscription creation in the above instructions.
+
+In this demonstration, the "Verify SMART Token Requests" test will also fail due to invalid
+token requests sent intentionally by the SMART Backend Services server tests.
+
+#### Optional Demo Modification: UDAP Client Credentials Auth
+
+To use the UDAP Client Credentials with the demo, choose the "UDAP B2B Client Credentials" Client
+Security Type option and replace the 3. Client Registration steps above with the following:
+
+- In the **UDAP Client URI** input, put `http://localhost:4567/custom/udap_security/fhir`
+- Click the **SUBMIT** button and a wait dialog will display asking the tester to perform UDAP dynamic
+  registration. Note the FHIR server endpoint displayed in the dialog.
+- Start an instance of the UDAP Security Server test suite.
+- Select the "Demo: Run Against the UDAP Security Client Suite" preset from the dropdown in the upper left.
+- Select the **2** UDAP Client Credentials Flow group from the list at the left and the click the "RUN ALL TESTS"
+  button in the upper right.
+- Update the **FHIR Server Base URL** input value to be the FHIR server endpoint from the wait dialog
+  in the PAS Client suite and then click "SUBMIT"
+- Once the tests have completed, find the access token to use for the data access request by opening
+  test **2.3.01** OAuth token exchange request succeeds when supplied correct information, click
+  on the "REQUESTS" tab, clicking on the "DETAILS" button, and expanding the "Response Body".
+  Copy the "access_token" value, which will be a ~100 character string of letters and numbers (e.g., eyJjbGllbnRfaWQiOiJzbWFydF9jbGllbnRfdGVzdF9kZW1vIiwiZXhwaXJhdGlvbiI6MTc0MzUxNDk4Mywibm9uY2UiOiJlZDI5MWIwNmZhMTE4OTc4In0)
+- In Postman, select the *PAS Client Suite Demonstration* Collection in postman and go to the "Variables" tab 
+  (see the collection's Overview tab for more details on what the variables control).
+- In the current value for the **access_token** variable, put access token value copied from the SMART tests.
+  Make sure that the **session_url_path** variable has a current value of `/`.
+- In the PAS Client suite tab, click the link in the wait dialog to continue the tests. Do the same
+  for the next wait dialog that appears until a **Subscription Creation Test** wait dialog appears.
+
+Continue the tests according to step 4 around Subscription creation in the above instructions.
+
+In this demonstration, the "Verify UDAP Client Credentials Token Requests" test may fail due
+to expired signatures if the test session has taken long enough.
+
+## Auth Configuration Details
+
+When running these tests there are 3 options for authentication, which also allows 
+Inferno to identify which session the requests are for. The choice is made when the
+session is created with the selected Client Security Type option, which determines
+what details the tester needs to provide during the Client Registration tests:
+
+- **SMART Backend Services**: the system under test will manually register
+  with Inferno and request access token used to access FHIR endpoints
+  as per the SMART Backend Services specification. It requires the
+  **SMART JSON Web Key Set (JWKS)** input to be populated with either a URL that resolves
+  to a JWKS or a raw JWKS in JSON format. Additionally, testers may provide
+  a **Client Id** if they want their client assigned a specific one.
+- **UDAP B2B Client Credentials**: the system under test will dynamically register
+  with Inferno and request access tokens used to access FHIR endpoints
+  as per the UDAP specification. It requires the **UDAP Client URI** input
+  to be populated with the URI that the client will use when dynamically
+  registering with Inferno. This will be used to generate a client id (each
+  unique UDAP Client URI will always get the same client id).
+- **Other Authentication**: Inferno will create a dedicated set of FHIR endpoints for this session
+  so that the system under test does not need to get access tokens or provide
+  them when interacting with Inferno during these tests. Since PAS requires
+  authentication of client systems, testers will be asked to attest that their
+  system supports another form of authentication, such as mutual authentication TLS.
+  This approach uses the **Session-specific URL path extension** input to create a
+  session-specific URL. This input can be provided for re-use across sessions, or
+  left blank to have Inferno generate a value.
+
 ## Response and Notification Content
 
 To assist in testers getting started with the PAS Client tests quickly, Inferno will generate
@@ -172,7 +296,7 @@ know ahead of time. These modifications fall into two categories:
     then Inferno will update the focus to use the ClaimResponse id that it generates.
 
 If the tester provides an input that is malformed in some way such that Inferno cannot get the details
-that Inferno needs to make the modifications, then the raw input will be used.
+that it needs to make the modifications, then the raw input will be used.
 
 ### Response and Notification Correspondence Requirements
 
@@ -188,21 +312,21 @@ and working messages for Inferno to use.
 ### Tester-provided Response and Notification Inputs
 
 The following test inputs control Inferno messaging behavior:
-- **Claim approved response JSON**: If populated, this is used in the **2.2.1** "Demonstrate Approval Workflow" tests
+- **Claim approved response JSON**: If populated, this is used in the "Approval Workflow" group
   to respond to `$submit` requests. The response needs to indicate to the system that the prior auth request has
   been approved.
-- **Claim denied response JSON**: If populated, this used in the **2.2.2** "Demonstrate Denial Workflow" tests
+- **Claim denied response JSON**: If populated, this used in the "Denial Workflow" group
   to respond to `$submit` requests. The response needs to indicate to the system that the prior auth request has
   been denied.
-- **Claim pended response JSON**: If populated, this used in the **2.2.3** "Demonstrate Pended Workflow" tests
+- **Claim pended response JSON**: If populated, this used in the "Pended Workflow" group
   to respond to `$submit` requests. The response needs to indicate to the system that the prior auth request has
   been pended.
-- **Claim updated notification JSON**: If populated, this used in the **2.2.3** "Demonstrate Pended Workflow" tests
+- **Claim updated notification JSON**: If populated, this used in the "Pended Workflow" group
   as the event notification sent for the Subscription indicating that a decision has been finalized for the
   pended prior auth request. The content of the notification needs to match the details of the Subscription
-  provided in the **2.1** "Subscription Setup" tests.
-- **Inquire approved response JSON**: If populated, this used in the **2.2.3** "Demonstrate Pended Workflow"
-  tests to respond to `$inquire` requests. The response needs to indicate to the system that the
+  provided in the "Subscription Setup" group.
+- **Inquire approved response JSON**: If populated, this used in the "Pended Workflow"
+  group to respond to `$inquire` requests. The response needs to indicate to the system that the
   prior auth request has been approved.
 
 ### Generation Logic