carin_for_blue_button_test_kit 0.13.3 → 0.14.0

data/lib/carin_for_blue_button_test_kit/client/v2.0.0/required_searches_tests/patient_required_searches.rb

@@ -26,7 +26,6 @@ module CarinForBlueButtonTestKit
         * birthdate+name
         * gender+name
     )
-    input :access_token
 
     verifies_requirements 'hl7.fhir.us.carin-bb_2.0.0@14'
 

data/lib/carin_for_blue_button_test_kit/client/v2.0.0/required_searches_tests/practitioner_required_searches.rb

@@ -18,7 +18,6 @@ module CarinForBlueButtonTestKit
         * name
         * identifier
     )
-    input :access_token
 
     verifies_requirements 'hl7.fhir.us.carin-bb_2.0.0@14'
 

data/lib/carin_for_blue_button_test_kit/client/v2.0.0/required_searches_tests/relatedperson_required_searches.rb

@@ -16,7 +16,6 @@ module CarinForBlueButtonTestKit
       Capability Statements:
         * _id
     )
-    input :access_token
 
     verifies_requirements 'hl7.fhir.us.carin-bb_2.0.0@14'
 

data/lib/carin_for_blue_button_test_kit/client/v2.0.0/tags.rb

@@ -4,4 +4,6 @@ module CarinForBlueButtonTestKit
   AUTH_TAG = 'carin_auth'
   RESOURCE_API_TAG = 'carin_resource_api'
   RESOURCE_ID_TAG = 'carin_resource_id'
+  AUTHORIZE_TAG = 'carin_smart_app_authorize'
+  TOKEN_TAG = 'carin_smart_app_token'
 end

data/lib/carin_for_blue_button_test_kit/client/v2.0.0/urls.rb

@@ -1,13 +1,16 @@
 module CarinForBlueButtonTestKit
-  TOKEN_PATH = '/mock_auth/token'
-  PATIENT_PATH = '/fhir/Patient'
-  RESOURCE_API_PATH = '/fhir/:endpoint'
-  RESOURCE_ID_PATH = '/fhir/:endpoint/:id'
-  METADATA_PATH = '/fhir/metadata'
-  BASE_FHIR_PATH = '/fhir'
-  RESUME_PASS_PATH = '/resume_pass'
-  RESUME_CLAIMS_DATA_PATH = '/resume_claims_data'
-  RESUME_FAIL_PATH = '/resume_fail'
+  TOKEN_PATH = '/mock_auth/token'.freeze
+  AUTH_PATH = '/mock_auth/authorization'.freeze
+  JKWS_PATH = '/.well-known/jwks.json'.freeze
+  SMART_CONFIG_PATH = '/.well-known/smart-configuration'.freeze
+  PATIENT_PATH = '/fhir/Patient'.freeze
+  RESOURCE_API_PATH = '/fhir/:endpoint'.freeze
+  RESOURCE_ID_PATH = '/fhir/:endpoint/:id'.freeze
+  METADATA_PATH = '/fhir/metadata'.freeze
+  BASE_FHIR_PATH = '/fhir'.freeze
+  RESUME_PASS_PATH = '/resume_pass'.freeze
+  RESUME_CLAIMS_DATA_PATH = '/resume_claims_data'.freeze
+  RESUME_FAIL_PATH = '/resume_fail'.freeze
 
   module URLs
     def base_url
@@ -18,6 +21,18 @@ module CarinForBlueButtonTestKit
       @token_url ||= base_url + TOKEN_PATH
     end
 
+    def authorization_url
+      @authorization_url ||= base_url + AUTH_PATH
+    end
+
+    def jwks_url
+      @jwks_url ||= base_url + JKWS_PATH
+    end
+
+    def smart_configuration_url
+      @smart_configuration_url ||= base_url + SMART_CONFIG_PATH
+    end
+
     def base_fhir_url
       @base_fhir_url ||= base_url + BASE_FHIR_PATH
     end
@@ -51,7 +66,7 @@ module CarinForBlueButtonTestKit
     end
 
     def suite_id
-      self.class.suite.id
+      CarinForBlueButtonTestKit::C4BBV200ClientSuite.id
     end
   end
 end

data/lib/carin_for_blue_button_test_kit/docs/carin_for_blue_button_v2.0.0_client_suite_description.md

@@ -40,7 +40,8 @@ During execution, Inferno will wait for the client under test to issue requests
 
 The following input must be provided by the tester to execute
 any tests in this suite:
-1. *Access Token*: A `Bearer` token that the client under test will send in the 
+1. *Client ID*: A Client ID that the client under test will use to connect to the test suite via SMART.
+   This client ID will be sent back to the client as the `Bearer` token that the client under test will send in the 
    `Authorization` header of HTTP requests made against Inferno. Inferno uses the
    value to identify incoming requests that belong to the testing session.
 
@@ -63,7 +64,7 @@ requests needed to pass all of the tests. Note that some requests within the col
 
 To run the client tests against the Postman collection:
 1. Start an Inferno session of the CARIN for Blue Button Client test suite.
-3. Click the "Run All Tests" button in the upper right and type in "SAMPLE_TOKEN" for the `access_token` input in the dialog that appears.
+3. Click the "Run All Tests" button in the upper right and type in "SAMPLE_CLIENT_ID" for the `Client ID` input in the dialog that appears.
 4. Click the "Submit" button. The simulated server will then be waiting for an interaction.
 4. Open Postman and import the `C4BB Client Search Tests` Postman collection.
 5. Send each of the requests listed under the `C4BB Client Search Tests` Postman collection and ensure a
@@ -77,8 +78,9 @@ to associate requests with sessions. If multiple concurrent sessions are configu
 to use the same token, they may interfere with each other. To prevent concurrent executors
 of these sample executions from disrupting your session it
 is recommended, but not required, to:
-1. Update the Authorization tab of the C4BB Client Search Tests collection in Postman to a random value
-2. When starting the tests, provide the same value in the access_token input.
+1. When starting the tests, provide a random client id in the `Client ID` input.
+2. A wait dialog will appear for the first test to wait for incoming requests. This window provides the bearer token Inferno expects to receive in the Authorization header of incoming requests.
+3. Update the Authorization tab of the C4BB Client Search Tests collection in Postman to this provided bearer token.
 
 ## Current Limitations
 
@@ -91,3 +93,4 @@ Specific current limitations to highlight include:
    - Inferno's simulated CARIN server does not support all required search parameters on the ExplanationOfBenefit resource, including service-date, service-start-date, billable-period-start, type, and _include=ExplanationOfBenefit:insurer. Inferno recognizes searches made using those parameters and will give the client credit for having performed them, but will always return an OperationOutcome indicating failure. 
    - Testers must manually configure their client system to connect to a specific target patient and ingest specific curated sample CARIN data. Future versions of the tests may allow more flexibility in the patient identity and the associated data.
    - Testers must attest to their system's ability to process and retain all received information. Currently, this is implemented as a single test. Future versions of the tests may split this test out into different attestations per profile or other more fine-grained organization.
+   - This test kit contains basic SMART App Launch capabilities that may not be complete. In particular, refresh tokens are not currently supported and scopes are not precise. To provide feedback and input on the design of this feature and help us prioritize improvements, submit a ticket [here](https://github.com/inferno-framework/carin-for-blue-button-test-kit/issues).

data/lib/carin_for_blue_button_test_kit/generated/v1.1.0/c4bb_test_suite.rb

@@ -20,12 +20,20 @@ module CarinForBlueButtonTestKit
 
         
       )
-      version VERSION
+
       links [
+        {
+          label: 'Report Issue',
+          url: 'https://github.com/inferno-framework/carin-for-blue-button-test-kit/issues'
+        },
         {
           label: 'Open Source',
           url: 'https://github.com/inferno-framework/carin-for-blue-button-test-kit'
         },
+        {
+          label: 'Download',
+          url: 'https://github.com/inferno-framework/carin-for-blue-button-test-kit/releases'
+        },
         {
           label: 'Implementation Guide',
           url: 'http://hl7.org/fhir/us/carin-bb/STU1.1'

data/lib/carin_for_blue_button_test_kit/generated/v2.0.0/c4bb_test_suite.rb

@@ -21,12 +21,20 @@ module CarinForBlueButtonTestKit
 
         
       )
-      version VERSION
+
       links [
+        {
+          label: 'Report Issue',
+          url: 'https://github.com/inferno-framework/carin-for-blue-button-test-kit/issues'
+        },
         {
           label: 'Open Source',
           url: 'https://github.com/inferno-framework/carin-for-blue-button-test-kit'
         },
+        {
+          label: 'Download',
+          url: 'https://github.com/inferno-framework/carin-for-blue-button-test-kit/releases'
+        },
         {
           label: 'Implementation Guide',
           url: 'http://hl7.org/fhir/us/carin-bb/STU2'

data/lib/carin_for_blue_button_test_kit/generated/v2.0.0-dev-nonfinancial/c4bb_test_suite.rb

@@ -23,12 +23,20 @@ module CarinForBlueButtonTestKit
         Development build for the [non-financial branch](https://build.fhir.org/ig/HL7/carin-bb/branches/non-financial/).
         
       )
-      version VERSION
+
       links [
+        {
+          label: 'Report Issue',
+          url: 'https://github.com/inferno-framework/carin-for-blue-button-test-kit/issues'
+        },
         {
           label: 'Open Source',
           url: 'https://github.com/inferno-framework/carin-for-blue-button-test-kit'
         },
+        {
+          label: 'Download',
+          url: 'https://github.com/inferno-framework/carin-for-blue-button-test-kit/releases'
+        },
         {
           label: 'Implementation Guide',
           url: 'http://hl7.org/fhir/us/carin-bb/history.html'

data/lib/carin_for_blue_button_test_kit/generator/templates/group.rb.erb

@@ -0,0 +1,24 @@
+<% test_file_list.each do |file_name| %>require_relative '<%= file_name %>'
+<% end %>
+module CarinForBlueButtonTestKit
+  module <%= module_name %>
+    class <%= class_name %> < Inferno::TestGroup
+      title '<%= title %> Tests'
+      short_description '<%= short_description %>'
+      description %(
+<%=description %>
+      )
+
+      id :<%= group_id %>
+      run_as_group<% if optional? %>
+      optional
+      <% end %>
+
+      def self.metadata
+        @metadata ||= Generator::GroupMetadata.new(YAML.load_file(File.join(__dir__, '<%= profile_identifier %>', 'metadata.yml'), aliases: true))
+      end
+  <% test_id_list.each do |id| %>
+      test from: :<%= id %><% end %>
+    end
+  end
+end

data/lib/carin_for_blue_button_test_kit/generator/templates/include_search.rb.erb

@@ -0,0 +1,37 @@
+require_relative '../../../carin_search_test'
+require_relative '../../../generator/group_metadata'
+
+module CarinForBlueButtonTestKit
+  module <%= module_name %>
+    class <%= class_name %> < Inferno::Test
+      include CarinForBlueButtonTestKit::CarinSearchTest
+
+      title 'Server returns valid results for <%= resource_type %> search by id + _include=<%= search_param %>'
+      description %(<%=
+        description %>
+      )
+
+      def self.properties
+        @properties ||= SearchTestProperties.new(
+          resource_type: '<%=resource_type%>',
+        search_param_names: ['_include'],
+        include_parameters: <%= include_parameters %>
+        )
+      end
+
+      id :<%= test_id %>
+
+      def self.metadata
+        @metadata ||= Generator::GroupMetadata.new(YAML.load_file(File.join(__dir__, 'metadata.yml'), aliases: true))
+      end
+
+      def scratch_resources
+        scratch[:<%= resource_type.downcase %>_resources] ||= {}
+      end
+
+      run do
+        run_include_search('<%=include_search_param%>')
+      end
+    end
+  end
+end

data/lib/carin_for_blue_button_test_kit/generator/templates/must_support.rb.erb

@@ -0,0 +1,44 @@
+require_relative '../../../must_support_test'
+require_relative './read_test'
+require_relative '../../../generator/group_metadata'
+
+module CarinForBlueButtonTestKit
+  module <%= module_name %>
+    class <%= class_name %> < Inferno::Test
+      include CarinForBlueButtonTestKit::MustSupportTest
+      include CarinForBlueButtonTestKit::<%= module_name %>
+
+      title 'All must support elements are provided in the <%= specific_resource_type %> resources returned'
+      description %(
+        CARIN for Blue Button Responders SHALL be capable of populating all data elements as
+        part of the query results as specified by the CARIN for Blue Button Capability
+        Statement. This test will look through the <%= specific_resource_type %> resources
+        found previously for the following must support elements:
+
+        <%= must_support_list_string %><% if uscdi_list_string.present? %>
+
+        For ONC USCDI requirements, each <%= resource_type %> must support the following additional elements:
+
+        <%= uscdi_list_string %><% end %>
+      )
+
+      id :<%= test_id %>
+
+      def resource_type
+        '<%= resource_type %>'
+      end
+
+      def self.metadata
+        @metadata ||= Generator::GroupMetadata.new(YAML.load_file(File.join(__dir__, 'metadata.yml'), aliases: true))
+      end
+
+      def scratch_resources
+        scratch[:<%= resource_type.downcase %>_resources] ||= {}
+      end
+
+      run do
+        perform_must_support_test(all_scratch_resources)
+      end
+    end
+  end
+end

data/lib/carin_for_blue_button_test_kit/generator/templates/read.rb.erb

@@ -0,0 +1,38 @@
+require_relative '../../../read_test'
+require 'json'
+
+module CarinForBlueButtonTestKit
+  module <%= module_name %>
+    class <%= class_name %> < Inferno::Test
+      include CarinForBlueButtonTestKit::ReadTest
+
+      title 'Server returns correct <%= resource_type %> resource from <%= resource_type %> read interaction'
+      description 'A server <%= conformance_expectation %> support the <%= resource_type %> read interaction.'
+
+      id :<%= test_id %>
+      <% if no_search_params? %>
+      input :additional_<%= profile_identifier %>_ids,
+        title: "Additional <%= profile_identifier %> IDs",
+        type: 'text',
+        description: "<%= input_description.strip %>. This is optional, but must be provided if executing only the <%= resource_type %> test group.",
+        optional: true
+      <% end %>
+      def resource_type
+        '<%= resource_type %>'
+      end
+
+      def scratch_resources
+        scratch[:<%= resource_type.downcase %>_resources] ||= {}
+      end
+
+      def resource_ids
+        return [] unless respond_to? :additional_<%= profile_identifier%>_ids
+        additional_<%= profile_identifier%>_ids.split(',').map(&:strip)
+      end
+
+      run do
+        perform_read_test(<%= resource_collection_string %>)
+      end
+    end
+  end
+end

data/lib/carin_for_blue_button_test_kit/generator/templates/resource_list.rb.erb

@@ -0,0 +1,9 @@
+module CarinForBlueButtonTestKit
+    module <%= module_name %>
+      module ResourceList
+        RESOURCE_LIST = [
+    <%= resource_list_string %>
+        ].freeze
+        end
+    end
+end

data/lib/carin_for_blue_button_test_kit/generator/templates/search.rb.erb

@@ -0,0 +1,43 @@
+require_relative '../../../carin_search_test'
+require_relative '../../../generator/group_metadata'
+
+module CarinForBlueButtonTestKit
+  module <%= module_name %>
+    class <%= class_name %> < Inferno::Test
+      include CarinForBlueButtonTestKit::CarinSearchTest
+
+      title 'Server returns valid results for <%= resource_type %> search by <%= search_param_name_string %>'
+      description %(<%=
+        description %>
+      )
+
+      id :<%= test_id %><% if optional? %>
+      optional
+      <% end %>
+      <% if first_search? %>
+      input :<%=input_id%>,
+        title: '<%=input_title.strip%>',
+        type: 'text',
+        description: '<%=input_description.strip%>'<%if !needs_patient_id?%>,
+        optional: true<% end %>
+      <% end %>
+      def self.properties
+        @properties ||= SearchTestProperties.new(
+  <%= search_test_properties_string.gsub("\n", "\n" + " "*2) %>
+        )
+      end
+
+      def self.metadata
+        @metadata ||= Generator::GroupMetadata.new(YAML.load_file(File.join(__dir__, 'metadata.yml'), aliases: true))
+      end
+
+      def scratch_resources
+        scratch[:<%= resource_type.downcase %>_resources] ||= {}
+      end
+
+      run do
+        run_search_test<% if first_search? && !needs_patient_id? %>(<%=input_id%>)<% end %>
+      end
+    end
+  end
+end

data/lib/carin_for_blue_button_test_kit/generator/templates/suite.rb.erb

@@ -0,0 +1,92 @@
+require 'inferno/dsl/oauth_credentials'
+require 'smart_app_launch_test_kit'
+require_relative '../../version'
+
+require_relative '../../capability_statement/capability_statement_group'
+require_relative '<%= smart_launch_file_name %>'
+
+<% group_file_list.each do |file_name| %>require_relative '<%= file_name %>'
+<% end %>
+module CarinForBlueButtonTestKit
+  module <%= module_name %>
+    class <%= class_name %> < Inferno::TestSuite
+      title '<%= title %>'
+      description %(
+        The CARIN for Blue Button test suite validates system conformance to the HL7® FHIR® [CARIN for Blue Button® Implementation Guide](<%= ig_link %>).
+
+        <% if module_name.include? ("FINANCIAL") %>
+        Development build for the [non-financial branch](https://build.fhir.org/ig/HL7/carin-bb/branches/non-financial/).
+        <% end %>
+      )
+
+      links [
+        {
+          label: 'Report Issue',
+          url: 'https://github.com/inferno-framework/carin-for-blue-button-test-kit/issues'
+        },
+        {
+          label: 'Open Source',
+          url: 'https://github.com/inferno-framework/carin-for-blue-button-test-kit'
+        },
+        {
+          label: 'Download',
+          url: 'https://github.com/inferno-framework/carin-for-blue-button-test-kit/releases'
+        },
+        {
+          label: 'Implementation Guide',
+          url: '<%= ig_link %>'
+        }
+      ]
+
+      def self.metadata
+        @metadata ||= YAML.load_file(File.join(__dir__, 'metadata.yml'), aliases: true)[:groups].map do |raw_metadata|
+            Generator::GroupMetadata.new(raw_metadata)
+          end
+      end
+
+      id :<%= suite_id %>
+
+      VALIDATION_MESSAGE_FILTERS = [
+        /\A\S+: \S+: URL value '.*' does not resolve/
+      ].freeze
+
+      fhir_resource_validator do
+        igs '<%= ig_relative_path %>'
+
+        exclude_message do |message|
+          VALIDATION_MESSAGE_FILTERS.any? { |filter| filter.match? message.message }
+        end
+      end
+
+      input :url,
+        title: 'FHIR Endpoint',
+        description: 'URL of the FHIR endpoint'
+
+      fhir_client do
+        url :url
+        oauth_credentials :smart_credentials
+      end
+
+      group from: :<%= smart_launch_group_id %>
+
+      group do
+        title 'C4BB FHIR API Tests'
+        description %(
+          The CARIN for Blue Button FHIR API Tests evaluate the ability of a system (C4BB FHIR Server)
+          to support required and optional FHIR operations and search parameters to return
+          conformant C4BB resources.
+        )
+
+        input :smart_credentials,
+            title: 'OAuth Credentials',
+            type: :oauth_credentials,
+            optional: true
+        input_order :url, :smart_credentials
+
+        group from: :capability_statement_group
+    <% group_id_list.each do |id| %>
+        group from: :<%= id %><% end %>
+      end
+    end
+  end
+end

data/lib/carin_for_blue_button_test_kit/generator/templates/validation.rb.erb

@@ -0,0 +1,34 @@
+require_relative '../../../validation_test'
+require_relative './read_test'
+
+module CarinForBlueButtonTestKit
+  module <%= module_name %>
+    class <%= class_name %> < Inferno::Test
+      include CarinForBlueButtonTestKit::ValidationTest
+      include CarinForBlueButtonTestKit::<%= module_name %>
+
+      id :<%= test_id %>
+      title '<%= resource_type %> resources returned during previous tests conform to the <%= profile_name %>'
+      description %(<%=
+        description %>
+      )
+
+      output :dar_code_found, :dar_extension_found
+
+      def resource_type
+        '<%= resource_type %>'
+      end
+
+      def scratch_resources
+        scratch[:<%= resource_type.downcase %>_resources] ||= {}
+      end
+
+      run do
+        perform_validation_test(scratch_resources[:all] || [],
+                                '<%= profile_url %>',
+                                '<%= profile_version %>',
+                                skip_if_empty: <%= skip_if_empty %>)
+      end
+    end
+  end
+end

data/lib/carin_for_blue_button_test_kit/metadata.rb

@@ -0,0 +1,70 @@
+require_relative 'version'
+
+module CarinForBlueButtonTestKit
+  class Metadata < Inferno::TestKit
+    id :carin_for_blue_button_test_kit
+    title 'CARIN IG for Blue Button Test Kit'
+    description <<~DESCRIPTION
+      The Carin IG for Blue Button Test Kit validates the conformance of server
+      and client implementations to the [CARIN IG for Blue
+      Button](https://hl7.org/fhir/us/carin-bb/) including versions:
+      - [**v1.1.0**](http://hl7.org/fhir/us/carin-bb/STU1.1) (server tests only)
+      - [**v2.0.0**](http://hl7.org/fhir/us/carin-bb/STU2) (server and client tests)
+      - **Non-financial Proposal** (server tests only)
+
+      <!-- break -->
+
+      This test kit is built using the [Inferno Framework](https://inferno-framework.github.io/inferno-core/). The Inferno Framework is
+      designed for reuse and aims to make it easier to build test kits for any
+      FHIR-based data exchange.
+
+      ## Status
+
+      These tests are intended to allow CARIN IG for Blue Button server implementers to perform checks of their server against CARIN IG for Blue Button requrirements. Future versions of these tests may validate other requirements and may change how these are tested.
+
+      For more details on the specific validations performed as part of testing the following requirements, see the test descriptions within the test suites.
+
+      ### Server Tests
+
+      The server suites currently test the following requirements:
+      - SMART on FHIR Discovery
+      - SMART App Launch Protocol's standalone launch sequence
+      - Support for Capability Statement
+      - Support for Must Support Elements
+      - Profile Validation
+      - Reference Validation
+      - Searching on ExplanationofBenefit Resource
+
+      ### Client Tests
+
+      The client suite currently tests that clients are able to
+      - Fetch instances conformant to each FHIR profile defined in the CARIN IG.
+      - Perform searches using each of the required search parameters.
+      - Correctly process instances containing populated examples of all must support elements.
+
+      You can find additional information on the IG requirements that the client test suite covers
+      and does not cover in the [Requirements Coverage](https://github.com/inferno-framework/carin-for-blue-button-test-kit/blob/main/lib/carin_for_blue_button_test_kit/requirements/generated/carin-for-blue-button-test-kit_requirements_coverage.csv)
+      CSV document.
+
+      Specific known limitations of the client tests include:
+      - Inferno's simulated CARIN server does not support all required search parameters on the ExplanationOfBenefit resource, including service-date, service-start-date, billable-period-start, type, and _include=ExplanationOfBenefit:insurer. Inferno recognizes searches made using those parameters and will give the client credit for having performed them but will always return an OperationOutcome indicating failure.
+      - Testers must manually configure their client system to connect to a specific target patient and ingest specific curated sample CARIN data. Future versions of the tests may allow more flexibility in the patient identity and the associated data.
+      - Testers must attest to their system's ability to process and retain all received information. Currently, this is implemented as a single test. Future versions of the tests may split this test out into different attestations per profile or other more fine-grained organization.
+
+      ## Repository
+
+      The Carin IG for Blue Button Test Kit GitHub repository can be [found here](https://github.com/inferno-framework/carin-for-blue-button-test-kit).
+
+      ## Providing Feedback and Reporting Issues
+
+      Please report any issues with this set of tests in the [issues section](https://github.com/inferno-framework/carin-for-blue-button-test-kit/issues) of the repository.
+    DESCRIPTION
+    suite_ids [:c4bb_v110, :c4bb_v200, :c4bb_v200_client, :c4bb_v200devnonfinancial]
+    tags ['CARIN']
+    last_updated LAST_UPDATED
+    version VERSION
+    maturity 'Medium'
+    authors ['Inferno Team']
+    repo 'https://github.com/inferno-framework/carin-for-blue-button-test-kit'
+  end
+end

data/lib/carin_for_blue_button_test_kit/requirements/carin-for-blue-button-test-kit_out_of_scope_requirements.csv

@@ -0,0 +1,53 @@
+Req Set,ID,Reason,Details
+hl7.fhir.us.carin-bb_2.0.0,7,Not Verifiable,Can't know if the Health Plan API actor knows the precise reason for the absence of data
+hl7.fhir.us.carin-bb_2.0.0,15,Not Verifiable,There's no point in checking that maybe something exists that could exist.
+hl7.fhir.us.carin-bb_2.0.0,16,Not Verifiable,Nothing to verify
+hl7.fhir.us.carin-bb_2.0.0,18,Not Verifiable,There's no point in checking that maybe something exists that could exist.
+hl7.fhir.us.carin-bb_2.0.0,20,Not Verifiable,Cannot know if the a business value is an exact match or not
+hl7.fhir.us.carin-bb_2.0.0,21,Not Verifiable,There's no point in checking that maybe something exists that could exist.
+hl7.fhir.us.carin-bb_2.0.0,22,Not Verifiable,There's no point in checking that maybe something exists that could exist.
+hl7.fhir.us.carin-bb_2.0.0,23,Not Verifiable,There's no point in checking that maybe something exists that could exist.
+hl7.fhir.us.carin-bb_2.0.0,24,Not Verifiable,Doesn't specify where the descriptions would be provided.
+hl7.fhir.us.carin-bb_2.0.0,25,Not Verifiable,There's no point in checking that maybe something exists that could exist.
+hl7.fhir.us.carin-bb_2.0.0,26,Not Verifiable,There's no point in checking that maybe something exists that could exist.
+hl7.fhir.us.carin-bb_2.0.0,27,Not Verifiable,This is a description of the CPCDS dataset rather than a requirement specification for an implementer.
+hl7.fhir.us.carin-bb_2.0.0,29,Not Verifiable,This defines a mapping not a requirement on the Health Plan.
+hl7.fhir.us.carin-bb_2.0.0,49,Not Verifiable,"""Proper"" undefined"
+hl7.fhir.us.carin-bb_2.0.0,50,Not Verifiable,"""Appropriately secured"" undefined"
+hl7.fhir.us.carin-bb_2.0.0,52,Not Verifiable,We can't know agreements between trading partners
+hl7.fhir.us.carin-bb_2.0.0,59,Not Tested,"""should consider"" not strong enough to warrant testing."
+hl7.fhir.us.carin-bb_2.0.0,82,Not Tested,"Maybe - It's a ""MAY"" requirement but I think the criteria will turn it into a ""SHALL"" requirement."
+hl7.fhir.us.carin-bb_2.0.0,84,Not Verifiable,Doesn't specify where and how these will be recorded so they are not verifiable.
+hl7.fhir.us.carin-bb_2.0.0,85,Not Verifiable,"Unclear wth ""IG related data"" is"
+hl7.fhir.us.carin-bb_2.0.0,88,Not Verifiable,There's no point in checking that maybe something exists that could exist.
+hl7.fhir.us.carin-bb_2.0.0,91,Not Verifiable,Cannot know what data was effective at what time
+hl7.fhir.us.carin-bb_2.0.0,92,Not Verifiable,There's no point in checking that maybe something exists that could exist.
+hl7.fhir.us.carin-bb_2.0.0,93,Not Verifiable,Cannot know when data was last changed
+hl7.fhir.us.carin-bb_2.0.0,94,Not Verifiable,Cannot verify the internal of an app
+hl7.fhir.us.carin-bb_2.0.0,96,Not Verifiable,Overly broad
+hl7.fhir.us.carin-bb_2.0.0,104,Not Verifiable,There's no point in checking that maybe something exists that could exist.
+hl7.fhir.us.carin-bb_2.0.0,106,Not Verifiable,There's no point in checking that maybe something exists that could exist.
+hl7.fhir.us.carin-bb_2.0.0,107,Not Verifiable,MAY requirement
+hl7.fhir.us.carin-bb_2.0.0,121,Not Verifiable,Cannot test how client apps do look ups
+hl7.fhir.us.carin-bb_2.0.0,122,Not Verifiable,Cannot confirm if client apps assign value to something
+hl7.fhir.us.carin-bb_2.0.0,123,Not Verifiable,There's no point in checking that maybe something exists that could exist.
+hl7.fhir.us.carin-bb_2.0.0,126,Not Verifiable,There's no point in checking that maybe something exists that could exist.
+hl7.fhir.us.carin-bb_2.0.0,137,Not Verifiable,Cannot confirm if client apps assign value to something
+hl7.fhir.us.carin-bb_2.0.0,138,Not Verifiable,Cannot confirm how a client uses values
+hl7.fhir.us.carin-bb_2.0.0,139,Not Verifiable,Cannot confirm if client apps assign value to something
+hl7.fhir.us.carin-bb_2.0.0,140,Not Verifiable,Cannot confirm how a client uses values
+hl7.fhir.us.carin-bb_2.0.0,143,Not Verifiable,Appears to be business logic not a calculation that we can confirm
+hl7.fhir.us.carin-bb_2.0.0,147,Not Verifiable,Cannot confirm how a client uses values
+hl7.fhir.us.carin-bb_2.0.0,154,Not Verifiable,Appears to be business logic not a calculation that we can confirm
+hl7.fhir.us.carin-bb_2.0.0,155,Not Verifiable,Appears to be business logic not a calculation that we can confirm
+hl7.fhir.us.carin-bb_2.0.0,166,Not Verifiable,Cannot know how clients interpret data.
+hl7.fhir.us.carin-bb_2.0.0,167,Not Verifiable,Cannot know how clients interpret data.
+hl7.fhir.us.carin-bb_2.0.0,169,Not Verifiable,Unclear if this should *only* be populated if it's different
+hl7.fhir.us.carin-bb_2.0.0,174,Not Verifiable,"This is a definition, not a requirement."
+hl7.fhir.us.carin-bb_2.0.0,176,Not Verifiable,Cannot know when data was in effect
+hl7.fhir.us.carin-bb_2.0.0,180,Not Verifiable,Cannot confirm how a client uses values
+hl7.fhir.us.carin-bb_2.0.0,181,Not Verifiable,Business rule that we cannot verify
+hl7.fhir.us.carin-bb_2.0.0,182,Not Verifiable,Cannot know 
+hl7.fhir.us.carin-bb_2.0.0,184,Not Verifiable,Cannot confirm how a client uses values
+hl7.fhir.us.carin-bb_2.0.0,185,Not Verifiable,Cannot know 
+hl7.fhir.us.carin-bb_2.0.0,188,Not Verifiable,Cannot confirm how a client uses values