ndr_lookup 0.2.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: ef8ffa13786096c8d95f8d8454d8b1e60611d71c9d54a72f57d6d64a8c13eeda
+  data.tar.gz: 87ded454a35954c619f0b785af5a1eaf6b95070d7a628f60d56dd662f3de628f
+SHA512:
+  metadata.gz: d70a25899667f9006538f119a430fff5984a62aa3e97390d0e0fa99b60da33f4084ae6c2f49527d36f84b49d848568738a206d176b46bed33b1baf83aa96a4e1
+  data.tar.gz: 68efe8f25d0763497b39d1e1e847b523a75e5c63123c4fcf4b618a14bad0d70a834d5c1e2155d14972e4be6ce6b79fff351f81ad3a0aaba966155855c881156d

data/CHANGELOG.md

@@ -0,0 +1,27 @@
+## [Unreleased]
+* no unreleased changes
+
+## 0.1.4 / 2025-09-19
+### Added
+* Added NHS FHIR ODT client
+
+## 0.1.3 / 2024-11-19
+### Fixed
+* Support Ruby 3.3, Rails 7.1, 7.2 and 8.0. Drop support for Ruby 3.0, Rails 6.1
+
+### Added
+* Support Ruby 3.2. Drop support for Ruby 2.7, Rails 6.0
+
+## 0.1.2 / 2022-12-08
+### Fixed
+* Drop support for Rails 5.2
+* Support Ruby 3.1, Rails 7.0
+
+## 0.1.1 / 2022-11-15
+### Added
+* Added NHS Digital ODS API client
+* Drop support for Ruby 2.6
+
+## 0.1.0 / Unreleased
+### Added
+* Added existing ArcGIS LocatorHub address `Rectify` client

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,74 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, gender identity and expression, level of experience,
+nationality, personal appearance, race, religion, or sexual identity and
+orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at timgentry@users.noreply.github.com. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2019 timgentry
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,117 @@
+# NdrLookup [![Build Status](https://github.com/NHSDigital/ndr_lookup/workflows/Test/badge.svg)](https://github.com/NHSDigital/ndr_lookup/actions?query=workflow%3Atest)
+
+This is the NHS Digital (NHSD) National Disease Registers (NDR) Lookup ruby gem,
+providing:
+
+1. an ArcGIS LocatorHub API client
+2. NHS Digital Organisation Data Service API Client
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'ndr_lookup'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install ndr_lookup
+
+## Usage
+
+### ArcGIS LocatorHub API Client
+
+`NdrLookup::LocatorHub::Client` is a client to access the LocatorHub API to rectify given addresses.
+
+The client isn't included by default, so add the following to your code:
+
+```ruby
+require 'ndr_lookup/locator_hub/client'
+```
+
+It requires some manual setup:
+
+Configuration | Description
+--- | ---
+`domain` | A string of the authenicating domain name.
+`username` | A string of the authenticating username.
+`password` | A string of the authenticating password.
+
+Within a Rails application, you should do this in an initializer, preferably using encrypted credentials, e.g.:
+
+```ruby
+# This file configures NdrLookup.
+Rails.application.config.to_prepare do
+  NdrLookup::LocatorHub::Client.domain = Rails.application.credentials.production[:locator_hub_api][:domain]
+  NdrLookup::LocatorHub::Client.username = Rails.application.credentials.production[:locator_hub_api][:username]
+  NdrLookup::LocatorHub::Client.password = Rails.application.credentials.production[:locator_hub_api][:password]
+end
+```
+
+### NHS Digital Organisation Data Service API Client
+
+NdrLookup::NhsdOds::Client is a client to access the NHS Digital ODS API search and sync endpoints.
+
+Sync will return a JSON format payload for all organisations that have been updated since the specified date:
+
+```ruby
+require 'ndr_lookup/nhsd_ods/client'
+
+organisation_ids = NdrLookup::NhsdOds::Client.sync(Date.new(2019, 6, 14))
+```
+
+*NOTE* that the client isn't included by default, so you have to require it specifically.
+
+Search will allow you to search based on parameters specified by ODS
+https://digital.nhs.uk/services/organisation-data-service/guidance-for-developers/search-endpoint#parameters
+
+For example:
+
+```ruby
+require 'ndr_lookup/nhsd_ods/client'
+
+results = NdrLookup::NhsdOds::Client.search(Name: 'NHS Digital')
+```
+
+NdrLookup::NhsdOds::Organisation will enable you to access the NHS Digital ODS API to find a specific organisation.
+
+Lookup will return an Organisation object with model like behaviours:
+
+```ruby
+require 'ndr_lookup/nhsd_ods/organisation'
+
+organisation = NdrLookup::NhsdOds::Organisation.find('X26')
+```
+
+NdrLookup::NhsdOds::CodeSystems will enable you to access the NHS Digital ODS API to find the meta data information included in the payload records returned via the endpoints. CodeSystems currently includes roles, relationships and record classes.
+
+```ruby
+require 'ndr_lookup/nhsd_ods/code_systems'
+
+role = NdrLookup::NhsdOds::Role.find('RO197')
+first_relationship = NdrLookup::NhsdOds::Rel.first
+all_record_classes = NdrLookup::NhsdOds::Recordclass.all
+```
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/NHSDigital/ndr_lookup. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+
+## Code of Conduct
+
+Everyone interacting in the NdrLookup project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/NHSDigital/ndr_lookup/blob/master/CODE_OF_CONDUCT.md).

data/Rakefile

@@ -0,0 +1,12 @@
+require 'bundler/gem_tasks'
+require 'rake/testtask'
+require 'ndr_dev_support/tasks'
+require 'ndr_lookup/tasks'
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << 'test'
+  t.libs << 'lib'
+  t.test_files = FileList['test/**/*_test.rb']
+end
+
+task default: :test

data/lib/ndr_lookup/fhir/base.rb

@@ -0,0 +1,111 @@
+require 'active_resource'
+require 'active_support'
+require 'active_support/core_ext'
+
+module NdrLookup
+  module Fhir
+    # Client for interacting with the NHS Digital FHIR API
+    class Base < ActiveResource::Base
+      class ApiError < StandardError; end
+      class ResourceNotFound < ApiError; end
+      class InvalidResponse < ApiError; end
+
+      class << self
+        attr_writer :additional_headers
+
+        def additional_headers
+          @additional_headers ||= {}
+        end
+
+        def sync
+          raise 'Must be defined in subclasses!'
+        end
+
+        # Finds a specific FHIR resource by type and ID
+        # @return [Hash] Parsed FHIR resource
+        def find(resource_type, id)
+          response = connection.get(
+            "#{endpoint}/#{resource_type}/#{id}",
+            headers
+          )
+          JSON.parse(response.body)
+        rescue ActiveResource::ResourceNotFound
+          raise ResourceNotFound, "#{resource_type} with ID '#{id}' not found"
+        rescue JSON::ParserError
+          raise InvalidResponse, 'Invalid JSON response from server'
+        rescue StandardError => e
+          raise ApiError, "Unexpected error: #{e.message}"
+        end
+
+        def endpoint
+          raise 'Must be defined in subclasses!'
+        end
+
+        # Searches for FHIR resources using the provided parameters
+        # @param resource_type [String] The type of FHIR resource to search for
+        # (e.g. 'Organization', 'OrganizationAffiliation')
+        # @param params [Hash] Search parameters specific to the resource type
+        # @return [Hash] FHIR Bundle containing search results
+        # @example Search for organizations
+        #   Client.search('Organization', lastUpdated: 'gt2024-01-01')
+        # @example Search for relationships
+        #   Client.search('OrganizationAffiliation', organization: 'RHAGX')
+        def search(resource_type, params = {})
+          url = construct_url(endpoint, resource_type, params)
+
+          # Make the request
+          response = connection.get(url, headers)
+
+          # Process response
+          payload = JSON.parse(response.body)
+          raise_unless_response_success(response, payload)
+
+          payload
+        rescue JSON::ParserError
+          raise InvalidResponse, 'Invalid JSON response from server'
+        rescue StandardError => e
+          raise ApiError, "Search failed: #{e.message}"
+        end
+
+        private
+
+        def construct_url(endpoint, resource_type, params = {})
+          url = "#{endpoint}/#{resource_type}"
+
+          # Add query parameters if any exist
+          if params.any?
+            # Convert params to proper query string format
+            query_string = params.map do |key, value|
+              "#{CGI.escape(key.to_s)}=#{CGI.escape(value.to_s)}"
+            end.join('&')
+
+            url = "#{url}?#{query_string}"
+          end
+
+          url
+        end
+
+        # @return [ActiveResource::Connection] Configured connection instance
+        def connection
+          @connection ||= ActiveResource::Connection.new(endpoint)
+        end
+
+        # @return [Hash] FHIR API required headers
+        def headers
+          {
+            'Accept' => 'application/fhir+json',
+            'Content-Type' => 'application/fhir+json'
+          }.merge(additional_headers)
+        end
+
+        # Raises an error unless response is successful
+        def raise_unless_response_success(response, payload)
+          return if response.code == '200'
+
+          error_message = payload['errorText'] || 'Unknown error'
+          raise ApiError, "#{payload['errorCode']} - #{error_message}"
+        end
+      end
+    end
+  end
+end

data/lib/ndr_lookup/fhir/odt/client.rb

@@ -0,0 +1,20 @@
+require_relative '../base'
+
+module NdrLookup
+  module Fhir
+    module Odt
+      # Client for interacting with the NHS Digital FHIR API
+      # See https://digital.nhs.uk/developer/api-catalogue/organisation-data-terminology
+      class Client < Base
+        FHIR_ODT_ENDPOINT = 'https://api.service.nhs.uk/organisation-data-terminology-api/fhir'.freeze
+
+        class << self
+          # Wrapped in method to enable stubbing in tests (constants are hard to stub)
+          def endpoint
+            FHIR_ODT_ENDPOINT
+          end
+        end
+      end
+    end
+  end
+end

data/lib/ndr_lookup/fhir/odt/organisation.rb

@@ -0,0 +1,88 @@
+require 'active_support'
+require 'active_resource'
+require_relative 'client'
+
+module NdrLookup
+  module Fhir
+    module Odt
+      # Represents an Organization resource from the NHS Digital FHIR API
+      # The main endpoint path we are using is 'OrganiSation' but full API path calls also include 'OrganiZation'
+      class Organisation < Base
+        self.include_format_in_path = false
+        self.site = Client::FHIR_ODT_ENDPOINT
+        self.collection_name = 'Organization'
+
+        class << self
+          # Finds a specific organization by ID
+          # @return [Organisation] The found organization
+          def find(id)
+            response = Client.find('Organization', id)
+            new(response)
+          rescue Client::ResourceNotFound => e
+            logger.info("Organization not found: #{e.message}")
+            nil
+          end
+
+          # ActiveRecord subs .all in to /all which then becomes like finding the id 'all'
+          # so we have to explicitly block this if we want it to not be a thing.
+          def all
+            raise NotImplementedError, 'Use search method instead of all'
+          end
+
+          # Searches for organizations matching the provided parameters
+          # @return [Array<Organisation>] Matching organizations
+          def search(params = {})
+            response = Client.search('Organization', params)
+            return [] unless response['entry']
+
+            response['entry'].map { |entry| new(entry['resource']) }
+          rescue Client::ApiError => e
+            logger.info("Organization search failed: #{e.message}")
+            []
+          end
+
+          def sync(date)
+            formatted_date = case date
+                             when Date, Time
+                               date.strftime('%Y-%m-%d')
+                             else
+                               date.to_s
+                             end
+
+            search_params = { _lastUpdated: "gt#{formatted_date}" }
+            search(search_params)
+          end
+
+          private
+
+          def logger
+            @logger ||= if defined?(Rails) && Rails.respond_to?(:logger)
+                          Rails.logger
+                        else
+                          Logger.new($stdout)
+                        end
+          end
+        end
+
+        # Initializes a new Organization with downcased attributes
+        def initialize(attributes = {})
+          attributes.deep_transform_keys!(&:downcase) if attributes.is_a?(Hash)
+          super
+        end
+
+        private
+
+        # Prevents Date constants from raising errors
+        # When ActiveResource tries to process something called 'Date'
+        # it would call const_valid?('Date', false).
+        # This would return false, making ActiveResource skip trying to create a Date constant and
+        # instead fall back to creating an :UnnamedResource.
+        def const_valid?(*const_args)
+          return false if const_args.first == 'Date'
+
+          super
+        end
+      end
+    end
+  end
+end

data/lib/ndr_lookup/locator_hub/client.rb

@@ -0,0 +1,83 @@
+require 'active_support/core_ext/module'
+require 'net/http'
+require 'rubyntlm'
+require 'httpi'
+require 'json'
+require_relative 'matched_record'
+
+module NdrLookup
+  module LocatorHub
+    # This class is the LocatorHub API client
+    class Client
+      LIST_LOCATORS_PATH = 'listlocators?format=json'.freeze
+
+      LOCATOR_ID = 'LocatorId'.freeze
+      LOCATOR_NAME = 'LocatorName'.freeze
+
+      ADDRESS_BASE_PREMIUM_WA = 'AddressBasePremiumWA'.freeze
+
+      # The authenticating domain credentials of the API,
+      # all of which must be configured by the host app
+      cattr_accessor :domain, :password, :username
+      self.domain = nil
+      self.password = nil
+      self.username = nil
+
+      def initialize(api_path)
+        check_credentials
+        @api_path = api_path
+      end
+
+      def locator_id
+        @locator_id ||= locator[LOCATOR_ID]
+      end
+
+      def rectify_address(address)
+        query = URI.encode_www_form(
+          Query: address,
+          Fuzzy: 'false',
+          ReturnCoordinateSystem: '-1',
+          format: 'json'
+        )
+        response = call_postcode_service("Rectify/#{locator_id}/ADDRESS?#{query}")
+        MatchedRecord.new(response)
+      end
+
+      # def capabilities
+      #   call_postcode_service("Capabilities/#{locator_id}?format=json")
+      # end
+
+      private
+
+      def listlocators
+        call_postcode_service(LIST_LOCATORS_PATH)
+      end
+
+      def locator
+        listlocators.find { |l| l[LOCATOR_NAME] == ADDRESS_BASE_PREMIUM_WA }
+      end
+
+      def request
+        @request ||= begin
+          request = HTTPI::Request.new
+          request.auth.ntlm(username, password, domain)
+          request
+        end
+      end
+
+      def call_postcode_service(path)
+        request.url = [@api_path, path].join('/')
+        response = HTTPI.get(request)
+        JSON.parse(response.body)
+      end
+
+      def check_credentials
+        unset_attributes = %i[domain password username].select { |attr| send(attr).nil? }
+        return if unset_attributes.empty?
+
+        raise "NdrLookup::LocatorHub::Client attributes #{unset_attributes.join(', ')} " \
+              'have not been configured'
+      end
+    end
+  end
+end

data/lib/ndr_lookup/locator_hub/matched_record.rb

@@ -0,0 +1,61 @@
+module NdrLookup
+  module LocatorHub
+    # This class wraps the client response, creating a hash of matched record
+    # columns and provides custom getter methods for cleaning/casting specific columns
+    class MatchedRecord
+      RECTIFY_RECORD_SCORE = 'RectifyRecordScore'.freeze
+      MATCHED_RECORD = 'MatchedRecord'.freeze
+      COLUMNS = 'Columns'.freeze
+      N = 'N'.freeze
+      R = 'R'.freeze
+
+      LOCATOR_DESCRIPTION = 'LOCATOR_DESCRIPTION'.freeze
+      ADMINISTRATIVE_AREA = 'ADMINISTRATIVE_AREA'.freeze
+      POSTCODE = 'POSTCODE'.freeze
+      UDPRN = 'DPA_UDPRN'.freeze
+
+      attr_reader :score
+
+      def initialize(response)
+        @score = response[RECTIFY_RECORD_SCORE]
+        @record = response[MATCHED_RECORD]
+      end
+
+      def locator_description
+        @locator_description ||= columns[LOCATOR_DESCRIPTION].
+                                 gsub(/\|LOCATOR_SEPARATOR\|/, ', ')
+      end
+
+      def administrative_area
+        @administrative_area ||= columns[ADMINISTRATIVE_AREA]
+      end
+
+      def locator_description_with_administrative_area
+        return locator_description if locator_description.include?(administrative_area)
+
+        parts = locator_description.split(', ')
+        (parts[0..-2] << administrative_area << parts[-1]).join(', ')
+      end
+
+      def postcode
+        @postcode ||= columns[POSTCODE]
+      end
+
+      def udprn
+        @udprn ||= columns[UDPRN] == columns[UDPRN].to_i.to_s ? columns[UDPRN].to_i : columns[UDPRN]
+      end
+
+      # keys are returned in response['MatchedRecord']['Columns']['N'] and values are returned
+      # in response['MatchedRecord']['Columns']['R'] and we turn them back into a normal hash
+      def columns
+        @columns ||= begin
+          columns = {}
+          @record && @record[COLUMNS].each.with_index do |column, i|
+            columns[column[N]] = @record[R][i]
+          end
+          columns
+        end
+      end
+    end
+  end
+end

data/lib/ndr_lookup/locator_hub/queue.rb

@@ -0,0 +1,219 @@
+require_relative 'client'
+
+module NdrLookup
+  module LocatorHub
+    # Thread queue
+    class Queue
+      BATCH_SIZE = 100
+
+      def initialize(api_path, klass, max_id, total_count)
+        @api_path = api_path
+        @klass = klass
+        @max_id = max_id
+        @total_count = total_count
+        @max_worker_count = 8
+        @worker_count = 1
+
+        queue_records
+
+        # @threads = [record_getter, max_rate_limiter, rate_limiter, record_setter, monitor]
+        @threads = [record_getter, record_setter]
+
+        t0 = Time.current
+        @max_worker_count.times { |i| @threads << worker(i) }
+
+        @threads.map(&:join)
+        Rails.logger.info "Total Time taken: #{Time.current - t0} secs"
+        Rails.logger.info "Time taken (per address): #{(Time.current - t0) / BATCH_SIZE} secs"
+      end
+
+      private
+
+      # Pretty form of address field.
+      def print_full_address(object)
+        "#{object.address&.gsub(/,/, ', ')&.titleize}, " \
+        "#{object.postcode}"
+      end
+
+      def input_queue
+        @input_queue ||= ::Queue.new
+      end
+
+      def output_queue
+        @output_queue ||= ::Queue.new
+      end
+
+      def worker(i)
+        looping_thread("worker #{i}", 120) do
+          connect_client do |locator_hub|
+            while (i + 1) > @worker_count
+              sleep 10
+              raise ThreadError if @worker_count.zero?
+            end
+            raise ThreadError if @worker_count.zero?
+
+            rec = input_queue.pop(true)
+            next if rec.address.blank? && rec.postcode.blank?
+            result = locator_hub.rectify_address(print_full_address(rec))
+            next unless result.postcode == rec.postcode
+
+            lcs_percent = lcs_commonality_percentage(print_full_address(rec),
+                                                     result.locator_description)
+
+            if result.score >= 85 || (result.score >= 70 && lcs_percent >= 80)
+              rec.udprn = result.udprn if result.udprn.present?
+              next unless rec.changed? && rec.valid?
+
+              output_queue.push(rec)
+            else
+              # rubocop:disable Rails/Output
+              puts
+              puts "locator_description: #{result.locator_description.inspect}"
+              debug_scores(rec, result, lcs_percent)
+
+              lcs_percent2 = lcs_commonality_percentage(
+                print_full_address(rec),
+                result.locator_description_with_administrative_area
+              )
+              if lcs_percent2 != lcs_percent
+                puts 'locator_description_with_administrative_area: ' +
+                     result.locator_description_with_administrative_area.inspect
+                debug_scores(rec, result, lcs_percent2)
+                # rubocop:enable Rails/Output
+              end
+              # require 'pry'; binding.pry
+            end
+          end # connect_client
+
+          Rails.logger.info 'Sleeping...'
+        end # looping_thread
+      end
+
+      def debug_scores(rec, result, lcs_percent)
+        # rubocop:disable Rails/Output
+        puts rec.id
+        puts print_full_address(rec)
+        puts "Match Score: #{result.score}"
+        puts "LCS: #{lcs_percent}%"
+        # rubocop:enable Rails/Output
+      end
+
+      # This method connects to LocatorHub, yields the connection and handles exceptions.
+      # It will supress everything except database and thread exceptions.
+      def connect_client
+        raise(ArgumentError, 'block required') unless block_given?
+
+        client = LocatorHub::Client.new(@api_path)
+        loop do
+          yield(client)
+        end
+      rescue OCIError, ThreadError => e
+        raise e
+      rescue StandardError => e
+        Rails.logger.info e.inspect
+      end
+
+      def looping_thread(name, sleep_duration)
+        Thread.new do
+          loop do
+            raise ThreadError if @worker_count.zero?
+
+            yield
+
+            sleep sleep_duration
+          end
+        rescue ThreadError
+          Rails.logger.info "#{name} thread finished!"
+        end
+      end
+
+      def monitor
+        @monitor ||= looping_thread('monitor', 1) do
+          Rails.logger.info "input_queue: #{input_queue.length} " \
+                            "output_queue: #{output_queue.length}"
+        end
+      end
+
+      def record_getter
+        @record_getter ||= looping_thread('record_getter', 1) do
+          queue_records if input_queue.length < 25
+
+          raise ThreadError if input_queue.empty?
+        end
+      end
+
+      def record_setter
+        @record_setter ||= looping_thread('record_setter', 1) do
+          persist_records if output_queue.length > BATCH_SIZE
+
+          raise ThreadError if input_queue.empty? && output_queue.empty?
+        end
+      end
+
+      def weekday?
+        (1..5).cover?(Time.current.wday)
+      end
+
+      def max_rate_limiter
+        @max_rate_limiter ||= looping_thread('max_rate_limiter', 10) do
+          @max_worker_count = ask('Max Threads: ').to_i
+        end
+      end
+
+      def rate_limiter
+        @rate_limiter ||= looping_thread('rate_limiter', 60) do
+          @worker_count = weekday? && (8..15).cover?(Time.current.hour) ? 2 : 10
+          @worker_count = @max_worker_count if @max_worker_count < @worker_count
+        end
+      end
+
+      def lcs_commonality_percentage(a, b)
+        squashed_a = a.upcase.gsub(/[,\.\s]+/, '')
+        squashed_b = b.upcase.gsub(/[,\.\s]+/, '')
+        shorter_length = [squashed_a, squashed_b].map(&:length).min
+
+        lcs = Diff::LCS.lcs(squashed_a, squashed_b)
+        100.0 * lcs.length / shorter_length
+      end
+
+      def queue_records
+        # Rails.logger.info "pushing #{BATCH_SIZE} records onto the queue"
+        records = get_records_below_id(@max_id, BATCH_SIZE)
+        records.each { |record| input_queue.push(record) }
+      end
+
+      def get_records_below_id(id, limit = BATCH_SIZE)
+        # find_in_batches ignores order
+        # records = Address.includes(:patient).where(udprn: nil).
+        records = @klass.where(udprn: nil).
+                  where("#{@klass.primary_key} < ?", id).
+                  order("#{@klass.primary_key} desc").
+                  limit(limit)
+        @max_id = records.last.id
+        records
+      end
+
+      def persist_records
+        Rails.logger.info 'persist_records'
+        @klass.transaction do
+          change_count = 0
+
+          output_queue.length.times do
+            rec = output_queue.pop(true)
+
+            # # next unless rec.changed? && rec.valid?
+            # # Rails.logger.info "CHANGES: #{rec.changes.inspect}"
+            next unless rec.save
+            change_count += 1
+            @total_count += 1
+          end
+          # rubocop:disable Rails/Output
+          puts "Change count: #{change_count} (Total: #{@total_count})" \
+               " Last id: #{@max_id}"
+          puts "worker_count: #{@worker_count} max_worker_count: #{@max_worker_count}"
+          # rubocop:enable Rails/Output
+        end
+      end
+    end
+  end
+end

data/lib/ndr_lookup/nhsd_ods/client.rb

@@ -0,0 +1,48 @@
+require 'active_resource'
+require 'active_support'
+require 'active_support/core_ext'
+require 'httpi'
+require 'json'
+
+module NdrLookup
+  module NhsdOds
+    # The API Client for hitting the sync and search endpoints
+    class Client
+      ENDPOINT = 'https://directory.spineservices.nhs.uk/ORD/2-0-0/'.freeze
+
+      class << self
+        def sync(date)
+          date = date.to_date
+          raise ArgumentError, 'invalid date' unless date.is_a?(Date)
+
+          request  = ENDPOINT + 'sync?LastChangeDate=' + date.strftime('%F')
+          response = HTTPI.get(request)
+          payload  = JSON.parse(response.body)
+
+          raise_unless_response_success(response, payload)
+
+          payload['Organisations']
+        end
+
+        def search(params = {})
+          # Expecting the keys to be valid query parameters
+          # https://digital.nhs.uk/services/organisation-data-service/guidance-for-developers/search-endpoint#parameters
+          query    = HTTPI::QueryBuilder::Flat.build(params)
+          request  = ENDPOINT + 'organisations?' + query
+          response = HTTPI.get(request)
+          payload  = JSON.parse(response.body)
+
+          raise_unless_response_success(response, payload)
+
+          payload['Organisations']
+        end
+
+        private
+
+        def raise_unless_response_success(response, payload)
+          raise "#{payload['errorCode']} - #{payload['errorText']}" unless response.code == 200
+        end
+      end
+    end
+  end
+end

data/lib/ndr_lookup/nhsd_ods/code_systems.rb

@@ -0,0 +1,31 @@
+require 'active_support'
+require 'active_resource'
+require_relative 'client'
+
+module NdrLookup
+  module NhsdOds
+    # Retrieve CodeSystems information for NHS Digital ODS meta data.
+    class CodeSystems < ActiveResource::Base
+      self.include_format_in_path = false
+      self.site = Client::ENDPOINT
+
+      def initialize(attributes, persisted = false)
+        attributes = attributes.first if attributes.is_a? Array
+        attributes.deep_transform_keys!(&:downcase)
+        super(attributes, persisted)
+      end
+    end
+
+    # Retrieve CodeSystems information for Record class meta data.
+    class Recordclass < CodeSystems
+    end
+
+    # Retrieve CodeSystems information for Relationships meta data.
+    class Rel < CodeSystems
+    end
+
+    # Retrieve CodeSystems information for Roles meta data.
+    class Role < CodeSystems
+    end
+  end
+end

data/lib/ndr_lookup/nhsd_ods/organisation.rb

@@ -0,0 +1,27 @@
+require 'active_support'
+require 'active_resource'
+require_relative 'client'
+
+module NdrLookup
+  module NhsdOds
+    # The orgaisation class to allow organisation to work like you would expect a model to work
+    # The API only supports .find <code>
+    class Organisation < ActiveResource::Base
+      self.include_format_in_path = false
+      self.site = Client::ENDPOINT
+
+      def initialize(attributes, persisted = false)
+        attributes.deep_transform_keys!(&:downcase)
+        super(attributes, persisted)
+      end
+
+      private
+
+      def const_valid?(*const_args)
+        return false if const_args.first == 'Date'
+
+        super(*const_args)
+      end
+    end
+  end
+end

data/lib/ndr_lookup/tasks.rb

@@ -0,0 +1 @@
+load 'tasks/locator_hub.rake'

data/lib/ndr_lookup/version.rb

@@ -0,0 +1,3 @@
+module NdrLookup
+  VERSION = '0.2.0'.freeze
+end

data/lib/ndr_lookup.rb

@@ -0,0 +1,14 @@
+require 'ndr_lookup/locator_hub/matched_record'
+require 'ndr_lookup/locator_hub/client'
+require 'ndr_lookup/locator_hub/queue'
+require 'ndr_lookup/nhsd_ods/code_systems'
+require 'ndr_lookup/nhsd_ods/client'
+require 'ndr_lookup/nhsd_ods/organisation'
+require 'ndr_lookup/version'
+require 'ndr_lookup/fhir/odt/client'
+require 'ndr_lookup/fhir/odt/organisation'
+
+module NdrLookup
+  class Error < StandardError; end
+  # Your code goes here...
+end

data/lib/tasks/locator_hub.rake

@@ -0,0 +1,37 @@
+namespace :locator_hub do
+  desc 'setup'
+  task :setup do
+    require 'highline/import'
+    require 'ndr_lookup/locator_hub/client'
+
+    NdrLookup::LocatorHub::Client.domain = ask('Domain name: ')
+    NdrLookup::LocatorHub::Client.username = ask('Domain Username: ')
+    NdrLookup::LocatorHub::Client.password = ask('Domain Password: ') { |q| q.echo = false }
+
+    @api_path = ask('API Path: ')
+  end
+
+  desc 'locator'
+  task locator: :setup do
+    HTTPI.log = false
+    # HTTPI.log_level = :warn
+
+    klass = Address
+    max_id = klass.where('udprn is not null').maximum(:id)
+    puts max_id
+    total_count = klass.where('udprn is not null').count
+
+    LocatorHub::Queue.new(@api_path, klass, max_id, total_count)
+  end
+
+  desc 'rectify_address'
+  task rectify_address: :setup do
+    # Usage: bundle exec rake locator_hub:rectify_address
+
+    address = ask('Address: ')
+
+    client = NdrLookup::LocatorHub::Client.new(@api_path)
+    matched_record = client.rectify_address(address)
+    puts matched_record.inspect
+  end
+end

metadata

@@ -0,0 +1,204 @@
+--- !ruby/object:Gem::Specification
+name: ndr_lookup
+version: !ruby/object:Gem::Version
+  version: 0.2.0
+platform: ruby
+authors:
+- NCRS Development Team
+autorequire:
+bindir: exe
+cert_chain: []
+date: 2025-09-25 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: activeresource
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '6.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '7'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '6.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '7'
+- !ruby/object:Gem::Dependency
+  name: activesupport
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '8.1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '8.1'
+- !ruby/object:Gem::Dependency
+  name: httpi
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '4.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '4.0'
+- !ruby/object:Gem::Dependency
+  name: rubyntlm
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.6'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.6'
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+- !ruby/object:Gem::Dependency
+  name: minitest
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.0'
+- !ruby/object:Gem::Dependency
+  name: ndr_dev_support
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '6.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '8.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '6.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '8.0'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 12.3.3
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 12.3.3
+- !ruby/object:Gem::Dependency
+  name: webmock
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+description: NDR library to consume lookup data APIs
+email: []
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- CODE_OF_CONDUCT.md
+- LICENSE.txt
+- README.md
+- Rakefile
+- lib/ndr_lookup.rb
+- lib/ndr_lookup/fhir/base.rb
+- lib/ndr_lookup/fhir/odt/client.rb
+- lib/ndr_lookup/fhir/odt/organisation.rb
+- lib/ndr_lookup/locator_hub/client.rb
+- lib/ndr_lookup/locator_hub/matched_record.rb
+- lib/ndr_lookup/locator_hub/queue.rb
+- lib/ndr_lookup/nhsd_ods/client.rb
+- lib/ndr_lookup/nhsd_ods/code_systems.rb
+- lib/ndr_lookup/nhsd_ods/organisation.rb
+- lib/ndr_lookup/tasks.rb
+- lib/ndr_lookup/version.rb
+- lib/tasks/locator_hub.rake
+homepage: https://github.com/NHSDigital/ndr_lookup
+licenses:
+- MIT
+metadata: {}
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.4.19
+signing_key:
+specification_version: 4
+summary: NDR Lookup library
+test_files: []