interscript 0.1.9 → 2.0.5

data/README.adoc

@@ -1,296 +0,0 @@
-= Interscript: Interoperable Script Conversion Systems, with a Ruby implementation
-
-image:https://github.com/interscript/interscript/workflows/test/badge.svg["Ruby build status", link="https://github.com/interscript/interscript/actions?workflow=test"]
-image:https://github.com/interscript/interscript/workflows/js/badge.svg["JavaScript build status", link="https://github.com/interscript/interscript/actions?workflow=js"]
-
-== Introduction
-
-This repository contains interoperable transliteration schemes from:
-
-* ALA-LC
-* BGN/PCGN
-* ICAO
-* ISO
-* UN (by UNGEGN)
-* Many, many other script conversion system authorities.
-
-The goal is to achieve interoperable transliteration schemes allowing quality comparisons.
-
-
-
-== Demonstration
-
-These transliteration systems are used in the demo:
-
-`bgnpcgn-rus-Cyrl-Latn-1947`:: BGN/PCGN Romanization of Russian
-`iso-rus-Cyrl-Latn-9-1995`::     ISO 9 Romanization of Russian
-`icao-rus-Cyrl-Latn-9303`::    ICAO MRZ Romanization of Russian
-`bas-rus-Cyrl-Latn-2017-bss`::      Bulgaria Academy of Science Streamlined System for Russian
-
-image:demo/20191118-interscript-demo-cast.gif["interscript screencast"]
-
-
-== Installation
-
-=== Prerequisites
-
-Linux:
-
-[source,sh]
-----
-apt-get install swig python3-setuptools
-----
-
-Windows:
-
-[source,sh]
-----
-choco install --no-progress swig
-----
-
-Interscript depends on Python and the https://github.com/sequitur-g2p/sequitur-g2p[`sequitur-g2p`] module
-
-[source,sh]
-----
-pip3 install -r requirments.txt
-----
-
-Interscript depends on Ruby. Once you manage to install Ruby, it's easy.
-
-[source,sh]
-----
-gem install interscript
-----
-
-== Usage
-
-Assume you have a file ready in the source script like this:
-
-[source,sh]
-----
-cat <<EOT > rus-Cyrl.txt
-Эх, тройка! птица тройка, кто тебя выдумал? знать, у бойкого народа ты
-могла только родиться, в той земле, что не любит шутить, а
-ровнем-гладнем разметнулась на полсвета, да и ступай считать версты,
-пока не зарябит тебе в очи. И не хитрый, кажись, дорожный снаряд, не
-железным схвачен винтом, а наскоро живьём с одним топором да долотом
-снарядил и собрал тебя ярославский расторопный мужик. Не в немецких
-ботфортах ямщик: борода да рукавицы, и сидит чёрт знает на чём; а
-привстал, да замахнулся, да затянул песню — кони вихрем, спицы в
-колесах смешались в один гладкий круг, только дрогнула дорога, да
-вскрикнул в испуге остановившийся пешеход — и вон она понеслась,
-понеслась, понеслась!
-
-Н.В. Гоголь
-EOT
-----
-
-You can run `interscript` on this text using different transliteration systems.
-
-[source,sh]
-----
-interscript rus-Cyrl.txt \
-  --system=bgnpcgn-rus-Cyrl-Latn-1947 \
-  --output=bgnpcgn-rus-Latn.txt
-
-interscript rus-Cyrl.txt \
-  --system=iso-rus-Cyrl-Latn-9-1995 \
-  --output=iso-rus-Latn.txt
-
-interscript rus-Cyrl.txt \
-  --system=icao-rus-Cyrl-Latn-9303 \
-  --output=icao-rus-Latn.txt
-
-interscript rus-Cyrl.txt \
-  --system=bas-rus-Cyrl-Latn-2017-bss \
-  --output=bas-rus-Latn.txt
-----
-
-It is then easy to see the exact differences in rendering between the systems.
-
-[source,sh]
-----
-diff bgnpcgn-rus-Latn.txt bas-rus-Latn.txt
-----
-
-== Adding transliteration system
-
-Transliteration systems stored in a `maps/` directory as YAML files.
-You can create a new file and add it to the directory.
-
-The file should be named as `<system-code>.yaml`, where `system-code`
-is in accordance with
-http://calconnect.gitlab.io/tc-localization/csd-transcription-systems[ISO/CC 24229].
-
-=== File structure
-
-[source,yaml]
-----
-authority_id: bgnpcgn
-id: 1947
-language: rus
-source_script: Cyrl
-destination_script: Latn
-name: ROMANIZATION OF RUSSIAN, BGN/PCGN 1947 System
-url: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/807920/ROMANIZATION_OF_RUSSIAN.pdf
-creation_date: 1947
-confirmation_date: 2019-06
-description: The BGN/PCGN system for Russian was adopted ...
-
-notes:
-  - The character e should be romanized ye initially, after the vowel ...
-
-tests:
-  - source: ДЛИННОЕ ПОКРЫВАЛО
-    expected: DLINNOYE POKRYVALO
-  - source: Еловая шишка
-    expected: Yelovaya shishka
-
-map:
-  rules:
-    - pattern: (?<=[АаЕеЁёИиОоУуЫыЭэЮюЯяЙйЪъЬь])\u0415 # Е after a, e, ё, и, о, у, ы, э, ю, я, й, ъ, ь
-      result: Ye
-    - pattern: \b\u0415 # Е initially
-      result: Ye
-
-  characters:
-    "\u0410": "A"
-    "\u0411": "B"
-    "\u0412": "V"
-----
-
-
-=== Rules
-
-The subsection `rules` is placed under the `map` key. All rules are applied in order they are placed before the subsection `characters` applying. Rules apply to an original text, not to a result of previous rules applying.
-
-Each rule has `pattern` and `result` elements.
-
-Pattern is a regex expression. It should be representing as a string without `//` or `%r{}` parentheses. For example `\b\u0415`. In case a rule is depend on previous or next content, lookahead or lookbehind could be used. For example a rule with the pattern `(?<=[АаЕеЁёИиОоУуЫыЭэЮюЯяЙйЪъЬь])\u0415` find every Е after upper or lower case symbols a, e, ё, и, о, у, ы, э, ю, я, й, ъ, ь.
-
-Result is a replacement a for pattern's match. It can contain a string, an Unicode characters specified by a hexadecimal number, a captured group reference. String with hexadecimal number or captured group reference should be double quoted. For example `"Y\u00eb"` or `"\\1\u00b7\\2"`. Captured group are referred by double backslash and group's number.
-
-Because rules are applied in order, multiple rules applicable to the same segment of a string can be addressed by rule ordering, and rules can be used as priority over characters. For example:
-
-[source,yaml]
-----
-map:
-  rules:
-    - pattern: \u03B3\u03B3    # γ (before Γ, Ξ, Χ)
-      result: ng
-    - pattern: (?<![Γγ])\u03B3(?=[ΕεέΗηήΙιίΥυύ])    # γ (before front vowels)
-      result: y
-----
-
-(γι maps to `yi`; but γγ maps to `ng`. In the case of γγι, the first rule takes priority, and the transliteration is `ngi`: it makes the second rule impossible.)
-
-[source,yaml]
-----
-map:
-  rules:
-    - pattern: (?<=\b)\u03BC[πΠ]  # μπ (initially)
-      result: b
-    - pattern: \u03BC[πΠ]         # μπ (medially)
-      result: mb
-----
-
-(The first rule applies at the start of a word; the second rule does not specify a context, as it applies in all other cases not covered by the first rule.)
-
-[source,yaml]
-----
-map:
-  rules:
-    - pattern: ";"
-      result: "?"
-
-  characters
-    "\u00B7": ";
-----
-
-(This guarantees that any `;` are converted to `?` before any new `;` are introduced; because all three are Latin script, they could be mixed up in ordering.)
-
-Normally rules "`bleed`" each other: once a rule applies to a segment, that segment cannot trigger other rules, because it is already converted to Roman. Exceptionally, it will be necessary to have a rule add or remove characters in the original script, rather than transliterate them, so that the same context can be invoked by two rules in succession:
-
-[source,yaml]
-----
-map:
-  rules:
-    - pattern: (?<=[АаЕеЁёИиОоУуЫыЭэЮюЯя])\u042b # Ы after any vowel character
-      result: "\u00b7Ы"
-    - pattern: \u042b(?=[АаУуЫыЭэ])              # Ы before а, у, ы, or э
-      result: "Ы\u00b7"
-----
-
-(If the result were `\u00B7Y`, the second rule could not be applied afterwards; but we want ОЫУ to transliterate as `O·Y·U`. In order to make that happen, we preserve the Ы during the rules phase, resulting in О·Ы·У; we only convert the letters to Roman script in the `characters` phase.)
-
-=== Testing transliteration systems
-
-To test all transliteration systems in the `maps/` directory, run:
-
-[source,sh]
-----
-bundle exec rspec
-----
-
-The command takes `source` texts from the `test` section, transforms
-them using `rules` and `charmaps` from the `map` key, and compares the
-results with `expected:` text from the `source:` section.
-
-To test a specific transliteration system, set the environment variable
-`TRANSLIT_SYSTEM` to the system code of the desired system
-(i.e. the "`basename`" of the system's YAML file):
-
-[source,sh]
-----
-TRANSLIT_SYSTEM=bgnpcgn-rus-Cyrl-Latn-1947 bundle exec rspec
-----
-
-
-== ISCS system codes
-
-In accordance with
-http://calconnect.gitlab.io/tc-localization/csd-transcription-systems[ISO/CC 24229],
-the system code identifying a script conversion system has the following components:
-
-e.g. `bgnpcgn-rus-Cyrl-Latn-1947`:
-
-`bgnpcgn`:: the authority identifier
-`rus`:: an ISO 639-{1,2,3,5} language code that this system applies to (For 639-2, use (T) code)
-`Cyrl`:: an ISO 15924 script code, identifying the source script
-`Latn`:: an ISO 15924 script code, identifying the target script
-`1947`:: an identifier unit within the authority to identify this system
-
-
-== Covered languages
-
-Currently the schemes cover Cyrillic, Armenian, Greek, Arabic and Hebrew.
-
-
-== Samples to play with
-
-* `rus-Cyrl-1.txt`: Copied from the XLS output from http://www.primorsk.vybory.izbirkom.ru/region/primorsk?action=show&global=true&root=254017025&tvd=4254017212287&vrn=100100067795849&prver=0&pronetvd=0&region=25&sub_region=25&type=242&vibid=4254017212287
-
-* `rus-Cyrl-2.txt`: Copied from the XLS output from http://www.yaroslavl.vybory.izbirkom.ru/region/yaroslavl?action=show&root=764013001&tvd=4764013188704&vrn=4764013188693&prver=0&pronetvd=0&region=76&sub_region=76&type=426&vibid=4764013188704
-
-
-== References
-
-Reference documents are located at the
-https://github.com/interscript/interscript-references[interscript-references repository].
-Some specifications that have distribution limitations may not be reproduced there.
-
-
-== Links to system definitions
-
-* https://www.iso.org/committee/48750.html[ISO/TC 46 (see standards published by WG 3)]
-* http://geonames.nga.mil/gns/html/romanization.html[BGN/PCGN and BGN Romanization systems (BGN)]
-* https://www.gov.uk/government/publications/romanization-systems[BGN/PCGN Romanization systems (PCGN)]
-* https://www.loc.gov/catdir/cpso/roman.html[ALA-LC Romanization systems in current use]
-* http://catdir.loc.gov/catdir/cpso/roman.html[ALA-LC Romanization systems from 1997]
-* http://www.eki.ee/wgrs/[UN Romanization systems]
-* http://www.eki.ee/knab/kblatyl2.htm[EKI KNAB systems]
-
-== Copyright and license
-
-This is a Ribose project. Copyright Ribose.
-

data/aliases.json

@@ -1 +0,0 @@
-{"pan_Deva2Latn_ALA_1997":"alalc-pan-Guru-Latn-1997","kor_Hang2Latn_ALA_1997":"alalc-kor-Hang-Latn-1997","asm_Deva2Latn_ALA_1997":"alalc-asm-Deva-Latn-1997","aze_Cyrl2Latn_ALA_1997":"alalc-aze-Cyrl-Latn-1997","ukr_Cyrl2Latn_GUP_1996":"ua-ukr-Cyrl-Latn-1996","tha_Thai2Latn_RIT_1968":"royin-tha-Thai-Latn-1968","bul_Cyrl2Latn_BGN_1952":"bgnpcgn-bul-Cyrl-Latn-1952","tam_Taml2Latn_ALA_1997":"alalc-tam-Taml-Latn-1997","kor_Hang2Latn_GKN_2002":"kp-kor-Hang-Latn-2002","ell_Grek2Latn_ELOT743_1996":"bgnpcgn-ell-Grek-Latn-1996","zho_Hani2Latn_AcadSin_2002":"acadsin-zho-Hani-Latn-2002","ara_Arab2Latn_SES_1930":"ses-ara-Arab-Latn-1930","tgk_Cyrl2Latn_BGN_1994":"bgnpcgn-tgk-Cyrl-Latn-1994","fas_Arab2Latn_BGN_1958":"bgnpcgn-fas-Arab-Latn-1956","sin_Sinh2Latn_ALA_1997":"alalc-sin-Sinh-Latn-1997","uas_Arab2Latn_BGN_2007":"bgnpcgn-urd-Arab-Latn-2007","ukr_Cyrl2Latn_ALA_1997":"alalc-ukr-Cyrl-Latn-1997","bak_Cyrl2Latn_BGN_2007":"bgnpcgn-bak-Cyrl-Latn-2007","tam_Taml2Latn_ALA_2011":"alalc-tam-Taml-Latn-2011","ara_Arab2Latn_BGN_1956":"bgnpcgn-ara-Arab-Latn-1956","ell_Grek2Latn_ALA_1997":"alalc-ell-Grek-Latn-1997","rus_Cyrl2Latn_GOST_1983":"gost-rus-Cyrl-Latn-16876-71-1983","mar_Deva2Latn_ALA_1997":"alalc-mar-Deva-Latn-1997","bel_Cyrl2Latn_ALA_1997":"alalc-bel-Cyrl-Latn-1997","kat_Geor2Latn_ALA_1997":"alalc-kat-Geor-Latn-1997","bul_Cyrl2Latn_ALA_1997":"alalc-bul-Cyrl-Latn-1997","ara_Arab2Latn_ALA_1997":"alalc-ara-Arab-Latn-1997","mon_Cyrl2Latn_ALA_1997":"alalc-mon-Cyrl-Latn-1997","div_Thaa2Latn_GMV_1988":"bgnpcgn-div-Thaa-Latn-1988","hin_Deva2Latn_ALA_1997":"alalc-hin-Deva-Latn-1997","bel_Cyrl2Latn_GBO_1998":"by-bel-Cyrl-Latn-1998","ukr_Cyrl2Latn_BGN_1965":"bgnpcgn-ukr-Cyrl-Latn-1965","rus_Cyrl2Latn_ALA_1997":"alalc-rus-Cyrl-Latn-1997","tir_Thai2Latn_RIT_2000":"royin-tha-Thai-Latn-1999","guj_Gujr2Latn_ALA_1997":"alalc-guj-Gujr-Latn-1997","tel_Telu2Latn_ALA_1997":"alalc-tel-Telu-Latn-1997","mkd_Cyrl2Latn_BGN_1981":"bgnpcgn-mkd-Cyrl-Latn-1981","ori_Orya2Latn_ALA_2011":"alalc-ori-Orya-Latn-2011","aze_Arab2Latn_ALA_1997":"alalc-aze-Arab-Latn-1997","ori_Orya2Latn_ALA_1997":"alalc-ori-Orya-Latn-1997","div_Thaa2Latn_ALA_1997":"alalc-div-Thaa-Latn-1997","rue_Cyrl2Latn_BGN_2016":"bgnpcgn-rue-Cyrl-Latn-2016","guj_Gujr2Latn_ALA_2011":"alalc-guj-Gujr-Latn-2011","kat_Geor2Latn_BGN_1981":"bgnpcgn-kat-Geor-Latn-1981","kor_Hang2Latn_MOCT_2000":"moct-kor-Hang-Latn-2000","sin_Sinh2Latn_ALA_2011":"alalc-sin-Sinh-Latn-2011","amh_Ethi2Latn_BGN_1967":"bgnpcgn-amh-Ethi-Latn-1967","srp_Cyrl2Latn_BGN_2005":"bgnpcgn-srp-Cyrl-Latn-2005","srp_Cyrl2Latn_ALA_1997":"alalc-srp-Cyrl-Latn-1997","mal_Mlym2Latn_ALA_2012":"alalc-mal-Mlym-Latn-2012","kat_Geor2Latn_GGG_2002":"ggg-kat-Geor-Latn-2002","mon_Cyrl2Latn_BGN_1964":"bgnpcgn-mon-Cyrl-Latn-1964","mal_Mlym2Latn_ALA_1997":"alalc-mal-Mlym-Latn-1997","ben_Beng2Latn_ALA_1997":"alalc-ben-Beng-Latn-1997","kor_Hang2Latn_MR_1939":"bgn-kor-Hang-Latn-1943","zho_Hani2Latn_GCH_1979":"sac-zho-Hans-Latn-1979","bul_Cyrl2Latn_BGN_2013":"bgnpcgn-bul-Cyrl-Latn-2013","ell_Grek2Latn_BGN_1962":"bgnpcgn-ell-Grek-Latn-1962","amh_Ethi2Latn_ALA_1997":"alalc-amh-Ethi-Latn-1997","pan_Deva2Latn_ALA_2011":"alalc-pan-Guru-Latn-2011","zho_Hani2Latn_WDG_1979":"var-zho-Hani-Latn-wd-1979","rus_Cyrl2Latn_BGN_1947":"bgnpcgn-rus-Cyrl-Latn-1947","bel_Cyrl2Latn_BGN_1979":"bgnpcgn-bel-Cyrl-Latn-1979","tat_Cyrl2Latn_BGN_2005":"bgnpcgn-tat-Cyrl-Latn-2007"}

data/lib/g2pwrapper.py

@@ -1,34 +0,0 @@
-import g2p, SequiturTool
-import numpy
-
-def transliterate(model, word):
-
-  class Struct:
-      def __init__(self, **entries):
-          self.__dict__.update(entries)
-
-  model_path = {
-    'pythainlp_lexicon': './lib/model-7', 
-    'wiktionary_phonemic': './lib/tha-pt-b-7'
-  }
-
-  connector_dict = {
-    'pythainlp_lexicon': '', 
-    'wiktionary_phonemic': '-'
-  }
-
-
-  modelFile = model_path[model]
-  connector = connector_dict[model]
-
-  options = Struct(**{'profile': None, 'resource_usage': None, 'psyco': None, 'tempdir': None, 'trainSample': None, 'develSample': None, 'testSample': None, 'checkpoint': None, 'resume_from_checkpoint': None, 'shouldTranspose': None, 'modelFile': modelFile , 'newModelFile': None, 'shouldTestContinuously': None, 'shouldSelfTest': None, 'lengthConstraints': None, 'shouldSuppressNewMultigrams': None, 'viterbi': None, 'shouldRampUp': None, 'shouldWipeModel': None, 'shouldInitializeWithCounts': None, 'minIterations': 20, 'maxIterations': 100, 'eager_discount_adjustment': None, 'fixed_discount': None, 'encoding': 'UTF-8', 'phoneme_to_phoneme': None, 'test_segmental': None, 'testResult': None, 'applySample': None, 'applyWord': word, 'variants_mass': None, 'variants_number': None, 'fakeTranslator': None, 'stack_limit': None})
-
-  loadSample = g2p.loadG2PSample
-
-  model = SequiturTool.procureModel(options, loadSample)
-  if not model:
-      return 1
-  translator = g2p.Translator(model)
-  del model
-
-  return connector.join(translator(tuple(word)))

data/lib/interscript/fs.rb

@@ -1,96 +0,0 @@
-require 'pathname'
-
-module Interscript
-  module Fs
-    def sub_replace(string, pos, size, repl)
-      string[pos..pos + size - 1] = repl
-      string
-    end
-
-    def root_path
-      @root_path ||= Pathname.new(File.join(File.dirname(__dir__), ".."))
-    end
-
-    def transliterate_file(system_code, input_file, output_file, maps={})
-      input = File.read(input_file)
-      output = transliterate(system_code, input, maps)
-
-      File.open(output_file, 'w') do |f|
-        f.puts(output)
-      end
-
-      puts "Output written to: #{output_file}"
-      output_file
-    end
-
-    def import_python_modules
-      begin
-        pyimport :g2pwrapper
-      rescue
-        pyimport :sys
-        sys.path.append(root_path.to_s + "/lib/")
-        pyimport :g2pwrapper
-      end
-    end
-
-    def external_process(process_name, string)
-      import_python_modules
-
-      case process_name
-      when 'sequitur.pythainlp_lexicon'
-        return g2pwrapper.transliterate('pythainlp_lexicon', string)
-      when 'sequitur.wiktionary_phonemic'
-        return g2pwrapper.transliterate('wiktionary_phonemic', string)
-      else
-        raise ExternalProcessNotRecognizedError.new
-      end
-
-    rescue
-      raise ExternalProcessUnavailableError.new
-    end
-
-    def external_processing(mapping, string)
-      # Segmentation
-      string = external_process(mapping.segmentation, string) if mapping.segmentation
-
-      # Transliteration/Transcription
-      string = external_process(mapping.transcription, string) if mapping.transcription
-
-      string
-    end
-
-    def aliases (refresh: false)
-      file = root_path.join("./aliases.json").to_s
-      if !refresh && File.exist?(file)
-        JSON.load(File.read(file))
-      elsif !refresh && @aliases
-        @aliases
-      else
-        @aliases = {}
-        Dir[root_path.join('./maps/*.yaml').to_s].each do |yaml_file|
-          org_name = File.basename(yaml_file, ".yaml")
-          map = YAML.load_file(yaml_file)
-          (map["alias"] || {}).each do |k,v|
-            @aliases[v["code"]] = org_name
-          end
-        end
-
-        # Try to save it to a file, but not force it.
-        File.write("aliases.json", JSON.dump(@aliases)) rescue nil
-
-        @aliases
-      end
-    end
-
-    private
-
-    def map_exist?(map)
-      File.exist?(root_path.join("./maps/" + map + ".yaml").to_s)
-    end
-
-    def mkregexp(regexpstring)
-      /#{regexpstring}/u
-    end
-
-  end
-end

data/lib/interscript/mapping.rb

@@ -1,144 +0,0 @@
-require 'rambling-trie'
-require 'yaml' unless RUBY_ENGINE == 'opal'
-require 'json'
-
-module Interscript
-
-  class Mapping
-    attr_reader(
-      :id,
-      :url,
-      :name,
-      :notes,
-      :rules,
-      :tests,
-      :language,
-      :postrules,
-      :characters,
-      :description,
-      :authority_id,
-      :creation_date,
-      :source_script,
-      :destination_script,
-      :chain,
-      :character_separator,
-      :word_separator,
-      :title_case,
-      :downcase,
-      :dictionary,
-      :characters_hash,
-      :dictionary_hash,
-      :segmentation,
-      :transcription,
-      :dictionary_trie
-    )
-
-    def initialize(system_code, options = {})
-      @system_code = system_code
-      @depth = options.fetch(:depth, 0).to_i
-
-      unless RUBY_ENGINE == 'opal'
-        @system_path = options.fetch(:system_code, default_path)
-      end
-
-      load_and_serialize_system_mappings
-    end
-
-    def self.for(system_code, options = {})
-      new(system_code, options)
-    end
-
-    def load_and_serialize_system_mappings
-      return if depth >= 5
-
-      mappings = load_system_mappings
-      serialize_system_mappings(mappings)
-    end
-
-    private
-
-    attr_reader :depth, :system_code, :system_path
-
-    def system_code_file
-      [system_code, "yaml"].join(".")
-    end
-
-    def default_path
-      @default_path ||= Interscript.root_path.join("maps")
-    end
-
-    def load_system_mappings
-      if RUBY_ENGINE == 'opal'
-        load_opal_mappings
-      else
-        load_fs_mappings
-      end
-    end
-
-    def load_opal_mappings
-      JSON.parse(`Opal.global.InterscriptMaps[#{system_code}]`)
-    end
-
-    def load_fs_mappings
-      YAML.load_file(system_path.join(system_code_file))
-    rescue Errno::ENOENT
-      raise Interscript::InvalidSystemError.new("No system mappings found")
-    end
-
-    def serialize_system_mappings(mappings)
-      @id = mappings.fetch("id", nil)
-      @url = mappings.fetch("url", nil)
-      @name = mappings.fetch("name", nil)
-      @notes = mappings.fetch("notes", nil)
-      @tests = mappings.fetch("tests", [])
-      @language = mappings.fetch("language", nil)
-      @description = mappings.fetch("description", nil)
-      @authority_id = mappings.fetch("authority_id", nil)
-      @creation_date = mappings.fetch("creation_date", nil)
-      @source_script = mappings.fetch("source_script", nil)
-      @destination_script = mappings.fetch("destination_script", nil)
-      @chain = mappings.fetch("chain", [])
-      @character_separator = mappings["map"]["character_separator"] || nil
-      @word_separator = mappings["map"]["word_separator"] || nil
-      @title_case = mappings["map"]["title_case"] || false
-      @downcase = mappings["map"]["downcase"] || false
-      @rules = mappings["map"]["rules"] || []
-      @postrules = mappings["map"]["postrules"] || []
-      @characters = mappings["map"]["characters"] || {}
-      @dictionary = mappings["map"]["dictionary"] || {}
-      @segmentation = mappings["map"]["segementation"] || nil
-      @transcription = mappings["map"]["transcription"] || nil
-
-      include_inherited_mappings(mappings)
-      build_hashes
-      build_trie
-    end
-
-    def include_inherited_mappings(mappings)
-      inherit_systems = [].push(mappings["map"]["inherit"]).flatten
-
-      inherit_systems.each do |inherit_system|
-        next unless inherit_system
-
-        inherited_mapping = Mapping.for(inherit_system, depth: depth + 1)
-
-        @rules = [rules, inherited_mapping.rules].flatten
-        @postrules = [inherited_mapping.postrules, postrules].flatten
-        @characters = (inherited_mapping.characters|| {}).merge(characters)
-        @dictionary = (inherited_mapping.dictionary|| {}).merge(dictionary)
-      end
-
-      @characters.compact! # the feature to ignore characters from inherited
-    end
-
-    def build_hashes
-      @characters_hash = characters&.sort_by { |k, _v| k.size }&.reverse&.to_h
-      @dictionary_hash = dictionary&.sort_by { |k, _v| k.size }&.reverse&.to_h
-    end
-
-    def build_trie
-      @dictionary_trie = Rambling::Trie.create
-      dictionary_trie.concat dictionary.keys
-    end
-  end
-end