folio-migration-tools 1.2.1__py3-none-any.whl → 1.9.10__py3-none-any.whl

folio_migration_tools-1.9.10.dist-info/METADATA

@@ -0,0 +1,169 @@
+Metadata-Version: 2.4
+Name: folio_migration_tools
+Version: 1.9.10
+Summary: A tool allowing you to migrate data from legacy ILS:s (Library systems) into FOLIO LSP
+License-Expression: MIT
+License-File: LICENSE
+Keywords: FOLIO,ILS,LSP,Library Systems,MARC21,Library data
+Author: Theodor Tolstoy
+Author-email: github.teddes@tolstoy.se
+Requires-Python: >=3.10,<4.0
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Provides-Extra: docs
+Requires-Dist: argparse-prompt (>=0.0.5,<0.0.6)
+Requires-Dist: art (>=6.5,<7.0)
+Requires-Dist: deepdiff (>=6.2.3,<7.0.0)
+Requires-Dist: defusedxml (>=0.7.1,<0.8.0)
+Requires-Dist: folio-data-import (>=0.4.1)
+Requires-Dist: folio-uuid (>=1.0.0,<2.0.0)
+Requires-Dist: folioclient (>=0.70.1,<0.71.0)
+Requires-Dist: pyaml (>=21.10.1,<22.0.0)
+Requires-Dist: pydantic (>=1.10.2,<2.0.0)
+Requires-Dist: pyhumps (>=3.7.3,<4.0.0)
+Requires-Dist: pymarc (>=5.2.3,<6.0.0)
+Requires-Dist: python-dateutil (>=2.8.2,<3.0.0)
+Requires-Dist: python-i18n (>=0.3.9,<0.4.0)
+Project-URL: Documentation, https://folio-migration-tools.readthedocs.io/en/latest/
+Project-URL: Homepage, https://github.com/folio-fse/folio_migration_tools
+Project-URL: Repository, https://github.com/FOLIO-FSE/folio_migration_tools
+Description-Content-Type: text/markdown
+
+# FOLIO Migration Tools
+![example workflow](https://github.com/FOLIO-FSE/MARC21-To-FOLIO/actions/workflows/python-app.yml/badge.svg)[![codecov](https://codecov.io/gh/FOLIO-FSE/folio_migration_tools/branch/main/graph/badge.svg?token=ZQL5ILWWGT)](https://codecov.io/gh/FOLIO-FSE/folio_migration_tools)   [![readthedocs](https://readthedocs.org/projects/docs/badge/?version=latest)](https://folio-migration-tools.readthedocs.io/)
+
+A toolkit that enables you to migrate data over from a legacy ILS system into [FOLIO LSP](https://www.folio.org/)
+
+# What is it good for?
+FOLIO Migration tools enables you to migrate libraries with the most common ILS:s over to FOLIO without data losses or any major data transformation tasks. 
+The tools transforms and loads the data providing you and the library with good actionable logs and data cleaning task lists together with the migrated data.
+
+## What data does it cover?
+FOLIO Migration Tools currently covers the following data sets:
+* Catalog (Inventory and SRS in FOLIO terminology)
+* Circulation transactions (Open loans and requests)
+* Users/Patrons (In FOLIO, these share the same app/database)
+* Courses and Reserves (Course reserves)
+* Organizations (Vendor records)
+* Orders (limited support)
+
+### What additional functionality is on the roadmap?
+This is the loose roadmap, in order of most likely implementations first
+* ERM-related objects
+* Financial records
+
+### Can I use the tools for ongoing imports and integrations?
+The tools are primarliy maintained for performing initial data migrations. We recommend that you use native FOLIO functionality for ongoing loads where possible. 
+In theory, these tools can be used for ongoing patron loads from systems like Banner, Workday, or PeopleSoft. But we recommend you to weigh your options carefully before going down this path. 
+
+# Contributing
+Want to contribute? Read the [CONTRIBUTING.MD](https://github.com/FOLIO-FSE/folio_migration_tools/blob/main/CONTRIBUTING.md)
+
+# Found an issue?
+Report it on the [Github Issue tracker](https://github.com/FOLIO-FSE/folio_migration_tools/issues)
+
+The scripts requires a FOLIO tenant with reference data properly set up. The script will throw messages telling what reference data is missing.
+# Installing
+Make sure you are running Python 3.9 or above. 
+## 1. Using pip and venv
+### 2.1. Create and activate a [virtual environment](https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/#creating-a-virtual-environment)   
+```   
+python -m venv ./.venv     # Creates a virtual env in the current folder
+source .venv/bin/activate  # Activates the venv    
+```
+### 2. Install using pip: 
+```
+python -m pip install folio_migration_tools
+```
+### 3. Test the installation by showing the help pages 
+```   
+python -m folio_migration_tools -h
+```    
+
+## 2. Using pipenv
+### 1. Run
+```   
+pipenv install folio-migration-tools
+```   
+### 2. Test the installation by showing the help pages
+```  
+pipenv run python3 -m folio_migration_tools -h
+```
+
+# FOLIO migration process
+This repo plays the main part in a process using a collection of tools. The process itself is documented in more detail, including example configuration files, at [this template repository](https://github.com/FOLIO-FSE/migration_repo_template)
+In order to perform migrations according to this process, you need the following:
+* An Installation of [FOLIO Migration Tools](https://pypi.org/project/folio-migration-tools/). Installation instructions above.
+* A clone, or a separate repo created from [migration_repo_template](https://github.com/FOLIO-FSE/migration_repo_template)
+* Access to the [Data mapping file creator](https://data-mapping-file-creator.folio.ebsco.com/data_mapping_creation) web tool
+* A FOLIO tenant running the latest or the second latest version of FOLIO
+
+# Internationalization
+
+This repo uses [Python-i18n](https://github.com/danhper/python-i18n) to translate reports between languages, and to handle large strings for templates.
+
+**Any English string which will end up in a report** should be wrapped in the function `i18n.t` from `i18n`:
+
+## Keys/Usage
+
+```js
+i18n.t("Reports")+":"
+```
+
+Templating is achieved with `%{[key]}` blocks, and keyword arguments in the internationaliation:
+
+```js
+i18n.t("Code '%{code}' not found in FOLIO",code=folio_code)
+```
+
+Long strings can use a placeholder key:
+
+```js
+i18n.t("blurbs.Introduction.description")
+```
+
+With the full string in ```translations/en.json```:
+
+```json
+"blurbs.Introduction.description": "<br/>Data errors preventing records from being migrated
+```
+
+## Translations Files
+
+Translation files live in the `translations` directory, with `en.json` as the default.
+
+Extract template files with the `extract_translations` script:
+
+```bash
+python scripts/extract_translations.py
+```
+
+## Internationalizations
+
+Other langauges translations live in `translations/[locale].json`.
+For example, Spanish would be `es.json`. 
+
+The keys must match the English keys, but the Values should be translated.
+
+You can update a language file's keys with:
+
+```bash
+python scripts/update_language.py --target-lang [locale]
+```
+
+Translate all new strings, which begin with `TRANSLATE`, then commit.
+
+## Tips
+
+* Internationalize entire phrases or paragraphs, not just the constitutent words. Syntax and grammar vary significantly between languages.
+* Name template variables as generically as possible in the circumstance, and check translations for reusable translations.
+* In a block with sentences separately followed by values, such as a table, you only need to translate the sentences. 
+
+# Running the scripts
+For information on syntax, what files are needed and produced by the toolkit, refer to the documentation and example files in the [template repository](https://github.com/FOLIO-FSE/migration_repo_template). We are building out the docs section in this repository as well:[Documentation](https://folio-migration-tools.readthedocs.io/en/latest/)
+¨
+

folio_migration_tools-1.9.10.dist-info/RECORD

@@ -0,0 +1,67 @@
+folio_migration_tools/__init__.py,sha256=DXvzUKFSpSZjflFWaNm0L8yhFk0u7RVIvQMskwMmKFc,238
+folio_migration_tools/__main__.py,sha256=kfo4iKf3GJD7deh4RvIizKnC4zvIgCpNo-Bs7HBM34s,8453
+folio_migration_tools/circulation_helper.py,sha256=iib2p0WGzOcAnwaJppDYiONc-1Jdna66dPAtBmAIuCE,14538
+folio_migration_tools/colors.py,sha256=GP0wdI_GZ2WD5SjrbPN-S3u8vvN_u6rGQIBBcWv_0ZM,227
+folio_migration_tools/config_file_load.py,sha256=zHHa6NDkN6EJiQE4DgjrFQPVKsd70POsfbGkB8308jg,2822
+folio_migration_tools/custom_dict.py,sha256=-FUnhKp90Dg8EHlY6twx-PYQxBUWEO7FgxL2b7pf-xk,678
+folio_migration_tools/custom_exceptions.py,sha256=1zgOKy3NBUVGG6i9YxK6w2Hntlea8MHmm7mdnjBtzvQ,2687
+folio_migration_tools/extradata_writer.py,sha256=fuchNcMc6BYb9IyfAcvXg7X4J2TfX6YiROfT2hr0JMw,1678
+folio_migration_tools/folder_structure.py,sha256=bZlmKGtxdytWcqjnM2lE4Vpx4nHyYRk7CNL1tZhLtXY,6917
+folio_migration_tools/helper.py,sha256=iWQhdcuzbGzVpEAiHQczO4hdhIH_iciLEp1SCGDynMI,2983
+folio_migration_tools/holdings_helper.py,sha256=yJpz6aJrKRBiJ1MtT5bs2vXAc88uJuGh2_KDuCySOKc,7559
+folio_migration_tools/i18n_config.py,sha256=3AH_2b9zTsxE4XTe4isM_zYtPJSlK0ix6eBmV7kAYUM,228
+folio_migration_tools/library_configuration.py,sha256=LzICsZQdOkXwIqdDfh59x0-Cx77Lb18qVnWroNqekS8,7046
+folio_migration_tools/mapper_base.py,sha256=IYER8Dq-4qLq3qiAvUpnzc33usUbfZtNKzyZJD6DNds,23567
+folio_migration_tools/mapping_file_transformation/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+folio_migration_tools/mapping_file_transformation/courses_mapper.py,sha256=yX8yCCw6B54r7JUn5IPhMYKBgImiagUzfwBxNVRf5MQ,8091
+folio_migration_tools/mapping_file_transformation/holdings_mapper.py,sha256=qT8LMWZytSaOyUC9OzfJeekVIkLkNim4OzfoGlAh75g,8290
+folio_migration_tools/mapping_file_transformation/item_mapper.py,sha256=ZmPc64X_l3hq8qc0WDhZ020TdjvucZoQciIVIpPzwiA,10472
+folio_migration_tools/mapping_file_transformation/manual_fee_fines_mapper.py,sha256=2-W2Z8hwAhWT77zfDWuwWqm20j4w1mfzeAXWiyssQ8I,13434
+folio_migration_tools/mapping_file_transformation/mapping_file_mapper_base.py,sha256=LqxpULlickaQhIv92iBA7SwSXy1PN7l-9swLHQdnx6g,38408
+folio_migration_tools/mapping_file_transformation/notes_mapper.py,sha256=vCmZmjrjyYtXeFCyVqvWfnP8y1jGGu15RXzXIHh12xY,3530
+folio_migration_tools/mapping_file_transformation/order_mapper.py,sha256=hYnVkm28hE8GtOXQJDHBO0yt2Kp34bR71Qwye83WgFI,18226
+folio_migration_tools/mapping_file_transformation/organization_mapper.py,sha256=u1Lb6tApn-nVLqbbJV38BuipKL3OK8Y2uQ4ogoyGQaI,14639
+folio_migration_tools/mapping_file_transformation/ref_data_mapping.py,sha256=rROcBiL5TE7bWsJ95A6shurPZ1e4In6PTwR5BN9amzU,8991
+folio_migration_tools/mapping_file_transformation/user_mapper.py,sha256=LJTj2F2dRKqyI37Ww0gY1AHLLT3dqyuKkY_RS_3-rg0,8543
+folio_migration_tools/marc_rules_transformation/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+folio_migration_tools/marc_rules_transformation/conditions.py,sha256=F78a70HXcDLnOXDs_vSTdgf4opMWHzXzOjvpWlOh4PM,47719
+folio_migration_tools/marc_rules_transformation/holdings_statementsparser.py,sha256=gM0ETZVVih35cSSpOBXA8wrBFhq2oeYaGsD89tnSNJs,13433
+folio_migration_tools/marc_rules_transformation/hrid_handler.py,sha256=SgnSYeNR0z_qarkizBMWZZWr8tOPZJ4fvlZjlM3nJOU,9999
+folio_migration_tools/marc_rules_transformation/loc_language_codes.xml,sha256=ztn2_yKws6qySL4oSsZh7sOjxq5bCC1PhAnXJdtgmJ0,382912
+folio_migration_tools/marc_rules_transformation/marc_file_processor.py,sha256=QhVbJSlsWkGQgUo7ZVmQvlwpEN20Tyon_kzrZOWECoE,12549
+folio_migration_tools/marc_rules_transformation/marc_reader_wrapper.py,sha256=9ATjYMRAjy0QcXtmNZaHVhHLJ5hE1WUgOcF6KMJjbgo,5309
+folio_migration_tools/marc_rules_transformation/rules_mapper_authorities.py,sha256=PGt2w8h2pj8_8sGjQe3L-odFDlquURtKnoNFRWQB3GI,9621
+folio_migration_tools/marc_rules_transformation/rules_mapper_base.py,sha256=loNZ9gEYaAwjkP2_wLlXGedjWvSdHoGF_oJN9g6gI3s,45928
+folio_migration_tools/marc_rules_transformation/rules_mapper_bibs.py,sha256=RD46EzS0NQArn5LCGbrxDm9vbbW9PO_6iNUQwJBAbSg,30364
+folio_migration_tools/marc_rules_transformation/rules_mapper_holdings.py,sha256=ZgyDxmNE7LwW8Cd55wRIEE-u6iyMKCRRXdq2ZRjm2nc,28779
+folio_migration_tools/migration_report.py,sha256=BkRspM1hwTBnWeqsHamf7yVEofzLj560Q-9G--O00hw,4258
+folio_migration_tools/migration_tasks/__init__.py,sha256=ZkbY_yGyB84Ke8OMlYUzyyBj4cxxNrhMTwQlu_GbdDs,211
+folio_migration_tools/migration_tasks/authority_transformer.py,sha256=AoXg9s-GLO3yEEDCrQV7hc4YVXxwxsdxDdpj1zhHydE,4251
+folio_migration_tools/migration_tasks/batch_poster.py,sha256=Oc04VXIs-D8HzvKDtFR3evGVuPZmw35iWyUyOnpP-RE,50410
+folio_migration_tools/migration_tasks/bibs_transformer.py,sha256=46d44pcDAodFXDYbrTCMRASISbDciXmA0CXYfhP2IaE,6298
+folio_migration_tools/migration_tasks/courses_migrator.py,sha256=CzXnsu-KGP7B4zcINJzLYUqz47D16NuFfzu_DPqRlTQ,7061
+folio_migration_tools/migration_tasks/holdings_csv_transformer.py,sha256=-lMsQCkxVj2kFa0awf0_wsD5do1hbRhUqIu4p0UiRNk,21955
+folio_migration_tools/migration_tasks/holdings_marc_transformer.py,sha256=c_ruhOgidyJdSnnRwWUs3wwFMiLqbVMPOhhCaYuH_TI,14343
+folio_migration_tools/migration_tasks/items_transformer.py,sha256=oTbFX2saF7-ZCb1mO3baLvODnBSEbbN5F_GtSth3iG4,19755
+folio_migration_tools/migration_tasks/loans_migrator.py,sha256=PF8DNpGKsppSDr7wX96Ao13UTFu6dl9cz2URLqSsOzE,40052
+folio_migration_tools/migration_tasks/manual_fee_fines_transformer.py,sha256=CnmlTge7nChUJ10EiUkriQtJlVxWqglgfhjgneh2_yM,7247
+folio_migration_tools/migration_tasks/migration_task_base.py,sha256=weQGuqmJz_OFvUUmV3tf19va7Hzc01OAOELbsn9ORR0,22664
+folio_migration_tools/migration_tasks/orders_transformer.py,sha256=3Fk3WgSYtCboIsRgHpt0Zd5-es8vT7FWoqDHkupUXLk,14318
+folio_migration_tools/migration_tasks/organization_transformer.py,sha256=Kuxkh1sKyUVBqm5qAK1Jrq-4xcyNz2JPZvvFRqfwI8s,16922
+folio_migration_tools/migration_tasks/requests_migrator.py,sha256=QP9OBezC3FfcKpI78oMmydxcPaUIYAgHyKevyLwC-WQ,14841
+folio_migration_tools/migration_tasks/reserves_migrator.py,sha256=4sSPer6_6yMwiiY1VYJmYZske_Ah1XG4KAM3NDadPhg,9952
+folio_migration_tools/migration_tasks/user_transformer.py,sha256=aylrMC9n47fdStgsNfW4ZbJh2E4FDSPypsaNv52ynKc,12330
+folio_migration_tools/task_configuration.py,sha256=73OWc8TX--fwPRptv3eQVEVv0-XmNaZcb3m__1HENSA,1161
+folio_migration_tools/test_infrastructure/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+folio_migration_tools/test_infrastructure/mocked_classes.py,sha256=BurU3NGU_Q8as_BGmW98q9O6bujZDkOfFmvKKdVw9t8,15056
+folio_migration_tools/transaction_migration/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+folio_migration_tools/transaction_migration/legacy_loan.py,sha256=sLw2fCBao1VWZykOx-BWpMndLHjJNj-HJZFYqbpVV5A,7561
+folio_migration_tools/transaction_migration/legacy_request.py,sha256=1ulyFzPQw_InOjyPzkWpGnNptgXdQ18nmri0J8Nlpkc,6124
+folio_migration_tools/transaction_migration/legacy_reserve.py,sha256=qzw0okg4axAE_ezXopP9gFsQ_e60o0zh7zqRzFBSWHY,1806
+folio_migration_tools/transaction_migration/transaction_result.py,sha256=cTdCN0BnlI9_ZJB2Z3Fdkl9gpymIi-9mGZsRFlQcmDk,656
+folio_migration_tools/translations/en.json,sha256=4Ac66PR5Y78ll4_grhBm2IdTMsoZUv0q3IJvX8SQiJI,41778
+folio_migration_tools-1.9.10.dist-info/METADATA,sha256=otbWpB6CpG0dn3fTGpgP9U1PMj7aMc5LpCQ-REYC3Oo,7472
+folio_migration_tools-1.9.10.dist-info/WHEEL,sha256=zp0Cn7JsFoX2ATtOhtaFYIiE2rmFAD4OcMhtUki8W3U,88
+folio_migration_tools-1.9.10.dist-info/entry_points.txt,sha256=Hbe-HjqMcU8FwVshVIkeWyZd9XwgT1CCMNf06EpHQu8,77
+folio_migration_tools-1.9.10.dist-info/licenses/LICENSE,sha256=PhIEkitVi3ejgq56tt6sWoJIG_zmv82cjjd_aYPPGdI,1072
+folio_migration_tools-1.9.10.dist-info/RECORD,,

{folio_migration_tools-1.2.1.dist-info → folio_migration_tools-1.9.10.dist-info}/WHEEL

@@ -1,5 +1,4 @@
 Wheel-Version: 1.0
-Generator: bdist_wheel (0.37.1)
+Generator: poetry-core 2.2.1
 Root-Is-Purelib: true
 Tag: py3-none-any
-

folio_migration_tools-1.9.10.dist-info/entry_points.txt

@@ -0,0 +1,3 @@
+[console_scripts]
+folio-migration-tools=folio_migration_tools.__main__:main
+

folio_migration_tools/generate_schemas.py

@@ -1,46 +0,0 @@
-import os
-
-from argparse_prompt import PromptParser
-
-from folio_migration_tools.library_configuration import LibraryConfiguration
-from folio_migration_tools.migration_tasks import *
-from folio_migration_tools.migration_tasks import migration_task_base
-
-
-def parse_args():
-    """Parse CLI Arguments"""
-    parser = PromptParser()
-    parser.add_argument("results_path", help="Path to store schemas")
-    return parser.parse_args()
-
-
-def main():
-    args = parse_args()
-    with open(
-        os.path.join(args.results_path, "LibraryConfigurationSchema.json"), "w"
-    ) as outfile:
-        outfile.write(LibraryConfiguration.schema_json(indent=4))
-
-    for t in inheritors(migration_task_base.MigrationTaskBase):
-        with open(
-            os.path.join(args.results_path, f"{t.__name__}Schema.json"), "w"
-        ) as outfile:
-            outfile.write(t.TaskConfiguration.schema_json(indent=4))
-    print("done generating schemas.")
-    sys.exit(0)
-
-
-def inheritors(base_class):
-    subclasses = set()
-    work = [base_class]
-    while work:
-        parent = work.pop()
-        for child in parent.__subclasses__():
-            if child not in subclasses:
-                subclasses.add(child)
-                work.append(child)
-    return subclasses
-
-
-if __name__ == "__main__":
-    main()

folio_migration_tools/mapping_file_transformation/mapping_file_mapping_base_impl.py

@@ -1,44 +0,0 @@
-from folio_migration_tools.library_configuration import LibraryConfiguration
-from folioclient import FolioClient
-from folio_uuid.folio_namespaces import FOLIONamespaces
-from folio_migration_tools.mapping_file_transformation.mapping_file_mapper_base import (
-    MappingFileMapperBase,
-)
-from folio_migration_tools.report_blurbs import Blurbs
-
-
-class MappingFileMappingBaseImpl(MappingFileMapperBase):
-    def __init__(
-        self,
-        library_configuration: LibraryConfiguration,
-        folio_client: FolioClient,
-        schema: dict,
-        record_map: dict,
-        object_type: FOLIONamespaces,
-        ignore_legacy_identifier: bool = False,
-    ):
-
-        super().__init__(
-            folio_client,
-            schema,
-            record_map,
-            None,
-            object_type,
-            library_configuration,
-            ignore_legacy_identifier,
-        )
-
-    def get_prop(self, legacy_item, folio_prop_name, index_or_id):
-        legacy_item_keys = self.mapped_from_legacy_data.get(folio_prop_name, [])
-
-        if len(legacy_item_keys) == 1 and folio_prop_name in self.mapped_from_values:
-            return self.mapped_from_values.get(folio_prop_name, "")
-
-        legacy_values = MappingFileMapperBase.get_legacy_vals(
-            legacy_item, legacy_item_keys
-        )
-        if len(legacy_item_keys) > 1:
-            self.migration_report.add(
-                Blurbs.Details, f"{legacy_item_keys} were concatenated"
-            )
-        return " ".join(legacy_values).strip()

folio_migration_tools/mapping_file_transformation/user_mapper_base.py

@@ -1,212 +0,0 @@
-import collections.abc
-from abc import abstractmethod
-from typing import Dict
-
-from folio_uuid import FOLIONamespaces, FolioUUID
-from folioclient import FolioClient
-from folio_migration_tools.custom_exceptions import TransformationProcessError
-from folio_migration_tools.library_configuration import LibraryConfiguration
-from folio_migration_tools.mapper_base import MapperBase
-from folio_migration_tools.migration_report import MigrationReport
-
-
-class UserMapperBase(MapperBase):
-    def __init__(
-        self, folio_client: FolioClient, library_configuration: LibraryConfiguration
-    ):
-        super().__init__(library_configuration, folio_client)
-        self.legacy_id_map: Dict[str, str] = {}
-
-        self.migration_report = MigrationReport()
-        self.folio_client = folio_client
-        self.mapped_folio_fields = {}
-        self.ref_data_dicts = {}
-        self.mapped_legacy_fields = {}
-
-    def print_mapping_report(self, total_records):
-        print("\n## Mapped FOLIO fields")
-        d_sorted = {
-            k: self.mapped_folio_fields[k] for k in sorted(self.mapped_folio_fields)
-        }
-        print("FOLIO Field | % | Has value")
-        print("--- | --- | --- :")
-        for k, v in d_sorted.items():
-            mp = v / total_records
-            mapped_per = "{:.0%}".format(max(mp, 0))
-            print(f"{k} | {mapped_per} | {v} ")
-        print("\n## Mapped Legacy fields")
-        d_sorted = {
-            k: self.mapped_legacy_fields[k] for k in sorted(self.mapped_legacy_fields)
-        }
-        print("Legacy Field | % | Has Value")
-        print("--- | --- | --- :")
-        for k, v in d_sorted.items():
-            mp = v / total_records
-            mapped_per = "{:.0%}".format(max(mp, 0))
-            print(f"{k} | {mapped_per} | {v}")
-
-    def report_legacy_mapping(self, legacy_object):
-        for field_name, value in legacy_object.items():
-            v = 1 if value else 0
-            if field_name not in self.mapped_legacy_fields:
-                self.mapped_legacy_fields[field_name] = [1, v]
-            else:
-                self.mapped_legacy_fields[field_name][0] += 1
-                self.mapped_legacy_fields[field_name][1] += v
-
-    def report_folio_mapping(self, folio_object):
-        flat_object = flatten(folio_object)
-        for field_name, value in flat_object.items():
-            v = 1 if value else 0
-            if field_name not in self.mapped_folio_fields:
-                self.mapped_folio_fields[field_name] = [1, v]
-            else:
-                self.mapped_folio_fields[field_name][0] += 1
-                self.mapped_folio_fields[field_name][1] += v
-
-    def instantiate_user(self, legacy_id):
-        if not legacy_id:
-            raise TransformationProcessError(
-                "",
-                (
-                    "Legacy id not present. Have you set "
-                    "the legacyIdentifier in the mapping file?"
-                ),
-            )
-        user_id = str(
-            FolioUUID(self.folio_client.okapi_url, FOLIONamespaces.users, legacy_id)
-        )
-        return {
-            "metadata": self.folio_client.get_metadata_construct(),
-            "id": user_id,
-            "type": "object",
-            "personal": {},
-            "customFields": {},
-        }
-
-    def validate(self, folio_user):
-        failures = []
-        self.migration_report.add(
-            "Number of addresses per user",
-            len(folio_user["personal"].get("addresses", [])),
-        )
-        req_fields = ["username", "email", "active"]
-        for req in req_fields:
-            if req not in folio_user:
-                failures.append(req)
-                self.migration_report.add(
-                    "Failed records that needs to get fixed",
-                    f"Required field {req} is missing from {folio_user['username']}",
-                )
-        if not folio_user["personal"].get("lastName", ""):
-            failures.append("lastName")
-            self.migration_report.add(
-                "Failed records that needs to get fixed",
-                f"Required field personal.lastName is missing from {folio_user['username']}",
-            )
-        if failures:
-            self.migration_report.add("User validation", "Total failed users")
-            for failure in failures:
-                self.migration_report.add("User validation", f"{failure}")
-            raise ValueError(
-                f"Record {folio_user['username']} failed validation {failures}"
-            )
-
-    def write_migration_report(self):
-        for a in self.migration_report:
-            print("")
-            print(f"## {a} - {len(self.migration_report[a])} things")
-            print("Measure | Count")
-            print("--- | ---:")
-            b = self.migration_report[a]
-            sortedlist = [(k, b[k]) for k in sorted(b, key=as_str)]
-            for b in sortedlist:
-                print(f"{b[0]} | {b[1]}")
-
-    def save_migration_report_to_disk(self, file_path, total_records):
-        with open(file_path, "w+") as report_file:
-            for a in self.migration_report:
-                report_file.write("\n")
-                report_file.write(f"## {a} - {len(self.migration_report[a])} things\n")
-                report_file.write("Measure | Count\n")
-                report_file.write("--- | ---:\n")
-                b = self.migration_report[a]
-                sortedlist = [(k, b[k]) for k in sorted(b, key=as_str)]
-                for b in sortedlist:
-                    report_file.write(f"{b[0]} | {b[1]}\n")
-            report_file.write("\n## Mapped FOLIO fields\n")
-            d_sorted = {
-                k: self.mapped_folio_fields[k] for k in sorted(self.mapped_folio_fields)
-            }
-            report_file.write("FOLIO Field | % | Has Value\n")
-            report_file.write("--- | --- | --- | ---:\n")
-            for k, v in d_sorted.items():
-                mp = v / total_records
-                mapped_per = "{:.0%}".format(max(mp, 0))
-                report_file.write(f"{k} | {mapped_per} | {v} \n")
-            report_file.write("\n## Mapped Legacy fields\n")
-            d_sorted = {
-                k: self.mapped_legacy_fields[k]
-                for k in sorted(self.mapped_legacy_fields)
-            }
-            report_file.write("Legacy Field | % | Has Value\n")
-            report_file.write("--- | --- | --- | ---:\n")
-            for k, v in d_sorted.items():
-                mp = v / total_records
-                mapped_per = "{:.0%}".format(max(mp, 0))
-                report_file.write(f"{k} | {mapped_per} | {v}\n")
-
-    @staticmethod
-    def print_dict_to_md_table(my_dict, h1="", h2=""):
-        d_sorted = {k: my_dict[k] for k in sorted(my_dict)}
-        print(f"{h1} | {h2}")
-        print("--- | ---:")
-        for k, v in d_sorted.items():
-            print(f"{k} | {v}")
-
-    def get_ref_data_tuple_by_code(self, ref_data, ref_name, code):
-        return self.get_ref_data_tuple(ref_data, ref_name, code, "code")
-
-    def get_ref_data_tuple_by_name(self, ref_data, ref_name, name):
-        return self.get_ref_data_tuple(ref_data, ref_name, name, "name")
-
-    def get_ref_data_tuple(self, ref_data, ref_name, key_value, key_type):
-        dict_key = f"{ref_name}{key_type}"
-        ref_object = self.ref_data_dicts.get(dict_key, {}).get(
-            key_value.lower().strip(), ()
-        )
-        # logging.info(f"{key_value} - {ref_object} - {dict_key}")
-        if ref_object:
-            return ref_object
-        else:
-            d = {}
-            for r in ref_data:
-                d[r[key_type].lower()] = (r["id"], r["name"])
-            self.ref_data_dicts[dict_key] = d
-        return self.ref_data_dicts.get(dict_key, {}).get(key_value.lower().strip(), ())
-
-    @abstractmethod
-    def do_map(self, legacy_user, object_map):
-        raise NotImplementedError
-
-    @abstractmethod
-    def get_users(self, source_file, file_format: str):
-        raise NotImplementedError
-
-
-def flatten(d, parent_key="", sep="."):
-    items = []
-    for k, v in d.items():
-        new_key = parent_key + sep + k if parent_key else k
-        if isinstance(v, collections.abc.MutableMapping):
-            items.extend(flatten(v, new_key, sep=sep).items())
-        else:
-            items.append((new_key, v))
-    return dict(items)
-
-
-def as_str(s):
-    try:
-        return str(s), ""
-    except ValueError:
-        return "", s