splink 4.0.0.dev6__tar.gz → 4.0.0.dev8__tar.gz

{splink-4.0.0.dev6 → splink-4.0.0.dev8}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: splink
-Version: 4.0.0.dev6
+Version: 4.0.0.dev8
 Summary: Fast probabilistic data linkage at scale
 Home-page: https://github.com/moj-analytical-services/splink
 License: MIT
@@ -27,7 +27,6 @@ Requires-Dist: jsonschema (>=3.2)
 Requires-Dist: numpy (>=1.17.3) ; python_version < "3.12"
 Requires-Dist: numpy (>=1.26.0) ; python_version >= "3.12"
 Requires-Dist: pandas (>1.3.5)
-Requires-Dist: phonetics (>=1.0.5)
 Requires-Dist: psycopg2-binary (>=2.8.0) ; extra == "postgres"
 Requires-Dist: pyspark (>=3.2.1) ; extra == "pyspark" or extra == "spark"
 Requires-Dist: sqlglot (>=13.0.0)
@@ -51,11 +50,11 @@ Splink is a Python package for probabilistic record linkage (entity resolution)
 
 ## Key Features
 
-⚡ **Speed:** Capable of linking a million records on a laptop in around a minute.  
-🎯 **Accuracy:** Support for term frequency adjustments and user-defined fuzzy matching logic.  
-🌐 **Scalability:** Execute linkage in Python (using DuckDB) or big-data backends like AWS Athena or Spark for 100+ million records.  
-🎓 **Unsupervised Learning:** No training data is required for model training.  
-📊 **Interactive Outputs:** A suite of interactive visualisations help users understand their model and diagnose problems.  
+⚡ **Speed:** Capable of linking a million records on a laptop in around a minute.<br>
+🎯 **Accuracy:** Support for term frequency adjustments and user-defined fuzzy matching logic.<br>
+🌐 **Scalability:** Execute linkage in Python (using DuckDB) or big-data backends like AWS Athena or Spark for 100+ million records.<br>
+🎓 **Unsupervised Learning:** No training data is required for model training.<br>
+📊 **Interactive Outputs:** A suite of interactive visualisations help users understand their model and diagnose problems.<br>
 
 Splink's linkage algorithm is based on Fellegi-Sunter's model of record linkage, with various customisations to improve accuracy.
 
@@ -75,19 +74,16 @@ and clusters these links to produce an estimated person ID:
 
 ## What data does Splink work best with?
 
-Before using Splink, input data should be standardised, with consistent column names and formatting (e.g., lowercased, punctuation cleaned up, etc.).
-
 Splink performs best with input data containing **multiple** columns that are **not highly correlated**. For instance, if the entity type is persons, you may have columns for full name, date of birth, and city. If the entity type is companies, you could have columns for name, turnover, sector, and telephone number.
 
-High correlation occurs when the value of a column is highly constrained (predictable) from the value of another column. For example, a 'city' field is almost perfectly correlated with 'postcode'. Gender is highly correlated with 'first name'. Correlation is particularly problematic if **all** of your input columns are highly correlated.
+High correlation occurs when one column is highly predictable from another - for instance, city can be predicted from postcode.  Correlation is particularly problematic if **all** of your input columns are highly correlated.
 
 Splink is not designed for linking a single column containing a 'bag of words'. For example, a table with a single 'company name' column, and no other details.
 
 ## Documentation
 
-The homepage for the Splink documentation can be found [here](https://moj-analytical-services.github.io/splink/). Interactive demos can be found [here](https://github.com/moj-analytical-services/splink/tree/master/docs/demos), or by clicking the following Binder link:
+The homepage for the Splink documentation can be found [here](https://moj-analytical-services.github.io/splink/), including a [tutorial](https://moj-analytical-services.github.io/splink/demos/tutorials/00_Tutorial_Introduction.html) and [examples](https://moj-analytical-services.github.io/splink/demos/examples/examples_index.html) that can be run in the browser.
 
-[![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/moj-analytical-services/splink/binder_branch?labpath=docs%2Fdemos%2Ftutorials%2F00_Tutorial_Introduction.ipynb)
 
 The specification of the Fellegi Sunter statistical model behind `splink` is similar as that used in the R [fastLink package](https://github.com/kosukeimai/fastLink). Accompanying the fastLink package is an [academic paper](http://imai.fas.harvard.edu/research/files/linkage.pdf) that describes this model. The [Splink documentation site](https://moj-analytical-services.github.io/splink/topic_guides/fellegi_sunter.html) and a [series of interactive articles](https://www.robinlinacre.com/probabilistic_linkage/) also explores the theory behind Splink.
 
@@ -143,43 +139,56 @@ The following code demonstrates how to estimate the parameters of a deduplicatio
 For more detailed tutorial, please see [here](https://moj-analytical-services.github.io/splink/demos/tutorials/00_Tutorial_Introduction.html).
 
 ```py
-from splink.duckdb.linker import DuckDBLinker
-import splink.duckdb.comparison_library as cl
-import splink.duckdb.comparison_template_library as ctl
-from splink.duckdb.blocking_rule_library import block_on
-from splink.datasets import splink_datasets
+import splink.comparison_library as cl
+import splink.comparison_template_library as ctl
+from splink import DuckDBAPI, Linker, SettingsCreator, block_on, splink_datasets
+
+db_api = DuckDBAPI()
 
 df = splink_datasets.fake_1000
 
-settings = {
-    "link_type": "dedupe_only",
-    "blocking_rules_to_generate_predictions": [
+settings = SettingsCreator(
+    link_type="dedupe_only",
+    comparisons=[
+        cl.JaroWinklerAtThresholds("first_name", [0.9, 0.7]),
+        cl.JaroAtThresholds("surname", [0.9, 0.7]),
+        ctl.DateComparison(
+            "dob",
+            input_is_string=True,
+            datetime_metrics=["year", "month"],
+            datetime_thresholds=[1, 1],
+        ),
+        cl.ExactMatch("city").configure(term_frequency_adjustments=True),
+        ctl.EmailComparison("email"),
+    ],
+    blocking_rules_to_generate_predictions=[
         block_on("first_name"),
         block_on("surname"),
-    ],
-    "comparisons": [
-        ctl.name_comparison("first_name"),
-        ctl.name_comparison("surname"),
-        ctl.date_comparison("dob", cast_strings_to_date=True),
-        cl.exact_match("city", term_frequency_adjustments=True),
-        ctl.email_comparison("email", include_username_fuzzy_level=False),
-    ],
-}
+    ]
+)
+
+linker = Linker(df, settings, db_api)
+
+linker.training.estimate_probability_two_random_records_match(
+    [block_on("first_name", "surname")],
+    recall=0.7,
+)
 
-linker = DuckDBLinker(df, settings)
-linker.estimate_u_using_random_sampling(max_pairs=1e6)
+linker.training.estimate_u_using_random_sampling(max_pairs=1e6)
 
-blocking_rule_for_training = block_on(["first_name", "surname"])
+linker.training.estimate_parameters_using_expectation_maximisation(
+    block_on("first_name", "surname")
+)
 
-linker.estimate_parameters_using_expectation_maximisation(blocking_rule_for_training, estimate_without_term_frequencies=True)
+linker.training.estimate_parameters_using_expectation_maximisation(block_on("dob"))
 
-blocking_rule_for_training = block_on("dob")
-linker.estimate_parameters_using_expectation_maximisation(blocking_rule_for_training, estimate_without_term_frequencies=True)
+pairwise_predictions = linker.inference.predict(threshold_match_weight=-10)
 
-pairwise_predictions = linker.predict()
+clusters = linker.clustering.cluster_pairwise_predictions_at_threshold(
+    pairwise_predictions, 0.95
+)
 
-clusters = linker.cluster_pairwise_predictions_at_threshold(pairwise_predictions, 0.95)
-clusters.as_pandas_dataframe(limit=5)
+df_clusters = clusters.as_pandas_dataframe(limit=5)
 ```
 
 ## Videos
@@ -187,13 +196,10 @@ clusters.as_pandas_dataframe(limit=5)
 - [A introductory presentation on Splink](https://www.youtube.com/watch?v=msz3T741KQI)
 - [An introduction to the Splink Comparison Viewer dashboard](https://www.youtube.com/watch?v=DNvCMqjipis)
 
-## Charts Gallery
-
-You can see all of the interactive charts provided in Splink by checking out the [Charts Gallery](https://moj-analytical-services.github.io/splink/charts/index.html).
 
 ## Support
 
-To find the best place to ask a question, report a bug or get general advice, please refer to our [Contributing Guide](./CONTRIBUTING.md).
+To find the best place to ask a question, report a bug or get general advice, please refer to our [Guide](./CONTRIBUTING.md).
 
 ## Use Cases
 
@@ -201,8 +207,6 @@ To see how users are using Splink in the wild, check out the [Use Cases](https:/
 
 ## Awards
 
-❓ Future of Government Awards 2023: Open Source Creation - [Shortlisted, result to be announced shortly](https://futureofgovernment.com/en)
-
 🥈 Civil Service Awards 2023: Best Use of Data, Science, and Technology - [Runner up](https://www.civilserviceawards.com/best-use-of-data-science-and-technology-award-2/)
 
 🥇 Analysis in Government Awards 2022: People's Choice Award - [Winner](https://analysisfunction.civilservice.gov.uk/news/announcing-the-winner-of-the-first-analysis-in-government-peoples-choice-award/)

{splink-4.0.0.dev6 → splink-4.0.0.dev8}/README.md

@@ -15,11 +15,11 @@ Splink is a Python package for probabilistic record linkage (entity resolution)
 
 ## Key Features
 
-⚡ **Speed:** Capable of linking a million records on a laptop in around a minute.  
-🎯 **Accuracy:** Support for term frequency adjustments and user-defined fuzzy matching logic.  
-🌐 **Scalability:** Execute linkage in Python (using DuckDB) or big-data backends like AWS Athena or Spark for 100+ million records.  
-🎓 **Unsupervised Learning:** No training data is required for model training.  
-📊 **Interactive Outputs:** A suite of interactive visualisations help users understand their model and diagnose problems.  
+⚡ **Speed:** Capable of linking a million records on a laptop in around a minute.<br>
+🎯 **Accuracy:** Support for term frequency adjustments and user-defined fuzzy matching logic.<br>
+🌐 **Scalability:** Execute linkage in Python (using DuckDB) or big-data backends like AWS Athena or Spark for 100+ million records.<br>
+🎓 **Unsupervised Learning:** No training data is required for model training.<br>
+📊 **Interactive Outputs:** A suite of interactive visualisations help users understand their model and diagnose problems.<br>
 
 Splink's linkage algorithm is based on Fellegi-Sunter's model of record linkage, with various customisations to improve accuracy.
 
@@ -39,19 +39,16 @@ and clusters these links to produce an estimated person ID:
 
 ## What data does Splink work best with?
 
-Before using Splink, input data should be standardised, with consistent column names and formatting (e.g., lowercased, punctuation cleaned up, etc.).
-
 Splink performs best with input data containing **multiple** columns that are **not highly correlated**. For instance, if the entity type is persons, you may have columns for full name, date of birth, and city. If the entity type is companies, you could have columns for name, turnover, sector, and telephone number.
 
-High correlation occurs when the value of a column is highly constrained (predictable) from the value of another column. For example, a 'city' field is almost perfectly correlated with 'postcode'. Gender is highly correlated with 'first name'. Correlation is particularly problematic if **all** of your input columns are highly correlated.
+High correlation occurs when one column is highly predictable from another - for instance, city can be predicted from postcode.  Correlation is particularly problematic if **all** of your input columns are highly correlated.
 
 Splink is not designed for linking a single column containing a 'bag of words'. For example, a table with a single 'company name' column, and no other details.
 
 ## Documentation
 
-The homepage for the Splink documentation can be found [here](https://moj-analytical-services.github.io/splink/). Interactive demos can be found [here](https://github.com/moj-analytical-services/splink/tree/master/docs/demos), or by clicking the following Binder link:
+The homepage for the Splink documentation can be found [here](https://moj-analytical-services.github.io/splink/), including a [tutorial](https://moj-analytical-services.github.io/splink/demos/tutorials/00_Tutorial_Introduction.html) and [examples](https://moj-analytical-services.github.io/splink/demos/examples/examples_index.html) that can be run in the browser.
 
-[![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/moj-analytical-services/splink/binder_branch?labpath=docs%2Fdemos%2Ftutorials%2F00_Tutorial_Introduction.ipynb)
 
 The specification of the Fellegi Sunter statistical model behind `splink` is similar as that used in the R [fastLink package](https://github.com/kosukeimai/fastLink). Accompanying the fastLink package is an [academic paper](http://imai.fas.harvard.edu/research/files/linkage.pdf) that describes this model. The [Splink documentation site](https://moj-analytical-services.github.io/splink/topic_guides/fellegi_sunter.html) and a [series of interactive articles](https://www.robinlinacre.com/probabilistic_linkage/) also explores the theory behind Splink.
 
@@ -107,43 +104,56 @@ The following code demonstrates how to estimate the parameters of a deduplicatio
 For more detailed tutorial, please see [here](https://moj-analytical-services.github.io/splink/demos/tutorials/00_Tutorial_Introduction.html).
 
 ```py
-from splink.duckdb.linker import DuckDBLinker
-import splink.duckdb.comparison_library as cl
-import splink.duckdb.comparison_template_library as ctl
-from splink.duckdb.blocking_rule_library import block_on
-from splink.datasets import splink_datasets
+import splink.comparison_library as cl
+import splink.comparison_template_library as ctl
+from splink import DuckDBAPI, Linker, SettingsCreator, block_on, splink_datasets
+
+db_api = DuckDBAPI()
 
 df = splink_datasets.fake_1000
 
-settings = {
-    "link_type": "dedupe_only",
-    "blocking_rules_to_generate_predictions": [
+settings = SettingsCreator(
+    link_type="dedupe_only",
+    comparisons=[
+        cl.JaroWinklerAtThresholds("first_name", [0.9, 0.7]),
+        cl.JaroAtThresholds("surname", [0.9, 0.7]),
+        ctl.DateComparison(
+            "dob",
+            input_is_string=True,
+            datetime_metrics=["year", "month"],
+            datetime_thresholds=[1, 1],
+        ),
+        cl.ExactMatch("city").configure(term_frequency_adjustments=True),
+        ctl.EmailComparison("email"),
+    ],
+    blocking_rules_to_generate_predictions=[
         block_on("first_name"),
         block_on("surname"),
-    ],
-    "comparisons": [
-        ctl.name_comparison("first_name"),
-        ctl.name_comparison("surname"),
-        ctl.date_comparison("dob", cast_strings_to_date=True),
-        cl.exact_match("city", term_frequency_adjustments=True),
-        ctl.email_comparison("email", include_username_fuzzy_level=False),
-    ],
-}
+    ]
+)
+
+linker = Linker(df, settings, db_api)
+
+linker.training.estimate_probability_two_random_records_match(
+    [block_on("first_name", "surname")],
+    recall=0.7,
+)
 
-linker = DuckDBLinker(df, settings)
-linker.estimate_u_using_random_sampling(max_pairs=1e6)
+linker.training.estimate_u_using_random_sampling(max_pairs=1e6)
 
-blocking_rule_for_training = block_on(["first_name", "surname"])
+linker.training.estimate_parameters_using_expectation_maximisation(
+    block_on("first_name", "surname")
+)
 
-linker.estimate_parameters_using_expectation_maximisation(blocking_rule_for_training, estimate_without_term_frequencies=True)
+linker.training.estimate_parameters_using_expectation_maximisation(block_on("dob"))
 
-blocking_rule_for_training = block_on("dob")
-linker.estimate_parameters_using_expectation_maximisation(blocking_rule_for_training, estimate_without_term_frequencies=True)
+pairwise_predictions = linker.inference.predict(threshold_match_weight=-10)
 
-pairwise_predictions = linker.predict()
+clusters = linker.clustering.cluster_pairwise_predictions_at_threshold(
+    pairwise_predictions, 0.95
+)
 
-clusters = linker.cluster_pairwise_predictions_at_threshold(pairwise_predictions, 0.95)
-clusters.as_pandas_dataframe(limit=5)
+df_clusters = clusters.as_pandas_dataframe(limit=5)
 ```
 
 ## Videos
@@ -151,13 +161,10 @@ clusters.as_pandas_dataframe(limit=5)
 - [A introductory presentation on Splink](https://www.youtube.com/watch?v=msz3T741KQI)
 - [An introduction to the Splink Comparison Viewer dashboard](https://www.youtube.com/watch?v=DNvCMqjipis)
 
-## Charts Gallery
-
-You can see all of the interactive charts provided in Splink by checking out the [Charts Gallery](https://moj-analytical-services.github.io/splink/charts/index.html).
 
 ## Support
 
-To find the best place to ask a question, report a bug or get general advice, please refer to our [Contributing Guide](./CONTRIBUTING.md).
+To find the best place to ask a question, report a bug or get general advice, please refer to our [Guide](./CONTRIBUTING.md).
 
 ## Use Cases
 
@@ -165,8 +172,6 @@ To see how users are using Splink in the wild, check out the [Use Cases](https:/
 
 ## Awards
 
-❓ Future of Government Awards 2023: Open Source Creation - [Shortlisted, result to be announced shortly](https://futureofgovernment.com/en)
-
 🥈 Civil Service Awards 2023: Best Use of Data, Science, and Technology - [Runner up](https://www.civilserviceawards.com/best-use-of-data-science-and-technology-award-2/)
 
 🥇 Analysis in Government Awards 2022: People's Choice Award - [Winner](https://analysisfunction.civilservice.gov.uk/news/announcing-the-winner-of-the-first-analysis-in-government-peoples-choice-award/)

{splink-4.0.0.dev6 → splink-4.0.0.dev8}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "splink"
-version = "4.0.0.dev6"
+version = "4.0.0.dev8"
 description = "Fast probabilistic data linkage at scale"
 authors = ["Robin Linacre <robinlinacre@hotmail.com>", "Sam Lindsay", "Theodore Manassis", "Tom Hepworth", "Andy Bond", "Ross Kennedy"]
 license = "MIT"
@@ -17,7 +17,6 @@ duckdb = ">=0.9.2"
 sqlglot = ">=13.0.0"
 altair = "^5.0.1"
 Jinja2 = ">=3.0.3"
-phonetics = ">=1.0.5"
 
 # need to manually specify numpy versions suitable for CI
 # 1.24.4 works with python 3.8, but not 3.12

{splink-4.0.0.dev6 → splink-4.0.0.dev8}/splink/__init__.py

@@ -44,7 +44,7 @@ def __getattr__(name):
     raise AttributeError(f"module 'splink' has no attribute '{name}'") from None
 
 
-__version__ = "4.0.0.dev6"
+__version__ = "4.0.0.dev8"
 
 
 __all__ = [

splink-4.0.0.dev8/splink/backends/athena.py

@@ -0,0 +1,3 @@
+from splink.internals.athena.database_api import AthenaAPI
+
+__all__ = ["AthenaAPI"]

{splink-4.0.0.dev6 → splink-4.0.0.dev8}/splink/blocking_analysis.py

@@ -2,10 +2,12 @@ from .internals.blocking_analysis import (
     count_comparisons_from_blocking_rule,
     cumulative_comparisons_to_be_scored_from_blocking_rules_chart,
     cumulative_comparisons_to_be_scored_from_blocking_rules_data,
+    n_largest_blocks,
 )
 
 __all__ = [
     "count_comparisons_from_blocking_rule",
     "cumulative_comparisons_to_be_scored_from_blocking_rules_chart",
     "cumulative_comparisons_to_be_scored_from_blocking_rules_data",
+    "n_largest_blocks",
 ]

{splink-4.0.0.dev6 → splink-4.0.0.dev8}/splink/comparison_library.py

@@ -4,13 +4,18 @@ from splink.internals.comparison_library import (
     ArrayIntersectAtSizes,
     CustomComparison,
     DamerauLevenshteinAtThresholds,
+    DateOfBirthComparison,
     DistanceFunctionAtThresholds,
     DistanceInKMAtThresholds,
+    EmailComparison,
     ExactMatch,
+    ForenameSurnameComparison,
     JaccardAtThresholds,
     JaroAtThresholds,
     JaroWinklerAtThresholds,
     LevenshteinAtThresholds,
+    NameComparison,
+    PostcodeComparison,
 )
 
 __all__ = [
@@ -26,4 +31,9 @@ __all__ = [
     "AbsoluteDateDifferenceAtThresholds",
     "ArrayIntersectAtSizes",
     "DistanceInKMAtThresholds",
+    "DateOfBirthComparison",
+    "EmailComparison",
+    "ForenameSurnameComparison",
+    "NameComparison",
+    "PostcodeComparison",
 ]

splink-4.0.0.dev8/splink/exploratory.py

@@ -0,0 +1,5 @@
+from .internals import similarity_analysis
+from .internals.completeness import completeness_chart
+from .internals.profile_data import profile_columns
+
+__all__ = ["completeness_chart", "profile_columns", "similarity_analysis"]

splink-4.0.0.dev8/splink/internals/athena/database_api.py

@@ -0,0 +1,266 @@
+from __future__ import annotations
+
+import json
+import logging
+import os
+from typing import Any, Sequence
+
+import awswrangler as wr
+import boto3
+import pandas as pd
+
+from ..database_api import AcceptableInputTableType, DatabaseAPI
+from ..dialects import AthenaDialect
+from ..sql_transform import sqlglot_transform_sql
+from .athena_helpers.athena_transforms import cast_concat_as_varchar
+from .athena_helpers.athena_utils import (
+    _verify_athena_inputs,
+)
+from .dataframe import AthenaDataFrame
+
+logger = logging.getLogger(__name__)
+
+
+# Dict because there's not really a 'tablish' type in Athena
+class AthenaAPI(DatabaseAPI[dict[str, Any]]):
+    sql_dialect = AthenaDialect()
+
+    def __init__(
+        self,
+        boto3_session: boto3.session.Session,
+        output_database: str,
+        output_bucket: str,
+        output_filepath: str = None,
+    ):
+        super().__init__()
+        if not type(boto3_session) == boto3.session.Session:
+            raise ValueError("Please enter a valid boto3 session object.")
+
+        self._sql_dialect_ = "presto"
+
+        _verify_athena_inputs(output_database, output_bucket, boto3_session)
+        self.boto3_session = boto3_session
+        self.output_schema = output_database
+        self.output_bucket = output_bucket
+
+        # If the default folder is blank, name it `splink_warehouse`
+        if output_filepath:
+            self.output_filepath = output_filepath
+        else:
+            self.output_filepath = "splink_warehouse"
+
+        self.ctas_query_info: dict[str, Any] = {}
+
+        # TODO: How to run this check without the input_tables?
+        # Run a quick check against our inputs to check if they
+        # exist in the database
+        # for table in input_tables:
+        #     if not isinstance(table, self.accepted_df_dtypes):
+        #         db, tb = self.get_schema_info(table)
+        #         self._check_table_exists(db, tb)
+
+    @property
+    def s3_output(self):
+        out_path = os.path.join(
+            "s3://",
+            self.output_bucket,
+            self.output_filepath,
+            self._cache_uid,  # added in the super() step
+        )
+        if out_path[-1] != "/":
+            out_path += "/"
+
+        return out_path
+
+    # TODO: Should output_filepath use getters and setters?
+    def change_output_filepath(self, new_filepath):
+        self.output_filepath = new_filepath
+
+    def get_schema_info(self, input_table: str) -> list[str]:
+        t = input_table.split(".")
+        return t if len(t) > 1 else [self.output_schema, input_table]
+
+    def _check_table_exists(self, db: str, tb: str) -> None:
+        # A quick function to check if a table exists
+        # and spit out a warning if it is not found.
+        table_exists = wr.catalog.does_table_exist(
+            database=db,
+            table=tb,
+            boto3_session=self.boto3_session,
+        )
+        if not table_exists:
+            raise wr.exceptions.InvalidTable(
+                f"Table '{tb}' was not found within your selected "
+                f"database '{db}'. Please verify your input table "
+                "exists."
+            )
+
+    def _drop_table_from_database_if_exists(self, table):
+        return wr.catalog.delete_table_if_exists(
+            database=self.output_schema, table=table, boto3_session=self.boto3_session
+        )
+
+    def _delete_table_from_s3(self, physical_name):
+        path = f"{self.s3_output}{physical_name}/"
+        # delete our folder
+        wr.s3.delete_objects(
+            path=path,
+            use_threads=True,
+            boto3_session=self.boto3_session,
+        )
+
+        metadata = self.ctas_query_info[physical_name]
+        if "output_location" in metadata:
+            metadata_urls = [
+                # metadata output location
+                f"{metadata['output_location']}.metadata",
+                # manifest location
+                metadata["manifest_location"],
+            ]
+            # delete our metadata
+            wr.s3.delete_objects(
+                path=metadata_urls,
+                use_threads=True,
+                boto3_session=self.boto3_session,
+            )
+
+        self.ctas_query_info.pop(physical_name)
+
+    def delete_table_from_database(self, name):
+        if name in self.ctas_query_info:
+            # Use ctas metadata to delete backing data
+            self._delete_table_from_s3(name)
+        else:
+            # If the location we want to write to already exists,
+            # clean this before continuing.
+            loc = f"{self.s3_output}{name}"
+            folder_exists = wr.s3.list_directories(
+                loc,
+                boto3_session=self.boto3_session,
+            )
+            if folder_exists:
+                # This will only delete objects we are required to delete
+                wr.s3.delete_objects(
+                    path=loc,
+                    use_threads=True,
+                    boto3_session=self.boto3_session,
+                )
+
+        self._drop_table_from_database_if_exists(name)
+
+    def _register_data_on_s3(self, table, alias):
+        out_loc = f"{self.s3_output}{alias}"
+
+        wr.s3.to_parquet(
+            df=table,
+            path=out_loc,
+            dataset=True,
+            mode="overwrite",
+            database=self.output_schema,
+            table=alias,
+            boto3_session=self.boto3_session,
+            compression="snappy",
+            use_threads=True,
+        )
+        # Construct the ctas metadata that we require
+        ctas_metadata = {
+            "ctas_database": self.output_schema,
+            "ctas_table": alias,
+        }
+        self.ctas_query_info.update({alias: ctas_metadata})
+
+    def _table_registration(self, input, table_name):
+        if isinstance(input, dict):
+            input = pd.DataFrame(input)
+        elif isinstance(input, list):
+            input = pd.DataFrame.from_records(input)
+
+        # Errors if an invalid data type is passed
+        self._register_data_on_s3(input, table_name)
+
+    def table_to_splink_dataframe(self, templated_name, physical_name):
+        return AthenaDataFrame(templated_name, physical_name, self)
+
+    def _create_table(self, sql, physical_name):
+        ctas_metadata = wr.athena.create_ctas_table(
+            sql=sql,
+            database=self.output_schema,
+            ctas_table=physical_name,
+            storage_format="parquet",
+            write_compression="snappy",
+            boto3_session=self.boto3_session,
+            s3_output=self.s3_output,
+            wait=True,
+        )
+        return ctas_metadata
+
+    def table_exists_in_database(self, table_name):
+        return wr.catalog.does_table_exist(
+            database=self.output_schema,
+            table=table_name,
+            boto3_session=self.boto3_session,
+        )
+
+    def _extract_ctas_metadata(self, ctas_metadata):
+        query_meta = ctas_metadata.pop("ctas_query_metadata")
+        out_locs = {
+            "output_location": query_meta.output_location,
+            "manifest_location": query_meta.manifest_location,
+        }
+        ctas_metadata.update(out_locs)
+        return ctas_metadata
+
+    def _setup_for_execute_sql(self, sql: str, physical_name: str) -> str:
+        self.delete_table_from_database(physical_name)
+        # This is a hack because execute_sql_against_backend
+        # needs the physical name but the _execute_sql_against_backend
+        # method just takes a  string
+        return json.dumps(
+            {
+                "physical_name": physical_name,
+                "sql": sql,
+            }
+        )
+
+    def _execute_sql_against_backend(self, sql):
+        sql_dict = json.loads(sql)
+        physical_name = sql_dict["physical_name"]
+        sql_query = sql_dict["sql"]
+        sql_query = sqlglot_transform_sql(
+            sql_query, cast_concat_as_varchar, dialect="presto"
+        )
+        sql_query = sql_query.replace("FLOAT", "double").replace("float", "double")
+
+        # create our table on athena and extract the metadata information
+        query_metadata = self._create_table(sql_query, physical_name=physical_name)
+        # append our metadata locations
+        query_metadata = self._extract_ctas_metadata(query_metadata)
+        self.ctas_query_info.update({physical_name: query_metadata})
+
+        return query_metadata
+
+    @property
+    def accepted_df_dtypes(self):
+        accepted_df_dtypes = [pd.DataFrame]
+        try:
+            # If pyarrow is installed, add to the accepted list
+            import pyarrow as pa
+
+            accepted_df_dtypes.append(pa.lib.Table)
+        except ImportError:
+            pass
+        return accepted_df_dtypes
+
+    def load_from_file(self, file_path: str) -> str:
+        raise NotImplementedError(
+            "Loading from file is not supported for Athena. "
+            "Please use the `table` method to load data."
+        )
+
+    def process_input_tables(
+        self, input_tables: Sequence[AcceptableInputTableType]
+    ) -> Sequence[AcceptableInputTableType]:
+        input_tables = super().process_input_tables(input_tables)
+        return [
+            self.load_from_file(t) if isinstance(t, str) else t for t in input_tables
+        ]