diffsync 2.1.0__tar.gz → 2.2.1__tar.gz

{diffsync-2.1.0 → diffsync-2.2.1}/LICENSE

@@ -1,12 +1,12 @@
-Copyright 2020-2023 Network to Code <info@networktocode.com>
-Network to Code, LLC
+Apache Software License 2.0
+
+Copyright (c) 2025, Network to Code, LLC
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
-
 You may obtain a copy of the License at
 
-   http://www.apache.org/licenses/LICENSE-2.0
+http://www.apache.org/licenses/LICENSE-2.0
 
 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,

{diffsync-2.1.0 → diffsync-2.2.1}/PKG-INFO

@@ -1,15 +1,16 @@
 Metadata-Version: 2.3
 Name: diffsync
-Version: 2.1.0
+Version: 2.2.1
 Summary: Library to easily sync/diff/update 2 different data sources
 License: Apache-2.0
 Keywords: source-of-truth,synchronization
 Author: Network to Code, LLC
 Author-email: info@networktocode.com
-Requires-Python: >=3.9,<4.0
+Requires-Python: >=3.10,<3.14
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
 Classifier: License :: OSI Approved :: Apache Software License
 Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
@@ -31,7 +32,7 @@ DiffSync is a utility library that can be used to compare and synchronize differ
 
 For example, it can be used to compare a list of devices from 2 inventory systems and, if required, synchronize them in either direction.
 
-# Primary Use Cases
+## Primary Use Cases
 
 DiffSync is at its most useful when you have multiple sources or sets of data to compare and/or synchronize, and especially if any of the following are true:
 
@@ -40,7 +41,7 @@ DiffSync is at its most useful when you have multiple sources or sets of data to
 - If various types of data in your data set naturally form a tree-like or parent-child relationship with other data.
 - If the different data sets have some attributes in common and other attributes that are exclusive to one or the other.
 
-# Overview of DiffSync
+## Overview of DiffSync
 
 DiffSync acts as an intermediate translation layer between all of the data sets you are diffing and/or syncing. In practical terms, this means that to use DiffSync, you will define a set of data models as well as the “adapters” needed to translate between each base data source and the data model. In Python terms, the adapters will be subclasses of the `Adapter` class, and each data model class will be a subclass of the `DiffSyncModel` class.
 
@@ -55,7 +56,7 @@ You can also ask DiffSync to “sync” one data set onto the other, and it will
 
 ![DiffSync Sync](https://raw.githubusercontent.com/networktocode/diffsync/develop/docs/images/diffsync_sync.png "DiffSync Sync")
 
-# Simple Example
+## Simple Example
 
 ```python
 A = DiffSyncSystemA()
@@ -77,34 +78,51 @@ A.sync_to(B)
 
 > You may wish to peruse the `diffsync` [GitHub topic](https://github.com/topics/diffsync) for examples of projects using this library.
 
-# Documentation
+## Documentation
 
-The documentation is available [on Read The Docs](https://diffsync.readthedocs.io/en/latest/index.html).
+Full documentation for this library can be found over on the [Diffsync Docs](https://diffsync.readthedocs.io/) website:
 
-# Installation
+- [User Guide](https://diffsync.readthedocs.io/user/app_overview/) - Overview, Using the Library, Getting Started.
+- [Administrator Guide](https://diffsync.readthedocs.io/admin/install/) - How to Install, Configure, Upgrade, or Uninstall the Library.
+- [Developer Guide](https://diffsync.readthedocs.io/dev/contributing/) - Extending the Library, Code Reference, Contribution Guide.
+- [Release Notes / Changelog](https://diffsync.readthedocs.io/admin/release_notes/).
+- [Frequently Asked Questions](https://diffsync.readthedocs.io/user/faq/).
 
-### Option 1: Install from PyPI.
+## Installation
 
-```
-$ pip install diffsync
+### Option 1: Install from PyPI
+
+```shell
+pip install diffsync
 ```
 
 ### Option 2: Install from a GitHub branch, such as main as shown below.
+
+```shell
+pip install git+https://github.com/networktocode/diffsync.git@main
 ```
-$ pip install git+https://github.com/networktocode/diffsync.git@main
-```
 
-# Contributing
+## Contributing
+
 Pull requests are welcomed and automatically built and tested against multiple versions of Python through GitHub Actions.
 
-The project is following Network to Code software development guidelines and are leveraging the following:
+The project is following Network to Code software development guidelines and is leveraging the following:
 
-- Black, Pylint, Bandit, flake8, and pydocstyle, mypy for Python linting, formatting and type hint checking.
+- Ruff, mypy for Python linting, formatting and type hint checking.
 - pytest, coverage, and unittest for unit tests.
 
 You can ensure your contribution adheres to these checks by running `invoke tests` from the CLI.
 The command `invoke build` builds a docker container with all the necessary dependencies (including the redis backend) locally to facilitate the execution of these tests.
 
-# Questions
-Please see the [documentation](https://diffsync.readthedocs.io/en/latest/index.html) for detailed documentation on how to use `diffsync`. For any additional questions or comments, feel free to swing by the [Network to Code slack channel](https://networktocode.slack.com/) (channel #networktocode). Sign up [here](http://slack.networktocode.com/)
+## Contributing to the Documentation
+
+You can find all the Markdown source for the App documentation under the [`docs`](https://github.com/networktocode/diffsync/tree/develop/docs) folder in this repository. For simple edits, a Markdown capable editor is sufficient: clone the repository and edit away.
+
+If you need to view the fully-generated documentation site, you can build it with [MkDocs](https://www.mkdocs.org/). A container hosting the documentation can be started using the `invoke` commands (details in the [Development Environment Guide](https://diffsync/dev/dev_environment/#docker-development-environment)) on [http://localhost:8001](http://localhost:8001). Using this container, as your changes to the documentation are saved, they will be automatically rebuilt and any pages currently being viewed will be reloaded in your browser.
+
+Any PRs with fixes or improvements are very welcome!
+
+## Questions
+
+For any questions or comments, please check the [FAQ](https://diffsync.readthedocs.io/en/latest/user/faq/) first. Feel free to also swing by the [Network to Code Slack](https://networktocode.slack.com/) (channel `#networktocode`), sign up [here](http://slack.networktocode.com/) if you don't have an account.
 

{diffsync-2.1.0 → diffsync-2.2.1}/README.md

@@ -4,7 +4,7 @@ DiffSync is a utility library that can be used to compare and synchronize differ
 
 For example, it can be used to compare a list of devices from 2 inventory systems and, if required, synchronize them in either direction.
 
-# Primary Use Cases
+## Primary Use Cases
 
 DiffSync is at its most useful when you have multiple sources or sets of data to compare and/or synchronize, and especially if any of the following are true:
 
@@ -13,7 +13,7 @@ DiffSync is at its most useful when you have multiple sources or sets of data to
 - If various types of data in your data set naturally form a tree-like or parent-child relationship with other data.
 - If the different data sets have some attributes in common and other attributes that are exclusive to one or the other.
 
-# Overview of DiffSync
+## Overview of DiffSync
 
 DiffSync acts as an intermediate translation layer between all of the data sets you are diffing and/or syncing. In practical terms, this means that to use DiffSync, you will define a set of data models as well as the “adapters” needed to translate between each base data source and the data model. In Python terms, the adapters will be subclasses of the `Adapter` class, and each data model class will be a subclass of the `DiffSyncModel` class.
 
@@ -28,7 +28,7 @@ You can also ask DiffSync to “sync” one data set onto the other, and it will
 
 ![DiffSync Sync](https://raw.githubusercontent.com/networktocode/diffsync/develop/docs/images/diffsync_sync.png "DiffSync Sync")
 
-# Simple Example
+## Simple Example
 
 ```python
 A = DiffSyncSystemA()
@@ -50,33 +50,50 @@ A.sync_to(B)
 
 > You may wish to peruse the `diffsync` [GitHub topic](https://github.com/topics/diffsync) for examples of projects using this library.
 
-# Documentation
+## Documentation
 
-The documentation is available [on Read The Docs](https://diffsync.readthedocs.io/en/latest/index.html).
+Full documentation for this library can be found over on the [Diffsync Docs](https://diffsync.readthedocs.io/) website:
 
-# Installation
+- [User Guide](https://diffsync.readthedocs.io/user/app_overview/) - Overview, Using the Library, Getting Started.
+- [Administrator Guide](https://diffsync.readthedocs.io/admin/install/) - How to Install, Configure, Upgrade, or Uninstall the Library.
+- [Developer Guide](https://diffsync.readthedocs.io/dev/contributing/) - Extending the Library, Code Reference, Contribution Guide.
+- [Release Notes / Changelog](https://diffsync.readthedocs.io/admin/release_notes/).
+- [Frequently Asked Questions](https://diffsync.readthedocs.io/user/faq/).
 
-### Option 1: Install from PyPI.
+## Installation
 
-```
-$ pip install diffsync
+### Option 1: Install from PyPI
+
+```shell
+pip install diffsync
 ```
 
 ### Option 2: Install from a GitHub branch, such as main as shown below.
+
+```shell
+pip install git+https://github.com/networktocode/diffsync.git@main
 ```
-$ pip install git+https://github.com/networktocode/diffsync.git@main
-```
 
-# Contributing
+## Contributing
+
 Pull requests are welcomed and automatically built and tested against multiple versions of Python through GitHub Actions.
 
-The project is following Network to Code software development guidelines and are leveraging the following:
+The project is following Network to Code software development guidelines and is leveraging the following:
 
-- Black, Pylint, Bandit, flake8, and pydocstyle, mypy for Python linting, formatting and type hint checking.
+- Ruff, mypy for Python linting, formatting and type hint checking.
 - pytest, coverage, and unittest for unit tests.
 
 You can ensure your contribution adheres to these checks by running `invoke tests` from the CLI.
 The command `invoke build` builds a docker container with all the necessary dependencies (including the redis backend) locally to facilitate the execution of these tests.
 
-# Questions
-Please see the [documentation](https://diffsync.readthedocs.io/en/latest/index.html) for detailed documentation on how to use `diffsync`. For any additional questions or comments, feel free to swing by the [Network to Code slack channel](https://networktocode.slack.com/) (channel #networktocode). Sign up [here](http://slack.networktocode.com/)
+## Contributing to the Documentation
+
+You can find all the Markdown source for the App documentation under the [`docs`](https://github.com/networktocode/diffsync/tree/develop/docs) folder in this repository. For simple edits, a Markdown capable editor is sufficient: clone the repository and edit away.
+
+If you need to view the fully-generated documentation site, you can build it with [MkDocs](https://www.mkdocs.org/). A container hosting the documentation can be started using the `invoke` commands (details in the [Development Environment Guide](https://diffsync/dev/dev_environment/#docker-development-environment)) on [http://localhost:8001](http://localhost:8001). Using this container, as your changes to the documentation are saved, they will be automatically rebuilt and any pages currently being viewed will be reloaded in your browser.
+
+Any PRs with fixes or improvements are very welcome!
+
+## Questions
+
+For any questions or comments, please check the [FAQ](https://diffsync.readthedocs.io/en/latest/user/faq/) first. Feel free to also swing by the [Network to Code Slack](https://networktocode.slack.com/) (channel `#networktocode`), sign up [here](http://slack.networktocode.com/) if you don't have an account.

{diffsync-2.1.0 → diffsync-2.2.1}/diffsync/__init__.py

@@ -16,6 +16,7 @@ limitations under the License.
 """
 
 import sys
+from copy import deepcopy
 from inspect import isclass
 from typing import (
     Any,
@@ -60,7 +61,7 @@ StrType = str
 class DiffSyncModel(BaseModel):
     """Base class for all DiffSync object models.
 
-    Note that read-only APIs of this class are implemented as `get_*()` functions rather than as properties;
+    Note that read-only APIs of this class are implemented as `get_*()` methods rather than as properties;
     this is intentional as specific model classes may want to use these names (`type`, `keys`, `attrs`, etc.)
     as model attributes and we want to avoid any ambiguity or collisions.
 
@@ -158,9 +159,11 @@ class DiffSyncModel(BaseModel):
             raise AttributeError(f"Fields {attr_child_overlap} are included in both _attributes and _children.")
 
     def __repr__(self) -> str:
+        """Return a string representation of this DiffSyncModel."""
         return f'{self.get_type()} "{self.get_unique_id()}"'
 
     def __str__(self) -> str:
+        """Return a string representation of this DiffSyncModel."""
         return self.get_unique_id()
 
     def dict(self, **kwargs: Any) -> Dict:
@@ -305,7 +308,7 @@ class DiffSyncModel(BaseModel):
 
     @classmethod
     def get_type(cls) -> StrType:
-        """Return the type AKA modelname of the object or the class
+        """Return the type AKA modelname of the object or the class.
 
         Returns:
             str: modelname of the class, used in to store all objects
@@ -447,7 +450,6 @@ class Adapter:  # pylint: disable=too-many-public-methods
 
         Subclasses should be careful to call super().__init__() if they override this method.
         """
-
         if isinstance(internal_storage_engine, BaseStore):
             self.store = internal_storage_engine
             self.store.adapter = self
@@ -480,6 +482,19 @@ class Adapter:  # pylint: disable=too-many-public-methods
             if not isclass(value) or not issubclass(value, DiffSyncModel):
                 raise AttributeError(f'top_level references attribute "{name}" but it is not a DiffSyncModel subclass!')
 
+    def __new__(cls, **kwargs):  # type: ignore[no-untyped-def]
+        """Document keyword arguments that were used to initialize Adapter."""
+        meta_kwargs = {}
+        for key, value in kwargs.items():
+            try:
+                meta_kwargs[key] = deepcopy(value)
+            except (TypeError, AttributeError):
+                # Some objects (e.g. Kafka Consumer, DB connections) cannot be deep copied
+                meta_kwargs[key] = value
+        instance = super().__new__(cls)
+        instance._meta_kwargs = meta_kwargs
+        return instance
+
     def __str__(self) -> StrType:
         """String representation of an Adapter."""
         if self.type != self.name:
@@ -487,6 +502,7 @@ class Adapter:  # pylint: disable=too-many-public-methods
         return self.type
 
     def __repr__(self) -> StrType:
+        """Representation of an Adapter."""
         return f"<{str(self)}>"
 
     def __len__(self) -> int:
@@ -575,6 +591,7 @@ class Adapter:  # pylint: disable=too-many-public-methods
             callback: Function with parameters (stage, current, total), to be called at intervals as the calculation of
                 the diff and subsequent sync proceed.
             diff: An existing diff to be used rather than generating a completely new diff.
+
         Returns:
             Diff between origin object and source
         Raises:
@@ -619,6 +636,7 @@ class Adapter:  # pylint: disable=too-many-public-methods
             callback: Function with parameters (stage, current, total), to be called at intervals as the calculation of
                 the diff and subsequent sync proceed.
             diff: An existing diff that will be used when determining what needs to be synced.
+
         Returns:
             Diff between origin object and target
         Raises:
@@ -729,7 +747,7 @@ class Adapter:  # pylint: disable=too-many-public-methods
         obj: Union[StrType, DiffSyncModel, Type[DiffSyncModel]],
         identifier: Union[StrType, Dict],
     ) -> Optional[DiffSyncModel]:
-        """Get one object from the data store based on its unique id or get a None
+        """Get one object from the data store based on its unique id or get a None.
 
         Args:
             obj: DiffSyncModel class or instance, or modelname string, that defines the type of the object to retrieve

{diffsync-2.1.0 → diffsync-2.2.1}/diffsync/exceptions.py

@@ -15,7 +15,7 @@ See the License for the specific language governing permissions and
 limitations under the License.
 """
 
-from typing import TYPE_CHECKING, Union, Any
+from typing import TYPE_CHECKING, Any, Union
 
 if TYPE_CHECKING:
     from diffsync import DiffSyncModel

diffsync-2.2.1/diffsync/log.py

@@ -0,0 +1,74 @@
+"""
+Logging utilities for diffsync.
+
+This module contains helpers and wrappers for making logging more consistent across applications.
+
+How to use me:
+
+    >>> from diffsync.log import initialize_logging
+    >>> log = initialize_logging(level="debug")
+"""
+
+import logging.config
+
+APP = "diffsync"
+
+
+def initialize_logging(config=None, level="INFO", filename=None):
+    """Initialize logging using sensible defaults.
+
+    Args:
+        config (dict): User provided configuration dictionary.
+        level (str): The level of logging for STDOUT logging.
+        filename (str): Where to output debug logging to file.
+
+    """
+    if not config:
+        config = {
+            "version": 1,
+            "disable_existing_loggers": False,
+            "formatters": {
+                "standard": {
+                    "format": "%(asctime)s [%(levelname)s] %(name)s: %(message)s",
+                    "datefmt": "%Y-%m-%dT%H:%M:%S%z",
+                },
+                "debug": {
+                    "format": "%(asctime)s [%(levelname)s] [%(module)s] [%(funcName)s] %(name)s: %(message)s",
+                    "datefmt": "%Y-%m-%dT%H:%M:%S%z",
+                },
+            },
+            "handlers": {
+                "standard": {
+                    "class": "logging.StreamHandler",
+                    "formatter": "standard",
+                    "level": level.upper(),
+                },
+            },
+            "loggers": {
+                "": {
+                    "handlers": ["standard"],
+                    "level": "DEBUG",
+                }
+            },
+        }
+
+        # If a filename is passed in, let's add a FileHandler
+        if filename:
+            config["handlers"].update(
+                {
+                    "file_output": {
+                        "class": "logging.FileHandler",
+                        "formatter": "debug",
+                        "level": "DEBUG",
+                        "filename": filename,
+                    }
+                }
+            )
+            config["loggers"][""]["handlers"].append("file_output")
+
+    # Configure the logging
+    logging.config.dictConfig(config)
+
+    # Initialize root logger and advise logging has been initialized
+    log = logging.getLogger(APP)
+    log.debug("Logging initialized.")

{diffsync-2.1.0 → diffsync-2.2.1}/diffsync/store/__init__.py

@@ -1,13 +1,13 @@
 """BaseStore module."""
 
-from typing import Dict, List, Tuple, Type, Union, TYPE_CHECKING, Optional, Set, Any
+from typing import TYPE_CHECKING, Any, Dict, List, Optional, Set, Tuple, Type, Union
+
 import structlog  # type: ignore
 
 from diffsync.exceptions import ObjectNotFound
 
 if TYPE_CHECKING:
-    from diffsync import DiffSyncModel
-    from diffsync import Adapter
+    from diffsync import Adapter, DiffSyncModel
 
 
 class BaseStore:

{diffsync-2.1.0 → diffsync-2.2.1}/diffsync/store/local.py

@@ -1,12 +1,11 @@
 """LocalStore module."""
 
 from collections import defaultdict
-from typing import List, Type, Union, TYPE_CHECKING, Dict, Set, Any
+from typing import TYPE_CHECKING, Any, Dict, List, Set, Type, Union
 
-from diffsync.exceptions import ObjectNotFound, ObjectAlreadyExists
+from diffsync.exceptions import ObjectAlreadyExists, ObjectNotFound
 from diffsync.store import BaseStore
 
-
 if TYPE_CHECKING:
     from diffsync import DiffSyncModel
 

{diffsync-2.1.0 → diffsync-2.2.1}/diffsync/store/redis.py

@@ -65,7 +65,7 @@ class RedisStore(BaseStore):
         """Get the object from Redis key."""
         pickled_object = self._store.get(key)
         if pickled_object:
-            obj_result = loads(pickled_object)  # nosec
+            obj_result = loads(pickled_object)  # noqa: S301
             obj_result.adapter = self.adapter
             return obj_result
         raise ObjectNotFound(f"{key} not present in Cache")
@@ -168,7 +168,7 @@ class RedisStore(BaseStore):
 
         existing_obj_binary = self._store.get(object_key)
         if existing_obj_binary:
-            existing_obj = loads(existing_obj_binary)  # nosec
+            existing_obj = loads(existing_obj_binary)  # noqa: S301
             existing_obj_dict = existing_obj.dict()
 
             if existing_obj_dict != obj.dict():

{diffsync-2.1.0 → diffsync-2.2.1}/diffsync/utils.py

@@ -16,7 +16,7 @@ limitations under the License.
 """
 
 from collections import OrderedDict
-from typing import Iterator, List, Dict, Optional, TypeVar, Callable, Generic
+from typing import Callable, Dict, Generic, Iterator, List, Optional, TypeVar
 
 SPACE = "    "
 BRANCH = "│   "
@@ -44,7 +44,7 @@ def symmetric_difference(lst1: List[T], lst2: List[T]) -> List[T]:
 class OrderedDefaultDict(OrderedDict, Generic[K, V]):
     """A combination of collections.OrderedDict and collections.DefaultDict behavior."""
 
-    def __init__(self, dict_type: Callable[[], V]) -> None:
+    def __init__(self, dict_type: Callable[[], V] = dict) -> None:  # type: ignore[assignment]
         """Create a new OrderedDefaultDict."""
         self.factory = dict_type
         super().__init__(self)

diffsync-2.2.1/pyproject.toml

@@ -0,0 +1,235 @@
+[tool.poetry]
+name = "diffsync"
+version = "2.2.1"
+description = "Library to easily sync/diff/update 2 different data sources"
+authors = ["Network to Code, LLC <info@networktocode.com>"]
+license = "Apache-2.0"
+readme = "README.md"
+homepage = "https://diffsync.readthedocs.io"
+repository = "https://github.com/networktocode/diffsync"
+documentation = "https://diffsync.readthedocs.io"
+keywords = ["source-of-truth", "synchronization"]
+classifiers = [
+    "Intended Audience :: Developers",
+    "Development Status :: 5 - Production/Stable",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+]
+include = [
+    "CHANGELOG.md",
+    "LICENSE",
+    "README.md",
+]
+
+[tool.poetry.dependencies]
+python = ">=3.10,<3.14"
+pydantic = "^2.0.0"
+structlog = ">= 20.1.0"
+packaging = ">= 21.3"
+colorama = {version = "^0.4.3", optional = true}
+redis = {version = "^4.3", optional = true}
+# typing.Self introduced in 3.11
+typing-extensions = { version = ">=4.0.1", python = "<3.11" }
+
+[tool.poetry.extras]
+redis = ["redis"]
+
+[tool.poetry.group.dev.dependencies]
+coverage = "*"
+pytest = "*"
+mock = "*"
+pyyaml = ">= 6.0.1"
+pylint = "*"
+yamllint = "^1.35.1"
+invoke = "^2.2.0"
+toml = "^0.10.2"
+attrs = "^23.2.0"
+towncrier = ">=23.6.0,<=24.8.0"
+ruff = "*"
+Markdown = "*"
+
+[tool.poetry.group.docs.dependencies]
+# Rendering docs to HTML
+mkdocs = "1.6.1"
+# Embedding YAML files into Markdown documents as tables
+markdown-data-tables = "1.0.0"
+# Render custom markdown for version added/changed/remove notes
+markdown-version-annotations = "1.0.1"
+# Automatically generate some files as part of mkdocs build
+mkdocs-gen-files = "0.5.0"
+# Image lightboxing in mkdocs
+mkdocs-glightbox = "0.4.0"
+# Use Jinja2 templating in docs - see settings.md
+mkdocs-macros-plugin = "1.3.7"
+# Material for mkdocs theme
+mkdocs-material = "9.6.15"
+# Handle docs redirections
+mkdocs-redirects = "1.2.2"
+# Automatically handle index pages for docs sections
+mkdocs-section-index = "0.3.10"
+# Automatic documentation from sources, for MkDocs
+mkdocstrings = "0.27.0"
+# Python-specific extension to mkdocstrings
+mkdocstrings-python = "1.13.0"
+griffe = "1.1.1"
+mypy = "*"
+pytest-cov = "*"
+pytest-structlog = "*"
+coverage = {extras = ["toml"], version = "*"}
+Sphinx = "^6.0.0"
+m2r2 = "*"
+sphinx-rtd-theme = "*"
+toml = "*"
+types-toml = "*"
+types-redis = "*"
+pytest-redis = "^2.4.0"
+types-requests = "^2.28.11.15"
+types-python-slugify = "^8.0.0.1"
+
+[tool.ruff]
+line-length = 120
+target-version = "py310"
+
+[tool.ruff.lint]
+select = [
+    "D",  # pydocstyle
+    "F", "E", "W",  # flake8
+    "S",  # bandit
+    "I",  # isort
+]
+ignore = [
+    # warning: `one-blank-line-before-class` (D203) and `no-blank-line-before-class` (D211) are incompatible.
+    "D203", # 1 blank line required before class docstring
+
+    # D212 is enabled by default in google convention, and complains if we have a docstring like:
+    # """
+    # My docstring is on the line after the opening quotes instead of on the same line as them.
+    # """
+    # We've discussed and concluded that we consider this to be a valid style choice.
+    "D212", # Multi-line docstring summary should start at the first line
+    "D213", # Multi-line docstring summary should start at the second line
+
+    # Produces a lot of issues in the current codebase.
+    "D401", # First line of docstring should be in imperative mood
+    "D407", # Missing dashed underline after section
+    "D416", # Section name ends in colon
+    "E501", # Line too long
+]
+
+[tool.ruff.lint.pydocstyle]
+convention = "google"
+
+[tool.ruff.lint.per-file-ignores]
+"tests/*" = [
+    "D",
+    "S"
+]
+
+[tool.mypy]
+warn_unused_configs = true
+disallow_untyped_defs = true
+ignore_missing_imports = true
+
+[tool.pylint.general]
+extension-pkg-whitelist = [
+    "pydantic"
+]
+
+[tool.pylint.basic]
+# No docstrings required for private methods (Pylint default), or for test_ functions, or for inner Meta classes.
+no-docstring-rgx = "^(_|test_|Meta$)"
+
+[tool.pylint.master]
+ignore=[".venv", "tests"]
+
+[tool.pylint.messages_control]
+# Line length is enforced by Black, so pylint doesn't need to check it.
+# Pylint and Black disagree about how to format multi-line arrays; Black wins.
+disable = [
+    "line-too-long",
+    "duplicate-code",
+    "cyclic-import",
+    ]
+
+[tool.pylint.miscellaneous]
+# Don't flag TODO as a failure, let us commit with things that still need to be done in the code
+notes = """,
+    FIXME,
+    XXX,
+    """
+
+[tool.pylint.similarities]
+# There's a lot of duplicate code in the examples/backend_*.py files - don't complain about it for now
+min-similarity-lines = 20
+
+[tool.pytest.ini_options]
+python_paths = "./"
+testpaths = [
+    "tests/"
+]
+addopts = "-vv --doctest-modules -p no:warnings --ignore-glob='*mock*'"
+
+[tool.towncrier]
+package = "diffsync"
+directory = "changes"
+filename = "docs/admin/release_notes/version_X.Y.md"
+template = "towncrier_template.j2"
+start_string = "<!-- towncrier release notes start -->"
+issue_format = "[#{issue}](https://github.com/networktocode/diffsync/issues/{issue})"
+
+[[tool.towncrier.type]]
+directory = "breaking"
+name = "Breaking Changes"
+showcontent = true
+
+[[tool.towncrier.type]]
+directory = "security"
+name = "Security"
+showcontent = true
+
+[[tool.towncrier.type]]
+directory = "added"
+name = "Added"
+showcontent = true
+
+[[tool.towncrier.type]]
+directory = "changed"
+name = "Changed"
+showcontent = true
+
+[[tool.towncrier.type]]
+directory = "deprecated"
+name = "Deprecated"
+showcontent = true
+
+[[tool.towncrier.type]]
+directory = "removed"
+name = "Removed"
+showcontent = true
+
+[[tool.towncrier.type]]
+directory = "fixed"
+name = "Fixed"
+showcontent = true
+
+[[tool.towncrier.type]]
+directory = "dependencies"
+name = "Dependencies"
+showcontent = true
+
+[[tool.towncrier.type]]
+directory = "documentation"
+name = "Documentation"
+showcontent = true
+
+[[tool.towncrier.type]]
+directory = "housekeeping"
+name = "Housekeeping"
+showcontent = true
+
+[build-system]
+requires = ["poetry-core>=2.0.0,<3.0.0"]
+build-backend = "poetry.core.masonry.api"

diffsync-2.1.0/CHANGELOG.md

@@ -1,212 +0,0 @@
-# Changelog
-
-All notable changes to this project will be documented in this file.
-
-The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
-
-## [2.1.0]
-
-### Changed
-
-- [284](https://github.com/networktocode/diffsync/pull/284) - Update pyyaml to 6.0.2 in poetry.lock by @gsnider2195
-- [286](https://github.com/networktocode/diffsync/pull/286) - Update GitHub Actions runner version by @mjbear
-- [289](https://github.com/networktocode/diffsync/pull/289) - Remove upper limit on Packaging by @jdrew82
-
-## New Contributors
-
-- @gsnider2195 made their first contribution in https://github.com/networktocode/diffsync/pull/284
-- @mjbear made their first contribution in https://github.com/networktocode/diffsync/pull/286
-- @jdrew82 made their first contribution in https://github.com/networktocode/diffsync/pull/289
-
-## [2.0.1]
-
-### Changed
-
-- #276 - Removed upper version bound for `structlog` dependency
-
-### Fixed
-
-- #281 - Properly deprecated `DiffSync` class name
-- #273 - Properly capitalized `DiffSync` in documentation
-- #273 - Removed more mentions of `DiffSync` in favor of `Adapter`
-- #274 - Fixed doc section title for getting started
-- #269 - Fixed wording for a couple of docstrings
-- #265 - Fixed readthedocs build
-
-## [2.0.0]
-
-### Changed
-
-- **BREAKING CHANGE** #236/240 - Upgrade to Pydantic v2. 
-
-## [1.10.0] - 2023-11-16
-
-### Fixed
-
-- #249 - Fixes natural deletion order flag
-- #247 - Fixes underspecified typing_extensions dependency
-
-### Changed
-
-- #247 - Deprecates Python 3.7
-
-## [1.9.0] - 2023-10-16
-
-### Added
-
-- #220 - Implement DiffSyncModelFlags.NATURAL_DELETION_ORDER.
-
-### Changed
-
-- #219 - Type hinting overhaul 
-
-## [1.8.0] - 2023-04-18
-
-### Added
-
-- #182 - Added `get_or_add_model_instance()` and `update_or_add_model_instance()` APIs.
-- #189 - Added note in `README.md` about running `invoke tests`.
-- #190 - Added note in `README.md` about running `invoke build`.
-
-### Changed
-
-- #77/#188 - `sync_from()` and `sync_to()` now return the `Diff` that was applied.
-- #211 - Loosened `packaging` and `structlog` library dependency constraints for broader compatibility.
-
-## [1.7.0] - 2022-11-03
-
-### Changed
-
-- #176 - Remove pytest-redislite in favor of pytest-redis.
-- #174 - Update Dockerfile to install build-essential
-
-### Added
-
-- #174 - Add methods to load data from dictionary and enable tree traversal
-- #174 - Add a `get_or_none` method to the DiffSync class
-- #168 - Add 'skip' counter to diff.summary()
-- #169/#170 - Add documentation about model processing order
-- #121/#140 - Add and configure renovate
-- #140 - Add renovate configuration validation to the CI
-
-### Fixed
-
-- #149 - Limit redundant CI concurrency
-
-## [1.6.0] - 2022-07-09
-
-### Changed
-
-- #120 - Dropped support for Python 3.6, new minimum is Python 3.7
-
-## [1.5.1] - 2022-06-30
-
-### Added
-
-- #111 - Added example 6, regarding IP prefixes.
-
-### Changed
-
-- #107 - Updated example 5 to use the Redis backend store.
-
-### Fixed
-
-- #115 - Fixed ReadTheDocs rendering pipeline
-- #118 - Fixed a regression in `DiffSync.get(modelname, identifiers)` introduced in 1.5.0
-
-## [1.5.0] - 2022-06-07
-
-### Added
-
-- #106 - Add a new, optional, backend store based in Redis
-
-## [1.4.3] - 2022-03-03
-
-### Fixed
-
-- #101 - Revert changed introduced in #90 that affected `DiffElement.action`
-
-**NOTE**: this change is a breaking change against DiffSync 1.4.0 through 1.4.2, but was necessary to restore backward compatibility with DiffSync 1.3.x and earlier. Apologies for any inconvenience this causes.
-
-### Changed
-
-- #103 - Update development dependencies
-
-## [1.4.2] - 2022-02-28
-
-**WARNING** - #90 inadvertently introduced a breaking API change in DiffSync 1.4.0 through 1.4.2 (#101); this change was reverted in #102 for DiffSync 1.4.3 and later. We recommend not using this release, and moving to 1.4.3 instead.
-
-### Fixed
-
-- #100 - Added explicit dependency on `packaging`.
-
-## [1.4.1] - 2022-01-26
-
-**WARNING** - #90 inadvertently introduced a breaking API change in DiffSync 1.4.0 through 1.4.2 (#101); this change was reverted in #102 for DiffSync 1.4.3 and later. We recommend not using this release, and moving to 1.4.3 instead.
-
-### Fixed
-
-- #95 - Removed optional dependencies on `sphinx`, `m2r2`, `sphinx-rtd-theme`, `toml`.
-
-## [1.4.0] - 2022-01-24
-
-**WARNING** - #90 inadvertently introduced a breaking API change in DiffSync 1.4.0 through 1.4.2 (#101); this change was reverted in #102 for DiffSync 1.4.3 and later. We recommend not using this release, and moving to 1.4.3 instead.
-
-### Added
-
-- #53 - Add a new example based on pynautobot and Nautobot REST API
-- #59 - Add proper documentation published in Read the doc
-- #68 - Cleanup Readme, add link to new documentation site
-- #70 - Add `add_or_update()` method to DiffSync class that requires a DiffSyncModel to be passed in and will attempt to add or update an existing object
-- #72 - Add core engine section in docs and rename example directories
-- #75 - Add support for Structlog v21 in addition to v20.
-- #80 - Add support for an existing Diff object to be passed to `sync_to()` & `sync_from()` to prevent another diff from being calculated.
-- #81 - Add a new example based on PeeringDB
-- #83 - Add support for Python 3.10
-- #87 - Add new model flags : `SKIP_UNMATCHED_BOTH`, `SKIP_UNMATCHED_SRC` & `SKIP_UNMATCHED_DST` to match the behavior of the global flags
-
-### Changed
-
-- #62 - Update CI Token
-- #69 - Replace Travis CI with Github Actions to run unit tests
-- #82 - Update lock file with latest versions.
-- #90 - Convert list of actions (`create`, `update`, `delete`) to proper Enum
-
-### Fixed
-
-- #51 - Update minimum Pydantic version due to security advisory GHSA-5jqp-qgf6-3pvh
-- #63 - Fix type in Readme
-
-## [1.3.0] - 2021-04-07
-
-### Added
-
-- #48 - added optional `callback` argument to `diff_from`/`diff_to`/`sync_from`/`sync_to` for use with progress reporting.
-
-## [1.2.0] - 2020-12-08
-
-### Added
-
-- #45 - minimum Python version lowered from 3.7 to 3.6, also now tested against Python 3.9.
-
-## [1.1.0] - 2020-12-01
-
-### Added
-
-- #37 - added `sync_complete` callback, triggered on `sync_from` completion with changes.
-- #41 - added `summary` API for Diff and DiffElement objects.
-- #44 - added `set_status()` and `get_status()` APIs so that DiffSyncModel implementations can provide details for create/update/delete logging
-
-### Changed
-
-- Now requires Pydantic 1.7.2 or later
-- #34 - in diff dicts, changed keys `src`/`dst`/`_src`/`_dst` to `-` and `+`
-- #43 - `DiffSync.get_by_uids()` now raises `ObjectNotFound` if any of the provided uids cannot be located; `DiffSync.get()` raises `ObjectNotFound` or `ValueError` on failure, instead of returning `None`.
-
-### Fixed
-
-- #44 - On CRUD failure, do not generate an extraneous "success" log message in addition to the "failed" message
-
-## [1.0.0] - 2020-10-23
-
-Initial release

diffsync-2.1.0/pyproject.toml

@@ -1,112 +0,0 @@
-[tool.poetry]
-name = "diffsync"
-version = "v2.1.0"
-description = "Library to easily sync/diff/update 2 different data sources"
-authors = ["Network to Code, LLC <info@networktocode.com>"]
-license = "Apache-2.0"
-readme = "README.md"
-homepage = "https://diffsync.readthedocs.io"
-repository = "https://github.com/networktocode/diffsync"
-documentation = "https://diffsync.readthedocs.io"
-keywords = ["source-of-truth", "synchronization"]
-include = [
-    "CHANGELOG.md",
-    "LICENSE",
-    "README.md",
-]
-
-[tool.poetry.dependencies]
-python = ">=3.9,<4.0"
-pydantic = "^2.0.0"
-structlog = ">= 20.1.0"
-packaging = ">= 21.3"
-colorama = {version = "^0.4.3", optional = true}
-redis = {version = "^4.3", optional = true}
-# typing.Self introduced in 3.11
-typing-extensions = { version = ">=4.0.1", python = "<3.11" }
-
-[tool.poetry.extras]
-redis = ["redis"]
-
-[tool.poetry.group.dev.dependencies]
-pytest = "*"
-pyyaml = ">= 6.0.1"
-black = "*"
-pylint = "*"
-pydocstyle = "*"
-yamllint = "*"
-bandit = "*"
-invoke = "*"
-flake8 = "*"
-mypy = "*"
-pytest-cov = "*"
-pytest-structlog = "*"
-coverage = {extras = ["toml"], version = "*"}
-Sphinx = "^6.0.0"
-m2r2 = "*"
-sphinx-rtd-theme = "*"
-toml = "*"
-types-toml = "*"
-types-redis = "*"
-pytest-redis = "^2.4.0"
-types-requests = "^2.28.11.15"
-types-python-slugify = "^8.0.0.1"
-
-[tool.black]
-line-length = 120
-include = '\.pyi?$'
-exclude = '''
-        /(
-          \.git
-        | \.tox
-        | \.venv
-        | env/
-        | _build
-        | build
-        | dist
-        )/
-    '''
-
-[tool.coverage.run]
-branch = true
-
-[tool.pylint.general]
-extension-pkg-whitelist = [
-    "pydantic"
-]
-
-[tool.pylint.basic]
-# No docstrings required for private methods (Pylint default), or for test_ functions.
-no-docstring-rgx="^(_|test_)"
-
-[tool.pylint.messages_control]
-# Line length is enforced by Black, so pylint doesn't need to check it.
-# Pylint and Black disagree about how to format multi-line arrays; Black wins.
-disable = """,
-    line-too-long,
-    """
-
-[tool.pylint.miscellaneous]
-# Don't flag TODO as a failure, let us commit with things that still need to be done in the code
-notes = """,
-    FIXME,
-    XXX,
-    """
-
-[tool.pylint.similarities]
-# There's a lot of duplicate code in the examples/backend_*.py files - don't complain about it for now
-min-similarity-lines = 20
-
-[tool.pytest.ini_options]
-testpaths = [
-    "tests"
-]
-
-[tool.mypy]
-warn_unused_configs = true
-disallow_untyped_defs = true
-ignore_missing_imports = true
-
-[build-system]
-requires = ["poetry_core>=1.0.8"]
-build-backend = "poetry.core.masonry.api"