pulpcore 3.22.2__tar.gz → 3.23.0__tar.gz

{pulpcore-3.22.2 → pulpcore-3.23.0}/CHANGES.rst

@@ -17,6 +17,159 @@ Changelog
 
 .. towncrier release notes start
 
+3.23.0 (2023-03-14)
+===================
+REST API
+--------
+
+Features
+~~~~~~~~
+
+- Added a repository exact and in filters to the Distributions API.
+  :github:`3394`
+- Added a new optional multi-tenancy namespacing feature: Domains. See workflow documentation for more
+  information on this tech-preview feature.
+  :github:`3403`
+- Add GCP support
+  :github:`3424`
+- Added ``retain_repo_versions`` filter to Repository endpoint.
+  :github:`3507`
+- Pass correlation id to signing script through ENV variable
+  :github:`3522`
+- Added ability to replicate distributions/repositories from another Pulp.
+  :github:`3566`
+- Started to collect statistics about RBAC (amd domain) usage.
+  :github:`3639`
+
+
+Bugfixes
+~~~~~~~~
+
+- Bump dynaconf version to 3.1.12 given it fixes a bug to handle failures when pwd does not exist.
+  :github:`3310`
+- Added validation to the repository modify endpoint.
+  :github:`3326`
+- Addressed a possible N+1 query performance issue within reclaim_space task.
+  :github:`3404`
+- Added repeating logic to signalling a task worker subprocess. This should fix a bug where the
+  task refuses to be terminated easily.
+  :github:`3407`
+- Added a ``start_repository_version`` parameter to the file system exporter.
+
+  If specified, it will export only content units that differed between two repository versions.
+  :github:`3413`
+- Fixed the label migration code for not null constraint errors when objects with and without labels
+  are migrated.
+  :github:`3495`
+- Added int64 format to integer fields in api schema.
+  :github:`3590`
+- Fixed a bug that caused Pulp to return duplicate content when a user was logged in as a
+  non-superuser while the content was part of multiple repositories.
+  :github:`3641`
+- Added missing validation to the ``repair`` endpoint.
+  :github:`3645`
+- Fixed bug related to pulpcore-content serving content from a remote (pull-through-cache).
+  :github:`3654`
+
+
+Improved Documentation
+~~~~~~~~~~~~~~~~~~~~~~
+
+- Revisited features in tech preview (e.g., ACS, RBAC, metadata signing, import/export) and marked
+  them production ready.
+  :github:`3429`
+- Documented PostgreSQL version compatibility and incompatibility with transaction based connection
+  pooling.
+  :github:`3505`
+- Updated the release-process doc to be more complete.
+  :github:`3514`
+- Added links to the Containerfiles used for building the pulp and pulp-minimal OCI images.
+  :github:`3627`
+
+
+Removals
+~~~~~~~~
+
+- Removed the PROFILE_STAGES_API setting along with the dependent code.
+  :github:`3595`
+
+
+Misc
+~~~~
+
+- :github:`3446`, :github:`3541`, :github:`3569`, :github:`3574`, :github:`3584`, :github:`3638`
+
+
+Plugin API
+----------
+
+Features
+~~~~~~~~
+
+- Refactored ``ArtifactDownloader`` stage logic into base class ``GenericDownloader`` and allow for
+  the progress reports message and code to be customized by subclasses.
+  :github:`1931`
+- Added version information to the worker records in the database.
+  :github:`3365`
+- Added new config option ``domain_compatible`` to ``PluginAppConfig`` to specify Domains feature
+  compatibility within a plugin. See Domains compatibility documentation in plugin-writer section for
+  more information.
+  :github:`3403`
+- Exported ``TaskSerializer`` in ``pulpcore.plugin.serializers``
+  :github:`3506`
+- Added Replicator class that plugins can implement to enable distribution/repository replication.
+  :github:`3566`
+- Exposed a function for extracting PKs from URIs and a function for validating content units
+  (``pulpcore.plugin.util.extract_pk``, ``pulpcore.plugin.util.raise_for_unknown_content_units``).
+  :github:`3604`
+- Add ``label_field_name`` arg to ``LabelFilter()`` filter that allows plugin devs to
+  customize the model field that ``LabelFilter`` filters on.
+  :github:`3631`
+
+
+Bugfixes
+~~~~~~~~
+
+- Added HiddenFieldsMixin to the plugin API.
+  :github:`3520`
+- Exported ``LabelFilter`` to the plugin api as a replacement for the deprecated
+  ``LabelSelectFilter``.
+  :github:`3570`
+
+
+Improved Documentation
+~~~~~~~~~~~~~~~~~~~~~~
+
+- Adds docs that plugin writers tasks must be backwards compatible until the next major Pulp version.
+  :github:`3368`
+- Adds docs on the need and possible options for writing migrations in a way that they could be
+  applied to a running Pulp system.
+  :github:`3443`
+
+
+Deprecations
+~~~~~~~~~~~~
+
+- Deprecated ``NamedModelViewSet.extract_pk`` in favour of ``pulpcore.plugin.util.extract_pk``.
+  Deprecated ``pulpcore.plugin.actions.raise_for_unknown_content_units`` in favour of
+  ``pulpcore.plugin.util.raise_for_unknown_content_units``.
+  :github:`3604`
+
+
+3.22.3 (2023-03-06)
+===================
+REST API
+--------
+
+No significant changes.
+
+
+Plugin API
+----------
+
+No significant changes.
+
+
 3.22.2 (2023-02-02)
 ===================
 REST API
@@ -211,6 +364,46 @@ Removals
   :github:`3400`
 
 
+3.21.5 (2023-02-02)
+===================
+REST API
+--------
+
+Bugfixes
+~~~~~~~~
+
+- Added a ``start_repository_version`` parameter to the file system exporter.
+
+  If specified, it will export only content units that differed between two repository versions.
+  :github:`3413`
+
+
+Plugin API
+----------
+
+No significant changes.
+
+
+3.21.4 (2023-01-20)
+===================
+REST API
+--------
+
+Bugfixes
+~~~~~~~~
+
+- Fixed an occasional 500 error when viewing a repository version content summary while performing a content deletion task.
+  :github:`3299`
+- Fix migrating Remotes with @ in path
+  :github:`3409`
+
+
+Plugin API
+----------
+
+No significant changes.
+
+
 3.21.3 (2022-11-15)
 ===================
 REST API
@@ -820,6 +1013,82 @@ Bugfixes
   :github:`2069`
 
 
+3.18.15 (2023-03-09)
+====================
+REST API
+--------
+
+Bugfixes
+~~~~~~~~
+
+- Added a ``start_repository_version`` parameter to the file system exporter.
+
+  If specified, it will export only content units that differed between two repository versions.
+  :github:`3413`
+
+
+Plugin API
+----------
+
+No significant changes.
+
+
+3.18.14 (2023-02-23)
+====================
+REST API
+--------
+
+Bugfixes
+~~~~~~~~
+
+- Fixed a bug that caused the import machinery to import the same content multiple times in a row.
+  :github:`3075`
+
+
+Plugin API
+----------
+
+No significant changes.
+
+
+3.18.13 (2023-01-20)
+====================
+REST API
+--------
+
+Bugfixes
+~~~~~~~~
+
+- Fixed an occasional 500 error when viewing a repository version content summary while performing a content deletion task.
+  :github:`3299`
+
+
+Plugin API
+----------
+
+No significant changes.
+
+
+3.18.12 (2023-01-10)
+====================
+REST API
+--------
+
+Bugfixes
+~~~~~~~~
+
+- Insured that pulp-export correctly locks repos-being-exported.
+  :github:`3370`
+- Fix migrating Remotes with @ in path
+  :github:`3409`
+
+
+Plugin API
+----------
+
+No significant changes.
+
+
 3.18.11 (2022-11-02)
 ====================
 REST API
@@ -1601,6 +1870,26 @@ Bugfixes
   :redmine:`9608`
 
 
+3.16.16 (2023-02-23)
+====================
+REST API
+--------
+
+Bugfixes
+~~~~~~~~
+
+- Fixed a bug that caused the import machinery to import the same content multiple times in a row.
+  :github:`3075`
+- Insured that pulp-export correctly locks repos-being-exported.
+  :github:`3370`
+
+
+Plugin API
+----------
+
+No significant changes.
+
+
 3.16.15 (2022-11-02)
 ====================
 REST API
@@ -2571,6 +2860,26 @@ Misc
 - :redmine:`8606`, :redmine:`9160`
 
 
+3.14.19 (2023-02-09)
+====================
+REST API
+--------
+
+Bugfixes
+~~~~~~~~
+
+- Fixed another rare deadlock for high-concurrency/overlapping-content syncs.
+  :github:`3111`
+- Another step on closing the deadlock-window when syncing overlapping content.
+  :github:`3284`
+
+
+Plugin API
+----------
+
+No significant changes.
+
+
 3.14.18 (2022-07-14)
 ====================
 REST API

{pulpcore-3.22.2 → pulpcore-3.23.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: pulpcore
-Version: 3.22.2
+Version: 3.23.0
 Summary: Pulp Django Application and Related Modules
 Home-page: https://pulpproject.org
 Author: Pulp Team
@@ -46,5 +46,6 @@ Requires-Python: >=3.8
 Description-Content-Type: text/markdown
 Provides-Extra: sftp
 Provides-Extra: s3
+Provides-Extra: google
 Provides-Extra: azure
 Provides-Extra: prometheus

{pulpcore-3.22.2 → pulpcore-3.23.0}/functest_requirements.txt

@@ -7,3 +7,5 @@ pulpcore-client
 pulp-file-client
 pytest
 python-gnupg
+proxy.py~=2.4.3
+trustme~=0.9.0

{pulpcore-3.22.2 → pulpcore-3.23.0}/pulpcore/app/apps.py

@@ -4,10 +4,10 @@ from collections import defaultdict
 from gettext import gettext as _
 from importlib import import_module
 
-from django.conf import settings
 from django import apps
+from django.conf import settings
 from django.core.exceptions import ImproperlyConfigured
-from django.db import transaction
+from django.db import connection, transaction
 from django.db.models.expressions import OuterRef, RawSQL
 from django.db.models.signals import post_migrate
 from django.utils.module_loading import module_has_submodule
@@ -19,6 +19,7 @@ VIEWSETS_MODULE_NAME = "viewsets"
 SERIALIZERS_MODULE_NAME = "serializers"
 URLS_MODULE_NAME = "urls"
 MODELRESOURCE_MODULE_NAME = "modelresource"
+REPLICA_MODULE_NAME = "replica"
 
 
 def pulp_plugin_configs():
@@ -86,7 +87,13 @@ class PulpPluginAppConfig(apps.AppConfig):
                 "attribute."
             )
             raise ImproperlyConfigured(msg.format(self.label))
-
+        if settings.DOMAIN_ENABLED:
+            if not getattr(self, "domain_compatible", None):
+                msg = (
+                    "The plugin `{}-{}` is not domain compatible. Please uninstall/update the "
+                    "plugin or turn off `DOMAIN_ENABLED` for Pulp to start."
+                )
+                raise ImproperlyConfigured(msg.format(self.python_package_name, self.version))
         # Module containing viewsets eg. <module 'pulp_plugin.app.viewsets'
         # from 'pulp_plugin/app/viewsets.py'>. Set by import_viewsets().
         # None if the application doesn't have a viewsets module, automatically set
@@ -101,6 +108,11 @@ class PulpPluginAppConfig(apps.AppConfig):
         # List of classes from self.modelresource_module that can be exported
         self.exportable_classes = None
 
+        # Module containing the replicator class for a plugin
+        self.replicator_module = None
+        # List of classes from the self.replicator_module
+        self.replicator_classes = None
+
         # Mapping of model names to viewset lists (viewsets unrelated to models are excluded)
         self.named_viewsets = None
         # Mapping of serializer names to serializers
@@ -111,6 +123,7 @@ class PulpPluginAppConfig(apps.AppConfig):
         self.import_serializers()
         self.import_urls()
         self.import_modelresources()
+        self.import_replicators()
         post_migrate.connect(
             _populate_access_policies,
             sender=self,
@@ -193,6 +206,19 @@ class PulpPluginAppConfig(apps.AppConfig):
             self.modelresource_module = import_module(modelrsrc_module_name)
             self.exportable_classes = self.modelresource_module.IMPORT_ORDER
 
+    def import_replicators(self):
+        """
+        If a plugin has a replicator.py, import it.
+
+        This exists when a plugin supports replication from an upstream Pulp.
+        """
+        if module_has_submodule(self.module, REPLICA_MODULE_NAME) and self.name != "pulpcore.app":
+            replica_module_name = "{name}.{module}".format(
+                name=self.name, module=REPLICA_MODULE_NAME
+            )
+            self.replicator_module = import_module(replica_module_name)
+            self.replicator_classes = self.replicator_module.REPLICATION_ORDER
+
 
 class PulpAppConfig(PulpPluginAppConfig):
     # The pulpcore app is itself a pulpcore plugin so that it can benefit from
@@ -207,15 +233,20 @@ class PulpAppConfig(PulpPluginAppConfig):
     label = "core"
 
     # The version of this app
-    version = "3.22.2"
+    version = "3.23.0"
 
     # The python package name providing this app
     python_package_name = "pulpcore"
 
+    domain_compatible = True
+
     def ready(self):
         super().ready()
         from . import checks  # noqa
 
+        post_migrate.connect(
+            _ensure_default_domain, sender=self, dispatch_uid="ensure_default_domain"
+        )
         post_migrate.connect(
             _populate_system_id, sender=self, dispatch_uid="populate_system_id_identifier"
         )
@@ -279,6 +310,24 @@ def _populate_system_id(sender, apps, verbosity, **kwargs):
         SystemID().save()
 
 
+def _ensure_default_domain(sender, **kwargs):
+    table_names = connection.introspection.table_names()
+    if "core_domain" in table_names:
+        from pulpcore.app.util import get_default_domain
+
+        default = get_default_domain()  # Cache the default domain
+        # Match the Pulp settings
+        if (
+            settings.HIDE_GUARDED_DISTRIBUTIONS != default.hide_guarded_distributions
+            or settings.REDIRECT_TO_OBJECT_STORAGE != default.redirect_to_object_storage
+            or settings.DEFAULT_FILE_STORAGE != default.storage_class
+        ):
+            default.hide_guarded_distributions = settings.HIDE_GUARDED_DISTRIBUTIONS
+            default.redirect_to_object_storage = settings.REDIRECT_TO_OBJECT_STORAGE
+            default.storage_class = settings.DEFAULT_FILE_STORAGE
+            default.save(skip_hooks=True)
+
+
 def _populate_roles(sender, apps, verbosity, **kwargs):
     role_prefix = f"{sender.label}."
     # collect all plugin defined roles