django-postpone-index 0.0.1__tar.gz → 0.0.3__tar.gz

{django_postpone_index-0.0.1/django_postpone_index.egg-info → django_postpone_index-0.0.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: django-postpone-index
-Version: 0.0.1
+Version: 0.0.3
 Summary: Postpone index creation to provide Zero Downtime Migration feature
 Home-page: https://github.com/nnseva/django-postpone-index
 Author: Vsevolod Novikov
@@ -169,6 +169,43 @@ calling the `apply_postponed` migration command again. All not-applied indexes w
 **NOTICE** the `apply_postponed` management command doesn't have any explicit locking mechanics. Avoid starting this
 command concurrently with itself or another `migrate` command on the same database.
 
+## Intermediate migration state
+
+Apart from standard Django migrations, using the `postpone_index` package leads to the *intermediate migration state*
+after the `migrate` management command finished:
+
+- new model structure is applied
+- indexes to be deleted are deleted
+- indexes to be created are *not created yet*
+
+You should be aware that if you introduce a new unique index or constraint, the database does not control uniqueness
+based on not yet created indexes at this time.
+
+Your code works now as expected everywhere, except the code which is based on new unique constraints introduced in applied migrations.
+
+Apply the `apply_postponed run` management command to make these new indexes work.
+
+Any error while `apply_postponed run` execution is stored in the `PostponedSQL` model instance.
+
+You can see erroneous lines using `apply_postponed list` command. See the `[E]` mark at the start of the line.
+
+You also can see the error details using the format parameter of the `apply_postponed list -f '... %(error)s'` management command.
+
+The `apply_postponed run -x` breaks execution on any error. You can see the error in the standard error or logging streams.
+
+The `apply_postponed run` (without `-x` parameter) doesn't stop on error, but outputs warning to the log stream instead.
+
+When the error happened, it most probably is caused by the non-unique records. Fix the data and try to execute
+`apply_postponed run` again to create an index.
+
+After the successfull `apply_postponed run` execution, the migration state is finalised to be equal as if you applied the migration
+without `postpone_index` package at all.
+
+The `apply_postponed run` management command marks all successfully executed `PostponedSQL` instances as `done`. You can see `[X]` mark
+at the start of the line produced by `apply_ponsponed list` management command.
+
+You can cleanup `done` instances using `apply_postponed cleanup` management command. This step is optional.
+
 ## Django testing
 
 Django migrates testing database before tests. Always use `POSTPONE_INDEX_IGNORE = True` settings to avoid postpone index

{django_postpone_index-0.0.1 → django_postpone_index-0.0.3}/README.md

@@ -120,6 +120,43 @@ calling the `apply_postponed` migration command again. All not-applied indexes w
 **NOTICE** the `apply_postponed` management command doesn't have any explicit locking mechanics. Avoid starting this
 command concurrently with itself or another `migrate` command on the same database.
 
+## Intermediate migration state
+
+Apart from standard Django migrations, using the `postpone_index` package leads to the *intermediate migration state*
+after the `migrate` management command finished:
+
+- new model structure is applied
+- indexes to be deleted are deleted
+- indexes to be created are *not created yet*
+
+You should be aware that if you introduce a new unique index or constraint, the database does not control uniqueness
+based on not yet created indexes at this time.
+
+Your code works now as expected everywhere, except the code which is based on new unique constraints introduced in applied migrations.
+
+Apply the `apply_postponed run` management command to make these new indexes work.
+
+Any error while `apply_postponed run` execution is stored in the `PostponedSQL` model instance.
+
+You can see erroneous lines using `apply_postponed list` command. See the `[E]` mark at the start of the line.
+
+You also can see the error details using the format parameter of the `apply_postponed list -f '... %(error)s'` management command.
+
+The `apply_postponed run -x` breaks execution on any error. You can see the error in the standard error or logging streams.
+
+The `apply_postponed run` (without `-x` parameter) doesn't stop on error, but outputs warning to the log stream instead.
+
+When the error happened, it most probably is caused by the non-unique records. Fix the data and try to execute
+`apply_postponed run` again to create an index.
+
+After the successfull `apply_postponed run` execution, the migration state is finalised to be equal as if you applied the migration
+without `postpone_index` package at all.
+
+The `apply_postponed run` management command marks all successfully executed `PostponedSQL` instances as `done`. You can see `[X]` mark
+at the start of the line produced by `apply_ponsponed list` management command.
+
+You can cleanup `done` instances using `apply_postponed cleanup` management command. This step is optional.
+
 ## Django testing
 
 Django migrates testing database before tests. Always use `POSTPONE_INDEX_IGNORE = True` settings to avoid postpone index

{django_postpone_index-0.0.1 → django_postpone_index-0.0.3}/dev/config/settings.py

@@ -124,11 +124,6 @@ LOGGING = {
         }
     },
     'loggers': {
-        'postpone_index.contrib.postgres.schema': {
-            'handlers': ['console'],
-            'level': os.environ.get('POSTPONE_INDEX_LOGGING', 'INFO'),
-            'propagate': False,
-        },
         'postpone_index': {
             'handlers': ['console'],
             'level': os.environ.get('POSTPONE_INDEX_LOGGING', 'INFO'),
@@ -139,7 +134,6 @@ LOGGING = {
             'level': os.environ.get('CONFIG_LOGGING', 'INFO'),
             'propagate': False,
         },
-        # Again, default Django configuration to email unhandled exceptions
         'django.db': {
             'handlers': ['console'],
             'level': os.environ.get('DATABASE_LOGGING', 'INFO'),

{django_postpone_index-0.0.1 → django_postpone_index-0.0.3/django_postpone_index.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: django-postpone-index
-Version: 0.0.1
+Version: 0.0.3
 Summary: Postpone index creation to provide Zero Downtime Migration feature
 Home-page: https://github.com/nnseva/django-postpone-index
 Author: Vsevolod Novikov
@@ -169,6 +169,43 @@ calling the `apply_postponed` migration command again. All not-applied indexes w
 **NOTICE** the `apply_postponed` management command doesn't have any explicit locking mechanics. Avoid starting this
 command concurrently with itself or another `migrate` command on the same database.
 
+## Intermediate migration state
+
+Apart from standard Django migrations, using the `postpone_index` package leads to the *intermediate migration state*
+after the `migrate` management command finished:
+
+- new model structure is applied
+- indexes to be deleted are deleted
+- indexes to be created are *not created yet*
+
+You should be aware that if you introduce a new unique index or constraint, the database does not control uniqueness
+based on not yet created indexes at this time.
+
+Your code works now as expected everywhere, except the code which is based on new unique constraints introduced in applied migrations.
+
+Apply the `apply_postponed run` management command to make these new indexes work.
+
+Any error while `apply_postponed run` execution is stored in the `PostponedSQL` model instance.
+
+You can see erroneous lines using `apply_postponed list` command. See the `[E]` mark at the start of the line.
+
+You also can see the error details using the format parameter of the `apply_postponed list -f '... %(error)s'` management command.
+
+The `apply_postponed run -x` breaks execution on any error. You can see the error in the standard error or logging streams.
+
+The `apply_postponed run` (without `-x` parameter) doesn't stop on error, but outputs warning to the log stream instead.
+
+When the error happened, it most probably is caused by the non-unique records. Fix the data and try to execute
+`apply_postponed run` again to create an index.
+
+After the successfull `apply_postponed run` execution, the migration state is finalised to be equal as if you applied the migration
+without `postpone_index` package at all.
+
+The `apply_postponed run` management command marks all successfully executed `PostponedSQL` instances as `done`. You can see `[X]` mark
+at the start of the line produced by `apply_ponsponed list` management command.
+
+You can cleanup `done` instances using `apply_postponed cleanup` management command. This step is optional.
+
 ## Django testing
 
 Django migrates testing database before tests. Always use `POSTPONE_INDEX_IGNORE = True` settings to avoid postpone index

django_postpone_index-0.0.3/django_postpone_index.egg-info/top_level.txt

@@ -0,0 +1,6 @@
+postpone_index
+postpone_index/contrib
+postpone_index/contrib/postgis
+postpone_index/contrib/postgres
+postpone_index/management
+postpone_index/management/commands

{django_postpone_index-0.0.1 → django_postpone_index-0.0.3}/postpone_index/_version.py

@@ -28,7 +28,7 @@ version_tuple: VERSION_TUPLE
 commit_id: COMMIT_ID
 __commit_id__: COMMIT_ID
 
-__version__ = version = '0.0.1'
-__version_tuple__ = version_tuple = (0, 0, 1)
+__version__ = version = '0.0.3'
+__version_tuple__ = version_tuple = (0, 0, 3)
 
-__commit_id__ = commit_id = 'g35054a7d6'
+__commit_id__ = commit_id = 'g98bba5fb9'

{django_postpone_index-0.0.1 → django_postpone_index-0.0.3}/postpone_index/contrib/postgres/schema.py

@@ -25,20 +25,10 @@ class DatabaseSchemaEditorMixin(Utils):
         """
         if self._base_tables_created:
             return
-
         # Avoid circular import
         from postpone_index.models import PostponedSQL
 
-        cursor = self.connection.cursor()
-        cursor.execute("SELECT 1 FROM information_schema.tables WHERE table_schema = 'public' AND table_name = '%s' limit 1" % PostponedSQL._meta.db_table)
-        if cursor.fetchall():
-            self._base_tables_created = True
-            logger.debug('[%s] The PostponedSQL storage already exists', self.connection.alias)
-            return
-        logger.info('[%s] The PostponedSQL storage is absent, creating', self.connection.alias)
-        with open(os.path.join(package_folder, 'sql/start.sql')) as f:
-            sql = f.read()
-        cursor.execute(sql)
+        PostponedSQL.objects.using(self.connection.alias)._create_base_tables()
         self._base_tables_created = True
 
     def _ignore(self):

django_postpone_index-0.0.3/postpone_index/models.py

@@ -0,0 +1,129 @@
+import logging
+import os
+import time
+
+from unlimited_char.fields import CharField
+
+from django.db import models, connections
+from django.utils.translation import gettext_lazy as _
+
+logger = logging.getLogger(__name__)
+package_folder = os.path.dirname(os.path.abspath(__file__))
+
+
+class PostponedSQLQuerySet(models.QuerySet):
+    """
+    QuerySet for PostponedSQL model.
+
+    Redefined for admin view when table is absent.
+    """
+
+    def _create_base_tables(self):
+        """
+        Create base package table if absent
+        """
+        if self._is_present():
+            logger.debug('[%s] The PostponedSQL storage already exists', self.db)
+            return
+        logger.info('[%s] The PostponedSQL storage is absent, creating', self.db)
+        with open(os.path.join(package_folder, 'sql/start.sql')) as f:
+            sql = f.read()
+        cursor = connections[self.db].cursor()
+        cursor.execute(sql)
+
+    def count(self):
+        """
+        Count only if table is present
+        """
+        if not self._is_present():
+            return 0
+        return super().count()
+
+    def _is_present(self):
+        """
+        Check if table is present
+        """
+        cursor = connections[self.db].cursor()
+        cursor.execute(
+            """
+                SELECT 1 FROM information_schema.tables
+                WHERE table_schema = 'public' AND table_name = '%s' limit 1
+            """ % PostponedSQL._meta.db_table
+        )
+        return bool(cursor.fetchall())
+
+
+class PostponedSQLManager(models.Manager):
+    """
+    Manager for PostponedSQL model.
+
+    Redefined for admin view when table is absent.
+    """
+
+    def get_queryset(self):
+        """
+        Get custom QuerySet
+        """
+        qs = PostponedSQLQuerySet(self.model, using=self._db)
+        if not qs._is_present():
+            return qs.none()
+        return qs
+
+
+class PostponedSQL(models.Model):
+    """Model to store postponed SQL commands"""
+
+    ts = models.BigIntegerField(
+        primary_key=True, editable=False,
+        default=time.time_ns,
+        verbose_name=_('Timestamp ns'),
+        help_text=_('Timestamp in nanoseconds')
+    )
+    description = CharField(
+        verbose_name=_('Description'),
+        help_text=_('Free-form short description')
+    )
+    sql = models.TextField(
+        verbose_name=_('SQL'),
+        help_text=_('Original SQL generated on the migration stage'),
+    )
+    table = CharField(
+        blank=True, null=True, db_index=True,
+        verbose_name=_('Table'),
+        help_text=_('Table name if applied to the table')
+    )
+    db_index = CharField(
+        blank=True, null=True, db_index=True,
+        verbose_name=_('Index'),
+        help_text=_('Index name if applied to the index')
+    )
+    fields = CharField(
+        blank=True, null=True,
+        verbose_name=_('Fields'),
+        help_text=_('Field names separated by comma if applied to the fields')
+    )
+    done = models.BooleanField(
+        default=False,
+        verbose_name=_('Done'),
+        help_text=_('Has the command been applied')
+    )
+    error = models.TextField(
+        blank=True, null=True,
+        verbose_name=_('Error'),
+        help_text=_('Last error reported when tried to apply')
+    )
+
+    objects = PostponedSQLManager()
+
+    @property
+    def d(self):
+        """Status mark"""
+        return '[X]' if self.done else '[E]' if self.error else '[ ]'
+
+    class Meta:
+        managed = False
+        verbose_name = _('Postponed SQL')
+        verbose_name_plural = _('Postponed SQLs')
+
+    def __str__(self):
+        return self.description

{django_postpone_index-0.0.1 → django_postpone_index-0.0.3}/setup.py

@@ -49,5 +49,15 @@ setup(
         'postgresql',
     ],
     license='LGPLv3',
-    packages=['postpone_index'],
+    packages=[
+        'postpone_index',
+        'postpone_index/contrib',
+        'postpone_index/contrib/postgres',
+        'postpone_index/contrib/postgis',
+        'postpone_index/management',
+        'postpone_index/management/commands',
+    ],
+    package_data={
+        'postpone_index': ['sql/start.sql'],
+    },
 )

django_postpone_index-0.0.1/django_postpone_index.egg-info/top_level.txt

@@ -1 +0,0 @@
-postpone_index

django_postpone_index-0.0.1/postpone_index/models.py

@@ -1,61 +0,0 @@
-import time
-
-from unlimited_char.fields import CharField
-
-from django.db import models
-from django.utils.translation import gettext_lazy as _
-
-
-class PostponedSQL(models.Model):
-    ts = models.BigIntegerField(
-        primary_key=True, editable=False,
-        default=time.time_ns,
-        verbose_name=_('Timestamp ns'),
-        help_text=_('Timestamp in nanoseconds')
-    )
-    description = CharField(
-        verbose_name=_('Description'),
-        help_text=_('Free-form short description')
-    )
-    sql = models.TextField(
-        verbose_name=_('SQL'),
-        help_text=_('Original SQL generated on the migration stage'),
-    )
-    table = CharField(
-        blank=True, null=True, db_index=True,
-        verbose_name=_('Table'),
-        help_text=_('Table name if applied to the table')
-    )
-    db_index = CharField(
-        blank=True, null=True, db_index=True,
-        verbose_name=_('Index'),
-        help_text=_('Index name if applied to the index')
-    )
-    fields = CharField(
-        blank=True, null=True,
-        verbose_name=_('Fields'),
-        help_text=_('Field names separated by comma if applied to the fields')
-    )
-    done = models.BooleanField(
-        default=False,
-        verbose_name=_('Done'),
-        help_text=_('Has the command been applied')
-    )
-    error = models.TextField(
-        blank=True, null=True,
-        verbose_name=_('Error'),
-        help_text=_('Last error reported when tried to apply')
-    )
-
-    @property
-    def d(self):
-        """Status mark"""
-        return '[X]' if self.done else '[E]' if self.error else '[ ]'
-
-    class Meta:
-        managed = False
-        verbose_name = _('Postponed SQL')
-        verbose_name_plural = _('Postponed SQLs')
-
-    def __str__(self):
-        return self.description