django-async-upload-kg 4.0.1__tar.gz

django-async-upload-kg-4.0.1/LICENSE

@@ -0,0 +1,20 @@
+The MIT License (MIT)
+
+Copyright (c) 2014 jonatron
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of
+this software and associated documentation files (the "Software"), to deal in
+the Software without restriction, including without limitation the rights to
+use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
+the Software, and to permit persons to whom the Software is furnished to do so,
+subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
+FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
+COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

django-async-upload-kg-4.0.1/MANIFEST.in

@@ -0,0 +1,3 @@
+include README.md
+include admin_resumable/templates/admin_resumable/file_input.html
+include admin_resumable/static/admin_resumable/js/resumable.js

django-async-upload-kg-4.0.1/PKG-INFO

@@ -0,0 +1,82 @@
+Metadata-Version: 2.1
+Name: django-async-upload-kg
+Version: 4.0.1
+Summary: A Django app for the uploading of large files from the django admin site.
+Home-page: https://github.com/bit/django-async-upload
+Author: Kajala Group Ltd.
+Author-email: support@kajala.com
+License: MIT License
+Classifier: Environment :: Web Environment
+Classifier: Framework :: Django
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Topic :: Internet :: WWW/HTTP
+Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
+License-File: LICENSE
+Requires-Dist: Django>=3.0.14
+
+django-async-upload
+===============================
+
+Based on fork from https://github.com/bit/django-async-upload. Allows install via pip:
+
+        pip install django-async-upload-kg
+
+django-async-upload is a django app to allow you to upload large files from within the django admin site asynchrously, that means that you can add any number of files on the admin page (e.g. through inline models) and continue editing other fields while files are uploading.
+
+django-async-upload is compatible with django-storages (tested with S3Storage)
+
+
+Installation
+------------
+
+* pip install django-async-upload
+* Add ``admin_async_upload`` to your ``INSTALLED_APPS``
+* Add ``url(r'^admin_async_upload/', include('admin_async_upload.urls')),`` to your urls.py
+* Add a model field eg: ``from admin_resumable.models import ResumableFileField``
+
+::
+
+    class Foo(models.Model):
+        bar = models.CharField(max_length=200)
+        foo = AsyncFileField()
+
+
+
+Optionally:
+
+* Set ``ADMIN_RESUMABLE_CHUNKSIZE``, default is ``"1*1024*1024"``
+* Set ``ADMIN_RESUMABLE_STORAGE``, default is setting of DEFAULT_FILE_STORAGE and ultimately ``'django.core.files.storage.FileSystemStorage'``.  If you don't want the default FileSystemStorage behaviour of creating new files on the server with filenames appended with _1, _2, etc for consecutive uploads of the same file, then you could use this to set your storage class to something like https://djangosnippets.org/snippets/976/
+* Set ``ADMIN_RESUMABLE_CHUNK_STORAGE``, default is ``'django.core.files.storage.FileSystemStorage'`` .  If you don't want the default FileSystemStorage behaviour of creating new files on the server with filenames appended with _1, _2, etc for consecutive uploads of the same file, then you could use this to set your storage class to something like https://djangosnippets.org/snippets/976/
+* Set ``ADMIN_RESUMABLE_SHOW_THUMB``, default is False. Shows a thumbnail next to the "Currently:" link.
+* Set ``ADMIN_SIMULTANEOUS_UPLOADS`` to limit number of simulteneous uploads, dedaults to `3`. If you have broken pipe issues in local development environment, set this value to `1`.
+
+
+Versions
+--------
+
+4.0.0   - inital fork of django-admin-async-upload 3.0.4 with support for Django 4 and later
+
+
+Compatibility
+-------------
+
+Tested on Django 4.2 running on python 3.11
+
+Thanks to
+---------
+
+original django-admin-resumable-js by jonatron https://github.com/jonatron/django-admin-resumable-js 
+
+django-admin-resumable-js fork by roxel https://github.com/roxel/django-admin-resumable-js
+
+django-admin-async-upload fork by DataGreed https://github.com/DataGreed/django-admin-async-upload
+
+Resumable.js https://github.com/23/resumable.js
+
+django-resumable https://github.com/jeanphix/django-resumable
+
+

django-async-upload-kg-4.0.1/README.rst

@@ -0,0 +1,62 @@
+django-async-upload
+===============================
+
+Based on fork from https://github.com/bit/django-async-upload. Allows install via pip:
+
+        pip install django-async-upload-kg
+
+django-async-upload is a django app to allow you to upload large files from within the django admin site asynchrously, that means that you can add any number of files on the admin page (e.g. through inline models) and continue editing other fields while files are uploading.
+
+django-async-upload is compatible with django-storages (tested with S3Storage)
+
+
+Installation
+------------
+
+* pip install django-async-upload
+* Add ``admin_async_upload`` to your ``INSTALLED_APPS``
+* Add ``url(r'^admin_async_upload/', include('admin_async_upload.urls')),`` to your urls.py
+* Add a model field eg: ``from admin_resumable.models import ResumableFileField``
+
+::
+
+    class Foo(models.Model):
+        bar = models.CharField(max_length=200)
+        foo = AsyncFileField()
+
+
+
+Optionally:
+
+* Set ``ADMIN_RESUMABLE_CHUNKSIZE``, default is ``"1*1024*1024"``
+* Set ``ADMIN_RESUMABLE_STORAGE``, default is setting of DEFAULT_FILE_STORAGE and ultimately ``'django.core.files.storage.FileSystemStorage'``.  If you don't want the default FileSystemStorage behaviour of creating new files on the server with filenames appended with _1, _2, etc for consecutive uploads of the same file, then you could use this to set your storage class to something like https://djangosnippets.org/snippets/976/
+* Set ``ADMIN_RESUMABLE_CHUNK_STORAGE``, default is ``'django.core.files.storage.FileSystemStorage'`` .  If you don't want the default FileSystemStorage behaviour of creating new files on the server with filenames appended with _1, _2, etc for consecutive uploads of the same file, then you could use this to set your storage class to something like https://djangosnippets.org/snippets/976/
+* Set ``ADMIN_RESUMABLE_SHOW_THUMB``, default is False. Shows a thumbnail next to the "Currently:" link.
+* Set ``ADMIN_SIMULTANEOUS_UPLOADS`` to limit number of simulteneous uploads, dedaults to `3`. If you have broken pipe issues in local development environment, set this value to `1`.
+
+
+Versions
+--------
+
+4.0.0   - inital fork of django-admin-async-upload 3.0.4 with support for Django 4 and later
+
+
+Compatibility
+-------------
+
+Tested on Django 4.2 running on python 3.11
+
+Thanks to
+---------
+
+original django-admin-resumable-js by jonatron https://github.com/jonatron/django-admin-resumable-js 
+
+django-admin-resumable-js fork by roxel https://github.com/roxel/django-admin-resumable-js
+
+django-admin-async-upload fork by DataGreed https://github.com/DataGreed/django-admin-async-upload
+
+Resumable.js https://github.com/23/resumable.js
+
+django-resumable https://github.com/jeanphix/django-resumable
+
+

django-async-upload-kg-4.0.1/admin_async_upload/fields.py

@@ -0,0 +1,14 @@
+from django.core.exceptions import ValidationError
+from django.forms import fields
+
+from admin_async_upload.widgets import ResumableAdminWidget
+
+
+class FormResumableFileField(fields.FileField):
+    widget = ResumableAdminWidget
+
+    def to_python(self, data):
+        if self.required:
+            if not data or data == "None":
+                raise ValidationError(self.error_messages['empty'])
+        return data

django-async-upload-kg-4.0.1/admin_async_upload/files.py

@@ -0,0 +1,150 @@
+# -*- coding: utf-8 -*-
+import fnmatch
+import tempfile
+
+from django.contrib.contenttypes.models import ContentType
+from django.core.files import File
+from django.utils.functional import cached_property
+
+from admin_async_upload.storage import ResumableStorage
+
+
+class ResumableFile(object):
+    """
+    Handles file saving and processing.
+    It must only have access to chunk storage where it saves file chunks.
+    When all chunks are uploaded it collects and merges them returning temporary file pointer
+    that can be used to save the complete file to persistent storage.
+
+    Chunk storage should preferably be some local storage to avoid traffic
+    as files usually must be downloaded to server as chunks and re-uploaded as complete files.
+    """
+
+    def __init__(self, field, user, params):
+        self.field = field
+        self.user = user
+        self.params = params
+        self.chunk_suffix = "_part_"
+
+    @cached_property
+    def resumable_storage(self):
+        return ResumableStorage()
+
+    @cached_property
+    def persistent_storage(self):
+        return self.resumable_storage.get_persistent_storage()
+
+    @cached_property
+    def chunk_storage(self):
+        return ResumableStorage().get_chunk_storage()
+
+    @property
+    def storage_filename(self):
+        instance_id = self.params.get("instance_id")
+        if instance_id:
+            instance = self.field.model.objects.filter(pk=instance_id).first()
+        else:
+            instance = None
+        return self.resumable_storage.full_filename(self.filename, self.upload_to, instance=instance)
+
+    @property
+    def upload_to(self):
+        return self.field.upload_to
+
+    @property
+    def chunk_exists(self):
+        """
+        Checks if the requested chunk exists.
+        """
+        return self.chunk_storage.exists(self.current_chunk_name) and \
+               self.chunk_storage.size(self.current_chunk_name) == int(self.params.get('resumableCurrentChunkSize'))
+
+    @property
+    def chunk_names(self):
+        """
+        Iterates over all stored chunks.
+        """
+        chunks = []
+        files = sorted(self.chunk_storage.listdir('')[1])
+        for file in files:
+            if fnmatch.fnmatch(file, '%s%s*' % (self.filename,
+                                                self.chunk_suffix)):
+                chunks.append(file)
+        return chunks
+
+    @property
+    def current_chunk_name(self):
+        # TODO: add user identifier to chunk name
+        return "%s%s%s" % (
+            self.filename,
+            self.chunk_suffix,
+            self.params.get('resumableChunkNumber').zfill(4)
+        )
+
+    def chunks(self):
+        """
+        Iterates over all stored chunks.
+        """
+        # TODO: add user identifier to chunk name
+        files = sorted(self.chunk_storage.listdir('')[1])
+        for file in files:
+            if fnmatch.fnmatch(file, '%s%s*' % (self.filename,
+                                                self.chunk_suffix)):
+                yield self.chunk_storage.open(file, 'rb').read()
+
+    def delete_chunks(self):
+        [self.chunk_storage.delete(chunk) for chunk in self.chunk_names]
+
+    @property
+    def file(self):
+        """
+        Merges file and returns its file pointer.
+        """
+        if not self.is_complete:
+            raise Exception('Chunk(s) still missing')
+        outfile = tempfile.NamedTemporaryFile("w+b")
+        for chunk in self.chunk_names:
+            outfile.write(self.chunk_storage.open(chunk).read())
+        return outfile
+
+    @property
+    def filename(self):
+        """
+        Gets the filename.
+        """
+        # TODO: add user identifier to chunk name
+        filename = self.params.get('resumableFilename')
+        if '/' in filename:
+            raise Exception('Invalid filename')
+        value = "%s_%s" % (self.params.get('resumableTotalSize'), filename)
+        return value
+
+    @property
+    def is_complete(self):
+        """
+        Checks if all chunks are already stored.
+        """
+        return int(self.params.get('resumableTotalSize')) == self.size
+
+    def process_chunk(self, file):
+        """
+        Saves chunk to chunk storage.
+        """
+        if self.chunk_storage.exists(self.current_chunk_name):
+            self.chunk_storage.delete(self.current_chunk_name)
+        self.chunk_storage.save(self.current_chunk_name, file)
+
+    @property
+    def size(self):
+        """
+        Gets size of all chunks combined.
+        """
+        size = 0
+        for chunk in self.chunk_names:
+            size += self.chunk_storage.size(chunk)
+        return size
+
+    def collect(self):
+        actual_filename = self.persistent_storage.save(self.storage_filename, File(self.file))
+        self.delete_chunks()
+        return actual_filename

django-async-upload-kg-4.0.1/admin_async_upload/models.py

@@ -0,0 +1,15 @@
+from django.db import models
+from admin_async_upload.widgets import ResumableAdminWidget
+from admin_async_upload.fields import FormResumableFileField
+
+
+class AsyncFileField(models.FileField):
+
+    def formfield(self, **kwargs):
+        defaults = {'form_class': FormResumableFileField}
+        if self.model and self.name:
+            defaults['widget'] = ResumableAdminWidget(attrs={
+                'model': self.model,
+                'field_name': self.name})
+        kwargs.update(defaults)
+        return super(AsyncFileField, self).formfield(**kwargs)