04.06.2018       Выпуск 233 (04.06.2018 - 10.06.2018)       Статьи

Загружаем файл из frontend'а в Amazon S3

На сервере Django


Экспериментальная функция:

Ниже вы видите текст статьи по ссылке. По нему можно быстро понять ссылка достойна прочтения или нет

Просим обратить внимание, что текст по ссылке и здесь может не совпадать.

A common problem appears when uploading large files to Heroku. Every request made to Heroku must last less than 30 seconds or it will get terminated, when uploading large files, 30 seconds might not be enough. More information can be found here.

One way to deal with this situation is to upload files to Amazon S3 directly from the browser. On this post we will show how to do this using Django.

There is currently a Django app that provides a complete solution for the problem and can be found here. If you need something a bit more flexible you might want to implement your own, if that is your case, continue reading and we will guide you through it.

The full example can be found on our repository.

Configuring S3

For this guide we will take into consideration that Django is already working with S3. For more information on how to get this setup going, take a look at How to configure Sass and Bower with django-compressor - part 2 (deployment to Heroku and S3).

To get your bucket ready go to S3 bucket list, right click on the bucket, then Properties, after Permissions and on Add/Edit CORS Configuration and fill as below:

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://S3.amazonaws.com/doc/2006-03-01/">

Once you have your domain you can change AllowedOrigin from * to yourdomain.com.


For this post we will be using the following model:

from django.db import models
import uuid

def document_upload_path(instance, filename):
    return 'documents/{}/{}'.format(uuid.uuid4(),

class Document(models.Model):
    name = models.CharField(max_length=100)
    doc_file = models.FileField(upload_to=document_upload_path)


We are going to need an endpoint for the client to authenticate with S3. For that endpoint we use boto to create the arguments for the POST request that will upload the file to S3. The code for the endpoint is pretty generic and you should only need to change path where to upload the file. You can see the endpoint here.


The form used to upload the document will look like:

from django import forms
from django.forms import ValidationError
from crispy_forms.helper import FormHelper
from crispy_forms.layout import Submit

from example.models import Document

class DocumentForm(forms.ModelForm):
    # since we are not going to send the file via POST we need a field to save
    # the already uploaded path of the uploaded file.
    file_name = forms.CharField(required=False)

    class Meta:
        model = Document
        fields = ('id', 'name', 'doc_file', 'file_name')

    def __init__(self, *args, **kwargs):
        # crispy forms
        self.helper = FormHelper()
        self.helper.form_class = 's3upload-form'
        self.helper.add_input(Submit('submit', 'Submit'))
        super(DocumentForm, self).__init__(*args, **kwargs)
        # doc_file is not required if file_name is set
        self.fields['doc_file'].required = False
        self.fields['file_name'].widget.attrs['readonly'] = True

    def clean(self):
        file_name = self.cleaned_data.get('file_name')
        doc_file = self.cleaned_data.get('doc_file')
        # if doc_file and file_name is not set we are missing the file
        if not doc_file and not file_name:
                                .error_messages['required'], code='required'))
        # if file_name is set with the path of the file it was uploaded by
        # the frontend
        elif not doc_file and file_name:
            self.cleaned_data['doc_file'] = file_name
        return super(DocumentForm, self).clean()


For the frontend we need to make a request to authenticate before we can POST to S3. The full code for the frontend can be found here. The main part of the code will look similar to this:

    // first we need to get signature for authorization
    $.ajax('/example/documents/s3auth/?' + 'file_name=' + filename).done(function (data) {
      // now we can construct the payload with the signature
      var fd = new FormData();

      for (var key in data.form_args.fields) {
        if (data.form_args.fields.hasOwnProperty(key)) {
          console.log(key, data.form_args.fields[key]);
          fd.append(key, data.form_args.fields[key]);
      fd.append('file', evt.target.files[0]);

      $.ajax({method: "POST",
             url: data.form_args.action,
             data: fd,
             processData: false,
             contentType: false,
             success: function(){
               // set the hidden field to the uploaded file's path
               // clear the input_file so we don't send it when submitting the form
             error: function(){
               // clear the field so the user can try again.


You might want to make sure it works on localhost without S3. A nice way to achieve this is by creating a context_processor:

# app.context_processors.py
from django.conf import settings

def use_s3(context):
    # add flag to check on template if we should upload to S3
    # on local host
    return {'USE_S3': settings.USE_S3}

Append the context processor to your settings.py:

# Django 1.8+
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'app.context_processors.use_s3', # add this!
# Django <1.8
    'app.context_processors.use_s3', # add this!

# and set USE_S3
USE_S3 = False

Now on your template you can check for USE_S3:

    {% if USE_S3 %}
      $( document ).ready(function() {
        // call your frontend code!
    {% endif %}

Leave your comments down bellow!

Лучшая Python рассылка

Разместим вашу рекламу

Пиши: mail@pythondigest.ru

Нашли опечатку?

Выделите фрагмент и отправьте нажатием Ctrl+Enter.

Система Orphus