16.03.2019       Выпуск 273 (11.03.2019 - 17.03.2019)       Статьи

Строим масштабируемый сервис обработки изображений на AWS Lambda

Читать>>




Экспериментальная функция:

Ниже вы видите текст статьи по ссылке. По нему можно быстро понять ссылка достойна прочтения или нет

Просим обратить внимание, что текст по ссылке и здесь может не совпадать.

Now let’s create something more useful than that.

Let’s build a function that takes an image as input and turns it to grayscale. We will be using OpenCV for that, specifically its Python bindings. Although using OpenCV might be overkill for such a task, it demonstrates how such a useful library can be included in your Lambda environment with relative ease.

We will now

  1. Generate a Lambda-ready Python package for OpenCV.
  2. Upload that package into Lambda Layers so it can be used in any function you build.
  3. Import OpenCV to our Lambda function.

1. Generate Lambda-ready Python package for OpenCV

I’ve put together a dead simple tool — a Docker image that can gather any pip package and generate a .ZIP we can upload to Lambda Layers. If you want to explore the tool you can find it from LambdaZipper.

If you have Docker installed you can open your terminal and just run

docker run --rm -v $(pwd):/package tiivik/lambdazipper opencv-python

That’s it! In your current working directory you’ll find opencv-python.zip

One of the most useful serverless toolkits isserverless. However we are not going to use it in this example. Re-inventing the wheel is rarely a good idea, with an exception when you want to learn how things work under the hood. Although mature frameworks such asserverlessexist, it is a good idea to dig into some of the core functionalities these frameworks abstract.

Let’s explore what the tool abstracted from us.

If you take a look at package.sh then you can see that it performed a pip install command with opencv-python argument. All that was executed in amazonlinux:2017.03 environment that, to some extent, mimics the AWS Lambda environment. You can explore the execution environment in the Dockerfile.

2. Upload the package into Lambda Layers so it can be used in any function you build

Let’s upload the opencv-python.zip to Lambda Layers so we can use that package from now on in all our functions. Think of Layers as data that can be used in any function you write. This can be Python modules, code snippets, binary files or anything.

Navigate to Layers panel in AWS Lambda and press Create layer.

Set up the layer name, description and upload the zip file. Make sure to select the correct runtime, in our case Python 3.6. Press Create layer.

As of writing this article, uploading the ZIP file from the web interface is limited to 50MB. Fortunately our opencv-python package is less than that. In case your package exceeds that you can provide the package as a link from an S3 bucket. Bear in mind that Lambda sets deployment package limit at 250MB.

After creating the function you should be greeted with a message

Successfully created layer opencv-python version 1.

Yay!

Let’s go back to our lambda-demo function and add the opencv-python layer to our function execution environment. Click on Layers > Add a layer and select your opencv-python layer.

3. Importing OpenCV

Let’s try importing the library as usual:

import json
import cv2
def lambda_handler(event, context):
# TODO implement
return {
'statusCode': 200,
'body': json.dumps('Hello from Lambda!')
}




Now let’s click on Test. We’re provided with the response:

Response:
{
"errorMessage": "Unable to import module 'lambda_function'"
}


For some reason Lambda wasn’t able to find our Python package. Let’s explore.

By default all Lambda layers are mounted to /opt. Let’s comment out our cv2 import and take a look what’s inside /opt.

import json
#import cv2
from os import listdir

def lambda_handler(event, context):
# TODO implement
print(listdir("/opt"))
return {
'statusCode': 200,
'body': json.dumps('Hello from Lambda!')
}





In the function logs we can see our cv2 module in /opt.

[‘bin’, ‘cv2’, ‘numpy’, ‘numpy-1.16.2.dist-info’, ‘opencv_python-4.0.0.21.dist-info’]

By default /opt/binis added to the $PATHenvironment variable. You can reference that from AWS docs. However our layer modules exist in /opt/not in/opt/bin. So, let’s include /opt into$PATH as well so Lambda can see our package.

In Environment Variables section, add the following environment variable. Key: PYTHONPATH Value: /opt/

Typically you can just import the package without altering the path, but in this case it’s necessary for Lambda environment to detect our package.

Let’s modify our code:

import json
import cv2
def lambda_handler(event, context):
# TODO implement
print(cv2.__version__)
return {
'statusCode': 200,
'body': json.dumps('Hello from Lambda!')
}





Save your changes and click Test. We are greeted with4.0.0 in the console, informing us the OpenCV version used.

Brilliant, Python OpenCV running in Lambda!



Лучшая Python рассылка




Разместим вашу рекламу

Пиши: mail@pythondigest.ru

Нашли опечатку?

Выделите фрагмент и отправьте нажатием Ctrl+Enter.

Система Orphus