Azure functions: Installing Python modules and extensions on consumption plan

Question

I am trying to run a python script with Azure functions. I had success updating the python version and installing modules on Azure functions under the App Services plan but I need to use it under the Consumption plan as my script will only execute once everyday and for only a few minutes, so I want to pay only for the time of execution. See: https://azure.microsoft.com/en-au/services/functions/

Now I'm still new to this but from my understanding the consumption plan spins up the vm and terminates it after your script has been executed unlike the App Service plan which is always on. I am not sure why this would mean that I can't have install anything on it. I thought that would just mean I have to install it every time I spin it up.

I have tried installing modules through the python script itself and the kudu command line with no success.

While under the app service plan it was simple, following this tutorial: https://prmadi.com/running-python-code-on-azure-functions-app/

Solution 1

On Functions Comsumption plan, Kudu extensions are not available. However, you can update pip to be able to install all your dependencies correctly:

• Create your Python script on Functions (let's say NameOfMyFunction/run.py)
• Open a Kudu console
• Go to the folder of your script (should be d:/home/site/wwwroot/NameOfMyFunction)
• Create a virtualenv in this folder (python -m virtualenv myvenv)
• Load this venv (cd myenv/Scripts and call activate.bat)

Your shell should be now prefixed by (myvenv)

• Update pip (python -m pip install -U pip)
• Install what you need (python -m pip install flask)

Now in the Azure Portal, in your script, update the sys.path to add this venv:

import sys, os.path
sys.path.append(os.path.abspath(os.path.join(os.path.dirname( __file__ ), 'myvenv/Lib/site-packages')))

You should be able to start what you want now.

(Reference: https://github.com/Azure/azure-sdk-for-python/issues/1044)

Edit: reading previous comment, it seems you need numpy. I just tested right now and I was able to install 1.12.1 with no issues.

Solution 2

You may upload the modules for the Python version of your choice in Consumption Plan. Kindly refer to the instructions at this link: https://github.com/Azure/azure-webjobs-sdk-script/wiki/Using-a-custom-version-of-Python

Comments

[mike]

Thanks Ling. For Python 2.7.13 I can't find an embeddable zip file. Am I right in thinking that updating the version and installing the modules that my scripts need will need to be done every time I start up a script (and consequently a vm) under the consumption plan.

[Ling Toh]

Sorry I am not familiar with Python and will defer the question about locating an embeddable zip file to others. As for your second question, the modules will be loaded when your Function App is first instantiated. A Function App is a process that hosts all the Functions. Under the Consumption Plan, each Function App has 5 mins to complete all its workloads before it is forcibly terminated. Your Function may be triggered and executed many times during this 5 min timeframe but the Python modules would have been loaded only once on instantiation of the Function App process.

[mike]

When trying to install numpy from inside my script I am met with: 2017-05-15T06:33:53.919 WindowsError: [Error 5] Access is denied: 'D:\\Python27\\Lib\\site-packages\\numpy'

[Ling Toh]

I have seen this error before. Unfortunately, you will not be able to install numpy on that path. Kindly see the following related SO thread: stackoverflow.com/a/40474485/6465830

[mike]

Thanks @Laurent Mazuel ! This worked perfectly and not only for numpy and pandas but for every module I tried.

[Ling Toh]

@Laurent Manuel, thanks for great instructions and your help in unblocking @mike!

[JrtPec]

This approach works for me, but importing pandas slows my process way down. The import alone takes more than 20 seconds. Did I do something wrong or is this expected behaviour?

[Quang Hoàng]

Please help, I run activate.bat but there is no prefix (myvenv) appear

[Thomas]

After installing external python module through the kudu console, Can I assume that it is gonna sclae properly ? If a new VM is spin up, the additional modules will also be installed ?

[Matthias Herrmann]

Is it possible to leave the requirements.txt empty only with a dot and it will run a custom setup script before deploying to the cloud?