Script timed out before returning headers: wsgi.py on elastic beanstalk

Question

I'm trying to deploy a Django application to Elastic Beanstalk. When I visit the page it never loads. The logs say:

Script timed out before returning headers: wsgi.py

I can ssh into the server and run manage.py runserver and then curl 127.0.0.1:8000 from another terminal, which will return the page successfully. So I'm assuming it must be an issue with the Apache configuration that is set up as a part of Elastic Beanstalk.

Here is more of the logs:

[pid 15880] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[so:warn] [pid 15880] AH01574: module wsgi_module is already loaded, skipping
[auth_digest:notice] [pid 15880] AH01757: generating secret for digest authentication ...
[lbmethod_heartbeat:notice] [pid 15880] AH02282: No slotmem from mod_heartmonitor
[mpm_prefork:notice] [pid 15880] AH00163: Apache/2.4.9 (Amazon) mod_wsgi/3.4 Python/2.7.5       configured -- resuming normal operations
[core:notice] [pid 15880] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
[:error] [pid 15881] /opt/python/run/venv/lib/python2.7/site-packages/numpy/oldnumeric/__init__.py:11: ModuleDeprecationWarning: The oldnumeric module will be dropped in Numpy 1.9
[:error] [pid 15881]   warnings.warn(_msg, ModuleDeprecationWarning)
[:error] [pid 15881] 
[core:error] [pid 15884] [client 10.248.110.45:58996] Script timed out before returning headers: wsgi.py

And my wsgi.py file:

import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "aurora.settings")

from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()

Any clues as to what could be causing this?

UPDATE:

I rebuilt my environment and ran into this issue again. I updated /etc/httpd/conf.d/wsgi.conf to include WSGIApplicationGroup %{GLOBAL} as mentioned here. I am using Scipy, Numpy, and GeoDjango (which uses GDAL). I know GDAL is not entirely thread safe and I'm not sure about the others but I'm assuming it was a thread safety issue.

Solution 1

UPDATE 8 FEB 2017

Previously my wsgi.conf was only using one process:

WSGIDaemonProcess wsgi processes=1 threads=15 display-name=%{GROUP}

I upped the processes to something more reasonable and haven't had any issues:

WSGIDaemonProcess wsgi processes=6 threads=15 display-name=%{GROUP}

This change along with the original addition of WSGIApplicationGroup %{GLOBAL} seems to have done the trick.

UPDATE 17 September 2015

I'm still occasionally running in to this issue. Usually, redeploying via eb deploy fixes the issue. It's hard to say what the underlying issue is.

Original Answer

I eventually got the project working but then tried creating an image to use for new instances, which reopened the problem. I'm not sure why it worked then stopped working but I rebuilt my custom AMI from scratch and then repushed my project. Turns out it was an issue in wsgi.py. The version I posted was actually the different from what was being deployed. For some reason another developer had put this in wsgi.py:

path = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
if path not in sys.path:
sys.path.append(path)

I removed this and it fixed the problem.

My advice for anyone having

Script timed out before returning headers: wsgi.py

is to check you wsgi.py file.

Solution 2

The fix for us was to add the WSGIApplicationGroup %{GLOBAL} setting as per Meistro's answer.

You'll want to ensure you edit your wsgi config by way of your .ebextensions/foobar.config file, so that the changes are permanent. See the .ebextensions config docs.

Add the following to your .ebextensions/foobar.config file:

files:
  "/etc/httpd/conf.d/wsgi_custom.conf":
    mode: "000644"
    owner: root
    group: root
    content: |
      WSGIApplicationGroup %{GLOBAL}

This will create (or overwrite) the contents of the /etc/httpd/conf.d/wsgi_custom.conf file with WSGIApplicationGroup %{GLOBAL}

Solution 3

It certainly does seem like an issue with WSGI and Apache like you mentioned. One thing to double check is the .ebextensions file in your source directory.

There should be a config in there that specifies the WSGI information like the location of the application. You might also want to check your Django settings and run it locally with an Apache setup using WSGI.

You've probably already read the official documentation for WSGI and Django, but this might catch some simplistic things that you might have missed: http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_Python_django.html#create_deploy_Python_django_update

Solution 4

Just my 2 cents on a similar issue I faced.

I had a similar problem. It turned out to be that the script (which makes a DB insert/update/delete call) being executed from django application was waiting in a deadlock state for the lock on the table to be released. Once it was released, the code went through without this error. I never had to re-deploy my EB application.

1. Make sure that you are not connected to the database through any other SQL client. If yes, query for any active locks. (In my case, I was connecting to redshift. Querying STV_LOCKS table lists the active locks).
2. Check who is holding the locks. (In my case, it was my SQL client, connected to redshift, which executed a CUD operation, and since COMMIT was not issued, it was holding a pending write lock on the table).
3. I issued a commit from my SQL client, and the lock was released. My EB code went through and there was no Script timed out error any more.

Solution 5

I had this problem until I realized that I was using a different Python version. Let me explain this. I was using a CentOS 7. The default Python version in CentOS 7 is Python 2.7, but my code use Python 3.6, so I installed Python 3.6 in the same machine by doing this:

sudo yum install centos-release-scl
sudo yum install rh-python36 rh-python36-python-pip rh-python36-python-virtualenv
scl enable rh-python36 bash

Then created a virtual environment and used it in WSGI:

    WSGIScriptAlias   / /var/www/myproject/myproject/wsgi.py
    WSGIDaemonProcess myproject python-home=/var/www/myproject python-path=/var/www/myproject:/var/www/myproject/lib/python3.6/site-packages
    WSGIProcessGroup  myproject    

And the "script timed out" problem appeared. Then I realize that the mod_wsgi.so was compiled against libpython2.7.so.1.0.

# ldd /usr/lib64/httpd/modules/mod_wsgi.so
linux-vdso.so.1 =>  (0x00007ffd3fcae000)
libpython2.7.so.1.0 => /lib64/libpython2.7.so.1.0 (0x00007f1240cd4000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f1240ab8000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f12408b3000)
libutil.so.1 => /lib64/libutil.so.1 (0x00007f12406b0000)
libm.so.6 => /lib64/libm.so.6 (0x00007f12403ae000)
libc.so.6 => /lib64/libc.so.6 (0x00007f123ffe0000)
/lib64/ld-linux-x86-64.so.2 (0x00005598a5aac000)

The solution was removing mod_wsgi package and installing rh-python36-mod_wsgi package.

Best regards!

View more solutions