Celery raises ValueError: not enough values to unpack

Question

Trying to run simple example with Celery and receiving an exception. RabbitMQ started in a Docker, also tried to start it locally. Celery works on a local Windows host

from celery import Celery

app = Celery('tasks', broker='amqp://192.168.99.100:32774')

@app.task()
def hello():
    print('hello')


if __name__ == '__main__':
    hello.delay()

Excerpt of my error text:

[2017-08-18 00:01:08,632: ERROR/MainProcess] Task handler raised error: ValueError('not enough values to unpack (expected 3, got 0)',)
Traceback (most recent call last):                                              
    File "c:\users\user\celenv\lib\site-packages\billiard\pool.py", line 358, in workloop                        
        result = (True, prepare_result(fun(*args, **kwargs)))                                  
    File "c:\users\user\celenv\lib\site-packages\celery\app\trace.py", line 525, in _fast_trace_task
        tasks, accept, hostname = _loc
ValueError: not enough values to unpack (expected 3, got 0)

Solution 1

Which celery version? As far as I remember celery isn't supported in windows since celery 4

Solution 2

Celery 4.0+ does not officially support Windows yet. But it still works on Windows for some development/test purposes.

Use eventlet instead as below:

pip install eventlet
celery -A <module> worker -l info -P eventlet

It works for me on Windows 10 + celery 4.1 + python 3.

===== update 2018-11 =====

Eventlet has an issue on subprocess.CalledProcessError:

https://github.com/celery/celery/issues/4063

https://github.com/eventlet/eventlet/issues/357

https://github.com/eventlet/eventlet/issues/413

So try gevent instead.

pip install gevent
celery -A <module> worker -l info -P gevent

This works for me on Windows 10 + celery 4.2 + python 3.6

Solution 3

I got this error on Windows 7 32bit system. So I did this to make it work.

Add this

`os.environ.setdefault('FORKED_BY_MULTIPROCESSING', '1')` 

before defining a celery instance in myproj/settings.py file in your django project.

It should like like

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myproj.settings')
os.environ.setdefault('FORKED_BY_MULTIPROCESSING', '1')

app = Celery('tasks', broker='redis://127.0.0.1:6379/0')

I am using redis as a messaging broker so defined broker='redis://127.0.0.1:6379/0'

Solution 4

For Celery 4.1 on Windows.

Set an environment variable FORKED_BY_MULTIPROCESSING=1. Then you can simply run celery -A <celery module> worker.

Solution 5

It worked for me:

celery -A my_project_name worker --pool=solo -l info

basically things become single threaded and are suppoted

View more solutions

Comments

[ItayB]

which celery version? as far as I remember celery doesn't supported in windows since celery 4

[Igor Nikolaev]

@ItayB Thank you! I've found that the problem exactly in celery on windows, but I didn't know this about version 4. I've used 4.1.0.

[ItayB]

add it as answer, you can vote/accept please

[IamMashed]

this should be selected answer

[uglycoyote]

gevent worked for me. eventlet gave me problems with eventlet version 0.24.1 and Celery 4.3.0 (TypeError: GreenSSLSocket does not have a public constructor. Instances are returned by SSLContext.wrap_socket().).

[Coffee and Code]

Thanks, this worked for me. Would have spent hours (on top of the ones already spent). I was just trying the example from docs.celeryproject.org: import os os.environ.setdefault('FORKED_BY_MULTIPROCESSING', '1') app = Celery('tasks', broker='amqp://guest@localhost//') etc

[Azima]

I tried running with -l info -c 5 gevent flags and got unrecognized arguments: gevent. It however works with -l info -P gevent. How can I fix this?

[Luis]

Still working on 2020 with Win10 + Celery 4.4.7 + Python 3.8.5

[Brontes]

Still working in 2021 with Win10 + Celery 5.1.2 + Python 3.6.8

[Arfath Yahiya]

You are a lifesaver man