Flask CLI throws 'OSError: [Errno 8] Exec format error' when run through docker-compose

Question

I'm running a Flask application with a Custom Script. Or trying to, anyway.

I'm on Windows 10 and the application ought to run in a linux Docker container with the command:

docker-compose up api

Docker-compose is version 1.23.2. In the dockerfile, the api service runs via the command:

command: python manage.py run --host "0.0.0.0" --with-threads

As it tries to start up, I see the exception

OSError: [Errno 8] Exec format error: '/api/manage.py'

I initially thought this would be the Dreaded Windows Line Endings, come for me once more, but running dos2unix on all my source files has not resolved the problem.

How can I avoid this error?

---

manage.py

    import click
    from flask.cli import FlaskGroup

    from my_app_api import create_app


    def create_my_app(info):
        return create_app()


    @click.group(cls=FlaskGroup, create_app=create_my_app)
    def cli():
        pass


    if __name__ == "__main__":
        cli()

Full traceback

api_1          | Traceback (most recent call last):
api_1          |   File "manage.py", line 22, in <module>
api_1          |     cli()
api_1          |   File "/usr/local/lib/python3.6/site-packages/click/core.py", line 764, in __call__
api_1          |     return self.main(*args, **kwargs)
api_1          |   File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 380, in main
api_1          |     return AppGroup.main(self, *args, **kwargs)
api_1          |   File "/usr/local/lib/python3.6/site-packages/click/core.py", line 717, in main
api_1          |     rv = self.invoke(ctx)
api_1          |   File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
api_1          |     return _process_result(sub_ctx.command.invoke(sub_ctx))
api_1          |   File "/usr/local/lib/python3.6/site-packages/click/core.py", line 956, in invoke
api_1          |     return ctx.invoke(self.callback, **ctx.params)
api_1          |   File "/usr/local/lib/python3.6/site-packages/click/core.py", line 555, in invoke
api_1          |     return callback(*args, **kwargs)
api_1          |   File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 64, in new_func
api_1          |     return ctx.invoke(f, obj, *args, **kwargs)
api_1          |   File "/usr/local/lib/python3.6/site-packages/click/core.py", line 555, in invoke
api_1          |     return callback(*args, **kwargs)
api_1          |   File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 438, in run_command
api_1          |     use_debugger=debugger, threaded=with_threads)
api_1          |   File "/usr/local/lib/python3.6/site-packages/werkzeug/serving.py", line 988, in run_simple
api_1          |     run_with_reloader(inner, extra_files, reloader_interval, reloader_type)
api_1          |   File "/usr/local/lib/python3.6/site-packages/werkzeug/_reloader.py", line 332, in run_with_reloader
api_1          |     sys.exit(reloader.restart_with_reloader())
api_1          |   File "/usr/local/lib/python3.6/site-packages/werkzeug/_reloader.py", line 176, in restart_with_reloader
api_1          |     exit_code = subprocess.call(args, env=new_environ, close_fds=False)
api_1          |   File "/usr/local/lib/python3.6/subprocess.py", line 287, in call
api_1          |     with Popen(*popenargs, **kwargs) as p:
api_1          |   File "/usr/local/lib/python3.6/subprocess.py", line 729, in __init__
api_1          |     restore_signals, start_new_session)
api_1          |   File "/usr/local/lib/python3.6/subprocess.py", line 1364, in _execute_child
api_1          |     raise child_exception_type(errno_num, err_msg, err_filename)
api_1          | OSError: [Errno 8] Exec format error: '/api/manage.py'

Solution 1

Looks like your api/manage.py doesn't have a shebang ([Wikipedia]: Shebang (Unix)), so the default (current) command processor (a shell - typically bash) is attempting to run it, which (obviously) fails.

To correct the problem, add a shebang (at the beginning of the file, making sure that your editor adds the Nix style line ending (\n, 0x0A, LF)):

• Default Python installation:

    #!/usr/bin/env python
  

  • Variant (specify Python 3 explicitly):

      #!/usr/bin/env python3
    

• Custom Python installation:

    #!/full/path/to/your/custom/python/executable
  

Note that you also need exec permissions on the file (chmod +x api/manage.py).

Example:

[cfati@cfati-5510-0:/cygdrive/e/Work/Dev/StackOverflow/q055271912]> ~/sopr.sh
*** Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ***

[prompt]> ls
code00.py  code01.py
[prompt]>
[prompt]> cat code00.py
print("This is:", __file__)

[prompt]> python3 -c "import os, subprocess;subprocess.Popen(os.path.join(os.getcwd(), \"code00.py\")).communicate()"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: '/cygdrive/e/Work/Dev/StackOverflow/q055271912/code00.py'
[prompt]>
[prompt]> cat code01.py
#!/usr/bin/env python3

print("This is:", __file__)

[prompt]> python3 -c "import os, subprocess;subprocess.Popen(os.path.join(os.getcwd(), \"code01.py\")).communicate()"
This is: /cygdrive/e/Work/Dev/StackOverflow/q055271912/code01.py

Another way would be to run the interpreter followed by the file name, but I don't know how to do it from Flask - actually that would require patching Werkzeug (_reloader.py: _get_args_for_reloading), but that would be just a lame workaround (gainarie) - see below.

---

Update #0

Looking at @AxelGrytt's answer, it turns out it's a known issue: [GitHub]: pallets/werkzeug - 0.15.0 causes OSError: [Errno 8] Exec format error: in Docker for Windows (hmm, submitted in the same day as this question (and 2 days after the release) :) ).

So, what I have stated above is correct, but it is worth mentioning that there is another way of fixing it: removing the exec permission for the file:

chmod -x api/manage.py

According to Werkzeug authors, from now on, this is desired behavior (also applies to v0.15.2):

• A file with exec permission set, should also have a shebang
• A file without a shebang, shouldn't have the exec permission set

Solution 2

This is a new behavior in Werkzeug 0.15. Downgrading to Werkzeug 0.14.1 may work, but 0.14 is no longer supported, so you will be better off correcting the issue with your file as described in the other answers.

Solution 3

If you disable debug mode (don't pass debug=True or set FLASK_DEBUG=0), the reloader will not be used and so this issue will not happen. The tradeoff is that you no longer have the reloader.

if __name__ == "__main__":
    connexion_app.run(host="0.0.0.0", port=constants.API_PORT, debug=True)

It is preferrable to fix this by ensuring files marked as executable have an interpreter line, like #!/usr/bin/env python3 (from https://stackoverflow.com/a/55272071).

Solution 4

The answer of @CristiFati worked for me with 1 additional step:

I also had to correct the EOL from \r\n to \n.

---

Sorry I don't have enough points to add a comment and have to open a new answer...

Solution 5

Werkzeug==0.15.4 has this question。

pip install --user Werkzeug==0.16

View more solutions