SyntaxError regarding async when attempting to run Jupyter in Python 3.7

Question

On attempting to launch Jupyter on Python 3.7, I observe the following:

$ jupyter notebook
Traceback (most recent call last):
  File "/usr/local/bin/jupyter-notebook", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/dist-packages/jupyter_core/application.py", line 266, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/traitlets/config/application.py", line 657, in launch_instance
    app.initialize(argv)
  File "<decorator-gen-7>", line 2, in initialize
  File "/usr/local/lib/python3.7/dist-packages/traitlets/config/application.py", line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/notebook/notebookapp.py", line 1629, in initialize
    self.init_webapp()
  File "/usr/local/lib/python3.7/dist-packages/notebook/notebookapp.py", line 1379, in init_webapp
    self.jinja_environment_options,
  File "/usr/local/lib/python3.7/dist-packages/notebook/notebookapp.py", line 158, in __init__
    default_url, settings_overrides, jinja_env_options)
  File "/usr/local/lib/python3.7/dist-packages/notebook/notebookapp.py", line 270, in init_settings
    nbextensions_path=jupyter_app.nbextensions_path,
  File "/usr/local/lib/python3.7/dist-packages/notebook/notebookapp.py", line 1066, in nbextensions_path
    from IPython.paths import get_ipython_dir
  File "/usr/local/lib/python3.7/dist-packages/IPython/__init__.py", line 54, in <module>
    from .core.application import Application
  File "/usr/local/lib/python3.7/dist-packages/IPython/core/application.py", line 25, in <module>
    from IPython.core import release, crashhandler
  File "/usr/local/lib/python3.7/dist-packages/IPython/core/crashhandler.py", line 27, in <module>
    from IPython.core import ultratb
  File "/usr/local/lib/python3.7/dist-packages/IPython/core/ultratb.py", line 119, in <module>
    from IPython.utils import path as util_path
  File "/usr/local/lib/python3.7/dist-packages/IPython/utils/path.py", line 17, in <module>
    from IPython.utils.process import system
  File "/usr/local/lib/python3.7/dist-packages/IPython/utils/process.py", line 19, in <module>
    from ._process_posix import system, getoutput, arg_split, check_pid
  File "/usr/local/lib/python3.7/dist-packages/IPython/utils/_process_posix.py", line 23, in <module>
    import pexpect
  File "/usr/lib/python3/dist-packages/pexpect/__init__.py", line 75, in <module>
    from .pty_spawn import spawn, spawnu
  File "/usr/lib/python3/dist-packages/pexpect/pty_spawn.py", line 15, in <module>
    from .spawnbase import SpawnBase
  File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 218
    def expect(self, pattern, timeout=-1, searchwindowsize=-1, async=False):
                                                                   ^
SyntaxError: invalid syntax

I've installed python3.7-distutils (which appears to feature pexpect) and I've reinstalled Jupyter using pip3, yet I still see this error. I've also uninstalled and reinstalled the following packages:

python3.7 python3.7-dev python3.7-distutils python3.7-tk

What should be done?

I see reference to this fix but, assuming it is a fix, don't know how to get it up and running on this 16.04 LTS install.

---

EDIT: On importing pexpect in Python 3.7, I get the same sort of thing:

>>> import pexpect
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/pexpect/__init__.py", line 75, in <module>
    from .pty_spawn import spawn, spawnu
  File "/usr/lib/python3/dist-packages/pexpect/pty_spawn.py", line 15, in <module>
    from .spawnbase import SpawnBase
  File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 218
    def expect(self, pattern, timeout=-1, searchwindowsize=-1, async=False):
                                                                   ^
SyntaxError: invalid syntax

Solution 1

I faced this problem while running django tests. Upgrade the version to the latest at the time (4.8), fixed the issue.

Solution 2

For me, it was conflicting with apt-get installed python3, so I had to first uninstall it/them:

sudo apt-get remove python-pexpect python3-pexpect

And then

sudo pip3.7 install --upgrade pexpect

Solution 3

Download pexpect4.6 source code from https://github.com/pexpect/pexpect/releases/tag/4.6

Extract the source code, get into the folder and install the pexpect by:

sudo python3.7 setup.py install

This should solve your problem. :)

Solution 4

I had the same problem and the following steps worked for me.

I went to /usrlib/python3/dist-packages/pexpect/ and there I

• renamed async.py to _async.py
• opened spawn.py and made these replacements:

  'async=' -> 'async_='
  'async:'-> 'async_:'
  '.async' -> '._async'
  

Hope it helps someone else, too.

View more solutions

Comments

[wjandrea]

Does python3.7 -c 'import pexpect' succeed?

[BlandCorporation]

@wjandrea Hey there, when I import pexpect in Python 3.7 I do get the same sort of error, and I've added the terminal output to the question. As I've linked in the question, it looks like something has been done to attempt to address this bug, which may arise because of changes in Python 3.7. Do you know how the current version of pexpect, or more generally distutils, can be installed in Ubuntu 16.04 LTS safely?

[wjandrea]

Yes, the issue is occurring because async was added as a reserved keyword in Python 3.7. When you import pexpect, it's importing the module written for Python 3.5 (which is the system version - /usr/lib/python3 is the system Python 3 library).

[wjandrea]

How did you install python3.7 and python3.7-distutils?

[BlandCorporation]

@wjandrea Thanks for your thoughts on this. I installed Python 3.7 and distutils for Python 3.7 in the following way: sudo apt install python3.7 python3.7-dev python3.7-distutils python3.7-tk (I've tried removing these packages and reinstalling them a few times now also).

[wjandrea]

The pexpect module should be at /usr/local/lib/python3.7/dist-packages/pexpect. Is it? Also what is the output of python3.7 -c 'import sys; print(sys.path)?

[Vedprakash Upraity]

Worked for me as well. Thanks :)

[igorkf]

This saved me. Thanks! (I'm using Python 3.8.0)

[cyrusbehr]

This solve the issue, thanks!

[vinh]

Upgrading linux mint from 19 to 20, eric threw this error. Editing the source solved the issue. Thanks @Jason_Information