EOFError: marshal data too short

Question

Why do I get the following error when I run the django server and how do I fix it ? :

EOFError: marshal data too short

details

Performing system checks...

Unhandled exception in thread started by <function check_errors.<locals>.wrapper at 0x7f6e5dbacea0>
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/django/utils/autoreload.py", line 227, in wrapper
    fn(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/django/core/management/commands/runserver.py", line 125, in inner_run
    self.check(display_num_errors=True)
  File "/usr/local/lib/python3.5/dist-packages/django/core/management/base.py", line 359, in check
    include_deployment_checks=include_deployment_checks,
  File "/usr/local/lib/python3.5/dist-packages/django/core/management/base.py", line 346, in _run_checks
    return checks.run_checks(**kwargs)
  File "/usr/local/lib/python3.5/dist-packages/django/core/checks/registry.py", line 81, in run_checks
    new_errors = check(app_configs=app_configs)
  File "/usr/local/lib/python3.5/dist-packages/django/core/checks/urls.py", line 16, in check_url_config
    return check_resolver(resolver)
  File "/usr/local/lib/python3.5/dist-packages/django/core/checks/urls.py", line 26, in check_resolver
    return check_method()
  File "/usr/local/lib/python3.5/dist-packages/django/urls/resolvers.py", line 254, in check
    for pattern in self.url_patterns:
  File "/usr/local/lib/python3.5/dist-packages/django/utils/functional.py", line 35, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/usr/local/lib/python3.5/dist-packages/django/urls/resolvers.py", line 405, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/usr/local/lib/python3.5/dist-packages/django/utils/functional.py", line 35, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/usr/local/lib/python3.5/dist-packages/django/urls/resolvers.py", line 398, in urlconf_module
    return import_module(self.urlconf_name)
  File "/usr/lib/python3.5/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 986, in _gcd_import
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 665, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/home/aphya1/work/1-erp-project/ERP/mis/mis/urls.py", line 14, in <module>
    url(r'^hr/', include('hr.urls', namespace='hr')),
  File "/usr/local/lib/python3.5/dist-packages/django/conf/urls/__init__.py", line 50, in include
    urlconf_module = import_module(urlconf_module)
  File "/usr/lib/python3.5/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 986, in _gcd_import
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 665, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/home/aphya1/work/1-erp-project/ERP/mis/hr/urls.py", line 2, in <module>
    from hr import views
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 661, in exec_module
  File "<frozen importlib._bootstrap_external>", line 765, in get_code
  File "<frozen importlib._bootstrap_external>", line 476, in _compile_bytecode
EOFError: marshal data too short

Solution

I also encountered the error:

EOFError: marshal data too short.

Even though no changes were made to any of the existing files, this error was thrown.

Some *.pyc files get created automatically which are compiled byte code of a script, which gets dynamically created when a Python script runs to speed up future launches.

I am using an Oracle VM VirtualBox manager, working on Python.

I was trying to install pyspark. The spark folder structure was something like this

spark****/python/pyspark 

Under this folder contained a folder by name '__pycache__' I went to the pyspark folder and deleted the '__pycache__' folder using the command:

rm -r __pycache__/

This eliminated the error for me and I am now able to import pyspark.

Hope this helps.

Comments

[Willem Van Onsem]

Because Python can not handle the marshal data, it is too short. Provide more marshal data :).

[Willem Van Onsem]

But now siriously: you need to provide more context. What part of the code is doing this?

[sshashank124]

Maybe this is relevant? groups.google.com/forum/#!topic/comp.lang.python/easB1mE3rJY

[sshashank124]

Seems to have to do with corrupted .pyc files

[shuuji3]

I had the same error. In my case, thanks to @sshashank124's comment, I can fix this problem by deleting not only __pychache__ of the main program directory but also ones of imported packages. You may find the __pycache__ by find command like this: $ find /usr/local/lib/python3.5/dist-packages/ -name __pycache__ | grep django/utils or so.

[L.Grozinger]

But does anyone know why this happens?

[ggorlen]

It happened to me when I did a search and replace in a file tree without filtering out .pyc files. But any other scenario that modifies/corrupts .pyc files could cause it.