How to resolve the "psycopg2.errors.UndefinedTable: relation "auth_user" does not exist" when running django unittests on Travis

Question

I'm using Travis for CI/CD as part of my Django app, with a postgresql database. (Django 2.1.4)

The build consistently fails on Travis as soon as the tests run. I receive this error:

psycopg2.errors.UndefinedTable: relation "auth_user" does not exist

I have tried: makemigrations, migrate auth, migrate myapp, migrate --run-syncdb. All of which fail with the same error.

The tests run locally with a sqlite3 database, and on a prod-like heroku environment with a postgresql database.

.travis.yml

...
before script:
-psql -c 'create database travis_ci_test;' -U postgres
services:
-postgresql
script:
-yes | python3 manage.py makemigrations
-python3 manage.py migrate auth
-python3 manage.py migrate --run-syncdb
-python3 manage.py tests test/unit_tests

settings.py

...
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'travis_ci_test',
        'USER': 'postgres',
        'PASSWORD': '',
        'HOST': 'localhost',
        }
    }

...
INSTALLED_APPS = [...
        'django.contrib.auth',
        ]

Here is the output from the failing build on Travis. 'migrate auth' is successful (I think this is the crucial line for auth_user : Applying auth.0001_initial... OK)

0.22s$ psql -c 'create database travis_ci_test;' -U postgres
CREATE DATABASE
1.50s$ yes | python3 manage.py makemigrations
TEST_ENV...
AWS_INTEGRATION...
Databases ... {'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'travis_ci_test', 'USER': 'postgres', 'PASSWORD': '', 'HOST': 'localhost'}
Installed Apps ... ['django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'races.apps.RacesConfig', 'storages']
No changes detected
The command "yes | python3 manage.py makemigrations" exited with 0.
1.68s$ python3 manage.py migrate auth
TEST_ENV...
AWS_INTEGRATION...
Databases ... {'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'travis_ci_test', 'USER': 'postgres', 'PASSWORD': '', 'HOST': 'localhost'}
Installed Apps ... ['django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'races.apps.RacesConfig', 'storages']
Operations to perform:
  Apply all migrations: auth
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
The command "python3 manage.py migrate auth" exited with 0.
1.57s$ python3 manage.py migrate --run-syncdb
TEST_ENV...
AWS_INTEGRATION...
Databases ... {'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'travis_ci_test', 'USER': 'postgres', 'PASSWORD': '', 'HOST': 'localhost'}
Installed Apps ... ['django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'myapp.apps.MyAppConfig', 'storages']
Operations to perform:
  Synchronize unmigrated apps: messages, myapp, staticfiles, storages
  Apply all migrations: admin, auth, contenttypes, sessions
Synchronizing apps without migrations:
  Creating tables...
    Creating table myapp_model1
    Creating table myapp_model2
    Creating table myapp_model3
    Creating table myapp_model4
    Creating table myapp_model5
    Running deferred SQL...
Running migrations:
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying sessions.0001_initial... OK
The command "python3 manage.py migrate --run-syncdb" exited with 0.
1.40s$ python3 manage.py test tests/unit_tests
TEST_ENV...
AWS_INTEGRATION...
Databases ... {'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'travis_ci_test', 'USER': 'postgres', 'PASSWORD': '', 'HOST': 'localhost'}
Installed Apps ... ['django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'app.apps.MyAppConfig', 'storages']
Creating test database for alias 'default'...
Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.UndefinedTable: relation "auth_user" does not exist
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/commands/test.py", line 26, in run_from_argv
    super().run_from_argv(argv)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/base.py", line 316, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/base.py", line 353, in execute
    output = self.handle(*args, **options)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/commands/test.py", line 56, in handle
    failures = test_runner.run_tests(test_labels)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/test/runner.py", line 604, in run_tests
    old_config = self.setup_databases()
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/test/runner.py", line 551, in setup_databases
    self.parallel, **kwargs
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/test/utils.py", line 174, in setup_databases
    serialize=connection.settings_dict.get('TEST', {}).get('SERIALIZE', True),
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/db/backends/base/creation.py", line 68, in create_test_db
    run_syncdb=True,
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/__init__.py", line 148, in call_command
    return command.execute(*args, **defaults)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/base.py", line 353, in execute
    output = self.handle(*args, **options)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/base.py", line 83, in wrapped
    res = handle_func(*args, **kwargs)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/commands/migrate.py", line 172, in handle
    self.sync_apps(connection, executor.loader.unmigrated_apps)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/commands/migrate.py", line 310, in sync_apps
    self.stdout.write("    Running deferred SQL...\n")
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/db/backends/base/schema.py", line 106, in __exit__
    self.execute(sql)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/db/backends/base/schema.py", line 133, in execute
    cursor.execute(sql, params)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "auth_user" does not exist
The command "python3 manage.py test tests/unit_tests" exited with 1.

Solution 1

I ran into the same problem after adding a model for an app that was already installed, but forgetting to creating the initial migration. OP hid the answer in the comments so here are explicit steps:

$ mkdir myapp/migrations
$ touch myapp/migrations/__init__.py

After this I got a new error when trying to run python manage.py test

psycopg2.errors.InvalidCursorName: cursor "_django_curs_140073820227328_58" does not exist

One more step was needed:

$ python manage.py makemigrations

And then tests ran fine.

Solution 2

I deleted the sqllite3 database then I ran

$ python manage.py migrate 

then

$ python manage.py migrate --run-syncdb

Comments

[Lord Elrond]

This appears to happen with any inconsistency between your models and migration files when running unit tests.