Missing Table When Running Django Unittest with Sqlite3

Question

I'm trying to run a unittest with Django 1.3. Normally, I use MySQL as my database backend, but since this is painfully slow to spinup for a single unittest, I'm using Sqlite3.

So to switch to Sqlite3 just for my unittests, in my settings.py I have:

import sys
if 'test' in sys.argv:
    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.sqlite3',
            'NAME':'/tmp/database.db',
            'USER'       : '',
            'PASSWORD' : '',
            'HOST'     : '',
        }
    }

When I run my unittest with python manage.py test myapp.Test.test_myfunc, I get the error:

DatabaseError: no such table: django_content_type

Googling shows there are a few of possible reasons for this error, none of which seem applicable to me. I'm not running Apache, so I don't see how permissions would be an issue. The file /tmp/database.db is being created, so /tmp is writable. The app django.contrib.contenttypes is included in my INSTALLED_APPS.

What am I missing?

Edit: I ran into this problem again in Django 1.5, but none of the proposed solutions work.

Solution 1

In Django 1.4, 1.5, 1.6, 1.7, or 1.8 it should be sufficient to use:

if 'test' in sys.argv:
    DATABASES['default']['ENGINE'] = 'django.db.backends.sqlite3'

It should not be necessary to override TEST_NAME¹, nor to call syncdb in order to run tests. As @osa points out, the default with the SQLite engine is to create the test database in memory (TEST_NAME=':memory:'). Calling syncdb should not be necessary because Django's test framework will do this automatically via a call to syncdb or migrate depending on the Django version.² You can observe this with manage.py test -v [2|3].

Very loosely speaking Django sets up the test environment by:

1. Loading the regular database NAME from your settings.py
2. Discovering and constructing your test classes (__init__() is called)
3. Setting the database NAME to the value of TEST_NAME
4. Running the tests against the database NAME

Here's the rub: At step 2, NAME is still pointing at your regular (non-test) database. If your tests contain class-level queries or queries in __init__(), they will be run against the regular database which is likely not what you are expecting. This is identified in bug #21143.

Don't do:

class BadFooTests(TestCase):
    Foo.objects.all().delete()     # <-- class level queries, and

    def __init__(self):
        f = Foo.objects.create()   # <-- queries in constructor
        f.save()                   #     will run against the production DB

    def test_foo(self):
        # assert stuff

since these will be run against the database specified in NAME. If NAME at this stage points to a valid database (e.g. your production database), the query will run, but may have unintended consequences. If you have overridden ENGINE and/or NAME such that it does not point to a pre-existing database, an exception will be thrown because the test database has yet to be created:

django.db.utils.DatabaseError: no such table: yourapp_foo  # Django 1.4
DatabaseError: no such table: yourapp_foo                  # Django 1.5
OperationalError: no such table: yourapp_foo               # Django 1.6+

Instead do:

class GoodFooTests(TestCase):

    def setUp(self):
        f = Foo.objects.create()   # <-- will run against the test DB
        f.save()                   #

    def test_foo(self):
        # assert stuff

So, if you are seeing errors, check to see that your tests do not include any queries that might hit the database outside of your test class method definitions.

---

[1] In Django >= 1.7, DATABASES[alias]['TEST_NAME'] is deprecated in favour of DATABASES[alias]['TEST']['NAME']
[2] See the create_test_db() method in db/backends/creation.py

Solution 2

Having tried all of the above I eventually discovered another reason this can happen:-

If any of your models are not created by one of your migrations.

I did some debugging and it seems that Django testing sets up the database by applying all your migrations in order, starting with 001_initial.py, before trying to SELECT from the tables based on your models.py

In my case a table had somehow not been added to the migrations, but added manually, so the full migration set couldn't be properly applied. When I manually fixed the 001_initial.py migration to create this table the OperationalError went away.

Solution 3

I had this problem, too. Turned out that I had to add a TEST_NAME property in the settings.py file to identify the test database properly. It solved the problem for me:

if 'test' in sys.argv:
    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.sqlite3',
            'NAME': os.path.join(os.path.dirname(__file__), 'test.db'),
            'TEST_NAME': os.path.join(os.path.dirname(__file__), 'test.db'),
       }
    }

Solution 4

Just to add another case to this:

If you are trying to upgrade to 1.8 from 1.6 (or from a non-migration setup to a migration setup), you might hit this error if you haven't run created migrations.

I had the same problem and had to create migrations so the test runner could use them, which was not intuitive because pre-migrations, the tests would just make a new DB based on doing syncdb, which always worked.

Solution 5

For future reference, this also happens if your application is not added to your INSTALLED_APPS, for example:

INSTALLED_APPS = (
   ...
   'myapp'
)

Otherwise you get;

OperationalError: no such table: myapp_mytable

View more solutions

Comments

[Cerin]

I doubt this will work. Django destroys the test database after each run, so if you're correct, I'd need to do a syncdb prior to each run...

[Philipp Zedler]

It DOES work. The test-Sqlite database is in-memory by default, so there are no files to destroy after a test. The file you create with syncdb acts merely as a template for the in-memory test database. A syncdb is only necessary when the database structure changes.

[Cerin]

This is very close to my actual solution.

[Sergey Orshanskiy]

That was very helpful! If TEST_NAME is specified, Django puts the test database in a file instead of keeping it in memory. Thus it was possible for me to suspend the testing process and open the file from another terminal with sqlite3 test.db. I don't think that NAME is useful at all for a testing configuration; TEST_NAME is probably all we need.

[Sergey Orshanskiy]

In fact, I conclude that my tests work with an on-disk database but do not work with an in-memory database.

[Cerin]

I recently ran into this problem again, now on Django 1.5, and this solution doesn't work. I just get the error "no such table myapp_mymodel".

[Tom]

This does not work for 1.7 (and probably not for >= 1.5)

[Cerin]

Just upgraded to 1.7, and now this error happens for django.contrib.sites, and the solution you provide does not work.

[Vaibhav Mishra]

In my case, since we were not using in-built migrations feature, just removing migrations folder from app made it work

[Bobort]

A most ridiculous construct! if True will always execute, so why waste your time checking it at all?

[user5359531]

I had a similar issue, this was especially confusing because the "missing" items were all in my models.py, so it was really baffling that they would not be present in the test_db. Turned out that I had forgotten to commit some of the migrations from my dev instance when I made the changes.