Should I be adding the Django migration files in the .gitignore file?

Question

Should I be adding the Django migration files in the .gitignore file?

I've recently been getting a lot of git issues due to migration conflicts and was wondering if I should be marking migration files as ignore.

If so, how would I go about adding all of the migrations that I have in my apps, and adding them to the .gitignore file?

Solution 1

Quoting from the Django migrations documentation:

The migration files for each app live in a “migrations” directory inside of that app, and are designed to be committed to, and distributed as part of, its codebase. You should be making them once on your development machine and then running the same migrations on your colleagues’ machines, your staging machines, and eventually your production machines.

If you follow this process, you shouldn't be getting any merge conflicts in the migration files.

When merging version control branches, you still may encounter a situation where you have multiple migrations based on the same parent migration, e.g. if to different developers introduced a migration concurrently. One way of resolving this situation is to introduce a merge_migration. Often this can be done automatically with the command

./manage.py makemigrations --merge

which will introduce a new migration that depends on all current head migrations. Of course this only works when there is no conflict between the head migrations, in which case you will have to resolve the problem manually.

---

Given that some people here suggested that you shouldn't commit your migrations to version control, I'd like to expand on the reasons why you actually should do so.

First, you need a record of the migrations applied to your production systems. If you deploy changes to production and want to migrate the database, you need a description of the current state. You can create a separate backup of the migrations applied to each production database, but this seems unnecessarily cumbersome.

Second, migrations often contain custom, handwritten code. It's not always possible to automatically generate them with ./manage.py makemigrations.

Third, migrations should be included in code review. They are significant changes to your production system, and there are lots of things that can go wrong with them.

So in short, if you care about your production data, please check your migrations into version control.

Solution 2

You can follow the below process.

You can run makemigrations locally and this creates the migration file. Commit this new migration file to repo.

In my opinion you should not run makemigrations in production at all. You can run migrate in production and you will see the migrations are applied from the migration file that you committed from local. This way you can avoid all conflicts.

IN LOCAL ENV, to create the migration files,

python manage.py makemigrations 
python manage.py migrate

Now commit these newly created files, something like below.

git add app/migrations/...
git commit -m 'add migration files' app/migrations/...

---

IN PRODUCTION ENV, run only the below command.

python manage.py migrate

Solution 3

Quote from the 2022 docs, Django 4.0. (two separate commands = makemigrations and migrate)

The reason that there are separate commands to make and apply migrations is because you’ll commit migrations to your version control system and ship them with your app; they not only make your development easier, they’re also useable by other developers and in production.

https://docs.djangoproject.com/en/4.0/intro/tutorial02/

Solution 4

TL;DR: commit migrations, resolve migration conflicts, adjust your git workflow.

Feels like you'd need to adjust your git workflow, instead of ignoring conflicts.

Ideally, every new feature is developed in a different branch, and merged back with a pull request.

PRs cannot be merged if there's a conflict, therefore who needs to merge his feature needs to resolve the conflict, migrations included. This might need coordination between different teams.

It is important though to commit migration files! If a conflict arises, Django might even help you solve those conflicts ;)

Solution 5

The solution usually used, is that, before anything is merged into master, the developer must pull any remote changes. If there's a conflict in migration versions, he should rename his local migration (the remote one has been run by other devs, and, potentially, in production), to N+1.

During development it might be okay to just not-commit migrations (don't add an ignore though, just don't add them). But once you've gone into production, you'll need them in order to keep the schema in sync with model changes.

You then need to edit the file, and change the dependencies to the latest remote version.

This works for Django migrations, as well as other similar apps (sqlalchemy+alembic, RoR, etc).

View more solutions

Comments

[Randy Tang]

We, a team of developers, have exactly the same problem, too. If one member performs makemigrations some_app, not only the models under that member's control will be affected, but also other related models will be affected, too. That is, the migrations files (00*_*) in other apps will be changed. And that causes many conflict problems during pushing to or pulling from GitHub. Since currently our system is not ready for production, we just .gitignore the migration file. We still don't know how to solve it when the system goes production. Does anybody has any solutions?

[lvthillo]

So if I good understand you have to pull the migrations from your project. When you change something you have to do a local makemigrations. Push it again and the buildserver will do the migrate? (Very important ofcourse everyone has the good versions)

[gurel_kaynak]

we never use django migrations. everyone connects to a central test database, if a change is needed, it is done manually at the db level when needed. This method works good if the system is mature enough when it won't need many database schema updates.

[Daniel Dror]

@yltang52 We are currently trying to figure it out as well, can you share any insights? I think once you go to production you have no choice but to maintain those migrations to allow for easier patches and overall easier version control. I also wonder what you do with Initial state data (like settings stored in db). How do you maintain them?

[Diansheng]

I don't think you should put migrations files into repo. It will spoil the migration states in other person's dev environment and other prod and stage environment. (refer to Sugar Tang's comment for examples). Tracking models file is sufficient.

[Diansheng]

i down voted this answer. i explained in my comments.

[Sven Marnach]

@Sean Noted. What you are suggesting doesn't work. And do you really think an answer suggesting to follow the documented process is not helpful?

[Sven Marnach]

This will only work for toy projects, and has many downsides. It immediately stops working for hand-written migrations, for services using multiple ephemeral app servers (i.e. any serious application) and for apps consisting of many Django apps each with their own migrations. And I don't understand what you are referring to with "manually merging migrations" – manage.py makemigrations --merge works fully automatically for me.

[swdev]

In my opinion, migrations file should only be part of the repo once the app deployed. That counted as initial migrations. If the app still in heavy development, we can safely ignore it. But once it goes live. That's it! That's the sign that migrations should be put into repo. Every other members then need to follow this migrations and put theirs when needed

[polarise]

Very good point to only run migrate and NEVER makemigrations for committed migrations. Never thought of that.

[Oded]

I will add that sometime makemigrations doesn't work flawlessly. Case in point, I removed a model and got InnoDB errors trying to migrate from the auto makemigrations file. Instead, I had to first remove all fields from the model, makemigration, then remove the model, and makemigration again. So I'm leaning towards checking them in. Any better alternative?

[Eric]

That's the right answer. An operational versioning system workflow seems to be implicit in the django documentation but it's fundamental.

[Sdra]

@RandyTang, gurel_kaynak, Diansheng : do not ignore migrations!!! migration conflicts must be resolved, not ignored. Use a proper git flow instead. Read answer below plz. Also this answer is good, but merge conflicts can happen and must be resolved.

[Diansheng]

@Sven Marnach, I was indeed working on small but serious applications. And it works for me.

[andho]

@DanielDubovski this is very late, but you add those Initial state data like settings through a migration. It's also documented in providing initial data with migrations.

[andho]

Great point about starting over from scratch with 0001 for release.

[Beki]

don't know if this still holds true for Python versions above 3.8... anyone?

[Beki]

link does not work