Capistrano deploy:migrate and db:migrate run all migrations every time

Question

So, I'm diddling around with rails (ruby 1.9.3p392, rails 3.2, sqlite3 db) and I'm trying to deploy the ubiquitous blog tutorial code to a "production" server (apache, passenger, ubuntu). My deploy.rb looks like this:

require 'bundler/capistrano'
require 'rvm/capistrano'
load 'deploy/assets'
set :rvm_ruby_string,  ENV['GEM_HOME'].gsub(/.*\//,"")
set :rvm_type, :user
set :user, 'blah'
set :application, 'railsTest'
set :domain, 'www.blah.com'
set :applicationdir, "/home/sean/public/blah.com/public"
set :scm, 'git'
set :repository,  "ssh://[email protected]/home/blah/public/bla.com/public/capDep.git"
#set :git_enable_submodules, 1 # if you have vendored rails
set :branch, 'master'
set :git_shallow_clone, 1
set :scm_verbose, true
set :use_sudo, false


# roles (servers)
role :web, domain
role :app, domain
role :db,  domain, :primary => true

# deploy config
set :deploy_to, applicationdir
set :deploy_via, :export
set :migrate_target, :latest
# additional settings
default_run_options[:pty] = true  # Forgo errors when deploying from windows
#ssh_options[:keys] = %w(/home/blah/.ssh/id_rsa)
ssh_options[:forward_agent] = true
# if you want to clean up old releases on each deploy uncomment this:

# If you are using Passenger mod_rails uncomment this:
namespace :deploy do
  task :start do ; end
  task :stop do ; end
  task :restart, :roles => :app, :except => { :no_release => true } do
    run "#{try_sudo} touch #{File.join(current_path,'tmp','restart.txt')}"
  end
end

#after "deploy:update_code", "deploy:migrate"

Now, I am sure that must look like a big hot mess to those who know what they are doing with capistrano, but I am an utter rube. In the end, despite my inadequacies, the deploy seems to work, because when I run the following

cap deploy:setup
cap deploy

my app is up and running and, just because I can, I add a few rows to a table in the db via the web ui that was created for me by rails. Now, I get bold and create a migration, adding a column to a table. I push my changes to git. To my horror, when I run

cap deploy

ALL the migrations are run, which recreates the tables, thus destroying all my data. I have repeated this painful process several times. My schema_migrations table looks like this:

20130620210004
20130620220229
20130628213331
20130628214946
20130628223002

What am I missing here?

UPDATE: I recently gave @TheMahrvin's suggestion regarding running deploy:migrations at the command line and removing it from the deploy.rb. It didn't work... once again, all migrations were run. My muse must have whispered something in my ear, because I decided to try running db:migrate on the server itself. I was astonished to see this output after running just "rake":

  20130717230110 CreateHighScores
  20130717230342 CreateGames
  20130717231041 AddGameTypeToGame
  20130717233707 AddGamePublisherToGame
  20130717234124 AddGameRatingToGame
  20130731210558 AddGameMechanicToGame

Only the last migrations should be pending. So, perhaps this isn't a problem with Capistrano at all (I've updated the title of this question to reflect that). So, why are the previous migrations still being flagged as pending? I know they were run in the past, both because I saw them in the output and verified the db schema after they ran.

UPDATE #2: Setup another migration and ssh'd into the server and cd'd my way to the "current" directory, which if I understand capistrano at all (fat chance) is where the current files are. Running

bundle exec rake db:migrate:status

got me:

 Status   Migration ID    Migration Name
--------------------------------------------------
  down    20130717230110  Create high scores
  down    20130717230342  Create games
  down    20130717231041  Add game type to game
  down    20130717233707  Add game publisher to game
  down    20130717234124  Add game rating to game
  down    20130731210558  Add game mechanic to game
  down    20130731212454  Add publish year to game
  down    20130731214515  Add game rank to game
  down    20130731214928  Add game abbr to game
  down    20130731215749  Add crazy field to game

I can't help feeling that there is something profoundly wrong with what I am trying to do.

Solution 1

OK, figured it out... though how anybody else in the stackosphere was supposed to do the same based on the red herrings in my original question is beyond me.

The problem was that my production database was set to

db/production.sqlite3

Because it was a sqlite database in the main project directory, it was getting axed every time I ran

cap deploy

Then, when I would run

cap deploy:migrate

It would find an empty database and think all migrations needed to be run. I solved this by changing the database path to

/my_absolute_path/shared/db/production.sqlite3

Thanks to @TheMahvin and anyone else who attempted to take on the hopeless task of answering my poorly worded question!

H/T to this question, which made the scales fall from my eyes:

Capistrano Deploy Wipes Database?

Solution 2

I haven't seen:

after "deploy:update_code", "deploy:migrate"

before. Try deleting that line and run:

 bundle exec cap deploy:migrations (deploys code and migrations)

or

 bundle exec cap deploy:migrate (runs the migrate rake task on the server)

instead. The rest of your deploy.rb seems OK to me, though I don't know anything about the rvm/capistrano integration or the windows adjustment.

Solution 3

How did you "add a few rows to a table in the db"?
I suspect your data loss results from mixing migrations and your own db changes. Rails expects you to do all the database changes via migrations.
There's some debate on migrations in general in the Rails community, but right now (especially if you're a beginner) always use migrations to change your database. That way you have a complete blueprint for your db allowing you to deploy on several machines from scratch without fiddling with your db and to make sure other contributors have the same db to work with.

I don't know a lot about these kinds of internals, but from what I understand your data loss resulted something like that:

After your manual changes Rails couldn't match the db-layout to the result of any migration (via the timestamps in your migrations and your schema respectively) thus treating the db as if it was new. To get to the state defined by all migrations, all of them needed to be executed, including the migrations that create tables, thus discarding everything in them.

I hope this helps,
Andy

Comments

[seanicus]

Thanks for the response... I thought this question had been consigned to languish in the outer darkness for eternity. I add the rows to the table in the db by using the web apps interface... that is to say, deploy, go to the site and navigate to the view for "create" for the controller, fill out the html form and hit submit. Then, rinse and repeat a couple of more times.

[Alex D]

No offense, but this doesn't really seem to make sense. Rails doesn't determine your migration level by comparing the DB schema to the result of migrations. It creates a table called schema_migrations, and stores the timestamp of every migration which has been run in it. When you run rake db:migrate, it compares the timestamps in schema_migrations with those of the migration files existing in db/migrate.

[A5308Y]

None taken. Actually I even think your comment fits with parts of my description. I was just being in the dark how the matching works in detail: The result of a migration is identifiable via a timestamp, as is the actual db-layout via the schema's timestamp. But yes, the discarding part does not really makes sense. I was just guessing. Should I delete the answer?