Deploying a Git subdirectory in Capistrano

Question

My master branch layout is like this:

/ <-- top level

/client <-- desktop client source files

/server <-- Rails app

What I'd like to do is only pull down the /server directory in my deploy.rb, but I can't seem to find any way to do that. The /client directory is huge, so setting up a hook to copy /server to / won't work very well, it needs to only pull down the Rails app.

Solution 1

Without any dirty forking action but even dirtier !

In my config/deploy.rb :

set :deploy_subdir, "project/subdir"

Then I added this new strategy to my Capfile :

require 'capistrano/recipes/deploy/strategy/remote_cache'

class RemoteCacheSubdir < Capistrano::Deploy::Strategy::RemoteCache

  private

  def repository_cache_subdir
    if configuration[:deploy_subdir] then
      File.join(repository_cache, configuration[:deploy_subdir])
    else
      repository_cache
    end
  end

  def copy_repository_cache
    logger.trace "copying the cached version to #{configuration[:release_path]}"
    if copy_exclude.empty? 
      run "cp -RPp #{repository_cache_subdir} #{configuration[:release_path]} && #{mark}"
    else
      exclusions = copy_exclude.map { |e| "--exclude=\"#{e}\"" }.join(' ')
      run "rsync -lrpt #{exclusions} #{repository_cache_subdir}/* #{configuration[:release_path]} && #{mark}"
    end
  end

end


set :strategy, RemoteCacheSubdir.new(self)

Solution 2

For Capistrano 3.0, I use the following:

In my Capfile:

# Define a new SCM strategy, so we can deploy only a subdirectory of our repo.
module RemoteCacheWithProjectRootStrategy
  def test
    test! " [ -f #{repo_path}/HEAD ] "
  end

  def check
    test! :git, :'ls-remote', repo_url
  end

  def clone
    git :clone, '--mirror', repo_url, repo_path
  end

  def update
    git :remote, :update
  end

  def release
    git :archive, fetch(:branch), fetch(:project_root), '| tar -x -C', release_path, "--strip=#{fetch(:project_root).count('/')+1}"
  end
end

And in my deploy.rb:

# Set up a strategy to deploy only a project directory (not the whole repo)
set :git_strategy, RemoteCacheWithProjectRootStrategy
set :project_root, 'relative/path/from/your/repo'

All the important code is in the strategy release method, which uses git archive to archive only a subdirectory of the repo, then uses the --strip argument to tar to extract the archive at the right level.

UPDATE

As of Capistrano 3.3.3, you can now use the :repo_tree configuration variable, which makes this answer obsolete. For example:

set :repo_url, 'https://example.com/your_repo.git'
set :repo_tree, 'relative/path/from/your/repo' # relative path to project root in repo

See http://capistranorb.com/documentation/getting-started/configuration.

Solution 3

We're also doing this with Capistrano by cloning down the full repository, deleting the unused files and folders and move the desired folder up the hierarchy.

deploy.rb

set :repository,  "[email protected]:name/project.git"
set :branch, "master"
set :subdir, "server"

after "deploy:update_code", "deploy:checkout_subdir"

namespace :deploy do

    desc "Checkout subdirectory and delete all the other stuff"
    task :checkout_subdir do
        run "mv #{current_release}/#{subdir}/ /tmp && rm -rf #{current_release}/* && mv /tmp/#{subdir}/* #{current_release}"
    end

end

As long as the project doesn't get too big this works pretty good for us, but if you can, create an own repository for each component and group them together with git submodules.

Solution 4

You can have two git repositories (client and server) and add them to a "super-project" (app). In this "super-project" you can add the two repositories as submodules (check this tutorial).

Another possible solution (a bit more dirty) is to have separate branches for client and server, and then you can pull from the 'server' branch.

Solution 5

For Capistrano 3, based on @Thomas Fankhauser answer:

set :repository,  "[email protected]:name/project.git"
set :branch, "master"
set :subdir, "relative_path_to_my/subdir"


namespace :deploy do

  desc "Checkout subdirectory and delete all the other stuff"
  task :checkout_subdir do

    subdir = fetch(:subdir)
    subdir_last_folder  = File.basename(subdir)
    release_subdir_path = File.join(release_path, subdir)

    tmp_base_folder = File.join("/tmp", "capistrano_subdir_hack")
    tmp_destination = File.join(tmp_base_folder, subdir_last_folder)

    cmd = []
    # Settings for my-zsh
    # cmd << "unsetopt nomatch && setopt rmstarsilent" 
    # create temporary folder
    cmd << "mkdir -p #{tmp_base_folder}"  
    # delete previous temporary files                
    cmd << "rm -rf #{tmp_base_folder}/*"  
    # move subdir contents to tmp           
    cmd << "mv #{release_subdir_path}/ #{tmp_destination}"   
    # delete contents inside release      
    cmd << "rm -rf #{release_path}/*"   
    # move subdir contents to release             
    cmd << "mv #{tmp_destination}/* #{release_path}" 
    cmd = cmd.join(" && ")

    on roles(:app) do
      within release_path do
        execute cmd
      end
    end
  end

end

after "deploy:updating", "deploy:checkout_subdir"

View more solutions

Comments

[Nazar]

Oh how I wish I could send you a few pints of cooold beer. Thank you!!

[Matt]

Perfect. Just what I needed. Thanks!

[Michiel de Mare]

Nice! A bit inefficient, but not an ugly hack at least.

[Michiel de Mare]

Lighthouse link is broken. I wonder if capistrano implemented this in the meantime.

[Thomas Fankhauser]

And don't forget to manually symlink the log directory as capistrano doesn't do that automatically anymore..

[Kevin]

For a more efficient approach. There is a gem someone created that adds a deploy strategy "copy_subdir". Only the subdirectory in the repo is archived and copied to the remote server. github.com/yyuu/capistrano-copy-subdir

[José Santos]

NB. anyone reading, this works if you're already using remote_cache as your :deploy_via mechanism (which relies upon SCM access at the server end.)

[M. Scott Ford]

This looks nice! Has anyone baked it into a gem?

[M. Scott Ford]

This looks like it might have potential... github.com/mcollina/capistrano-remote-cache-with-project-roo‌​t

[leojh]

I could not set the custom strategy using 'set :git_strategy,' it kept using the DefaultStrategy

[Tsuneo Yoshioka]

I also needed to copy and paste "fetch_revision" method from original.(github.com/capistrano/capistrano/blob/master/lib/c‌​apistrano/…)

[vcardillo]

See Mr Friendly's comment buried below for Capistrano 3.

[sarat]

any idea how we do this role based?

[Mr Friendly]

@TsuneoYoshioka Yes, the "fetch_revision" method was added in Capistrano 3.1. However, 3.1 also added the 'repo_tree' configuration variable, which (happily) makes this answer obsolete. See github.com/capistrano/capistrano#configuration for details.

[Altonymous]

:repo_tree was actually added in 3.3.3 not 3.1. For those that see this and it doesn't work for them.

[chetang]

Hi, I am using capistrano 3.4.0 but I am still not able to use :repo_tree variable to make it work. I have a git repo called demo_app which has two sub-dir server and client. I want to use server sub-dir for deployment. How do I set that using :repo_tree? None of these worked:- set :repo_tree, 'demo_app/server' or set :repo_tree, 'server'

[Jorge Orpinel Pérez]

Accidentally scrolled all the way down and found this. Works for me, +1.

[Sjors Branderhorst]

mentioning the 'git way' does not help anything or anyone.

[Admin]

it works fine for me too... thanks +1 I can now deploy api versions v1, v2 , ...

[leandroico]

Thanks for sharing this great solution. I just wonder if this is the right Capistrano way to extend & couple things inside of it. What about the install_plugin method?

[Anna Völkl]

Thanks for the hint regarding set :repo_tree, 'relative/path/from/your/repo' in Capistrano > 3.3.3 :)

[i_use_the_internet]

I keep getting a load error: require 'capistrano/recipes/deploy/strategy/remote_cache' file not found. I am using Capistrano 3