How to git clone private repos within Vagrant VM, using host's SSH keys?

Question

I'm able to use PuppetLabs vcsrepo to clone public git repos, but I'd like to also be able to clone private repos, using the host's SSH keys.

What would the configuration look like for Vagrantfile and/or manifests/default.pp in order to accomplish this?

Solution 1

I can't help with the Puppet part, but you can forward the SSH Agent by setting:

Vagrant.configure("2") do |config|
  config.ssh.forward_agent = true
  # ...
end

This way the SSH connections (also make by git) try to use your private keys from the host.

Solution 2

Works on my machine!

Vagrantfile:

VAGRANTFILE_API_VERSION = '2'

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = 'precise64'
  config.vm.box_url = 'http://files.vagrantup.com/precise64.box'

  #
  # Use host authenticaton for git and maven.
  #
  # Ensure host private key is registered with host SSH agent:
  #
  # ssh-add -L
  # ssh-add ~/.ssh/id_rsa
  # ssh-add -L
  #

  config.ssh.private_key_path = ['~/.vagrant.d/insecure_private_key', '~/.ssh/id_rsa']
  config.ssh.forward_agent = true

  config.vm.synced_folder "~/.m2", "/home/vagrant/.m2"

  config.vm.provision :shell, path: 'upgrade-puppet.sh'

  # Install puppet modules
  config.vm.provision :shell, path: 'bootstrap.rb', args: %w(
    puppetlabs-stdlib
    puppetlabs/apt
    puppetlabs/vcsrepo
  )

  config.vm.provision :puppet do |puppet|
    puppet.options = ENV['PUPPET_OPTIONS']
  end
end

upgrade-puppet.sh:

#!/bin/bash

apt-get install --yes lsb-release > /dev/null
DISTRIB_CODENAME=$(lsb_release --codename --short)
DEB="puppetlabs-release-${DISTRIB_CODENAME}.deb"
DEB_PROVIDES="/etc/apt/sources.list.d/puppetlabs.list" # Assume that this file's existence means we have the Puppet Labs repo added

if [ ! -e $DEB_PROVIDES ]
then
    # Print statement useful for debugging, but automated runs of this will interpret any output as an error
    # print "Could not find $DEB_PROVIDES - fetching and installing $DEB"
    wget -q http://apt.puppetlabs.com/$DEB
    sudo dpkg -i $DEB
fi
sudo apt-get update > /dev/null
sudo apt-get install --yes puppet > /dev/null

mkdir -p /etc/puppet
touch /etc/puppet/hiera.yaml

bootstrap.sh:

#!/usr/bin/env ruby

modules_dir = '/etc/puppet/modules'

puts `mkdir -p #{modules_dir}` unless File::exists? modules_dir

mods = ARGV

installed = `puppet module list`.split "\n"

mods.each do |mod|
  puts `puppet module install #{mod}` unless installed.any? { |i| i.include?(mod.sub('/','-')) }
end

manifests/default.pp:

exec { 'ssh know github':
  command => 'ssh -Tv [email protected] -o StrictHostKeyChecking=no; echo Success',
  path    => '/bin:/usr/bin',
  user    => 'vagrant'
}

vcsrepo { '/home/vagrant/a-private-repo':
  ensure   => latest,
  provider => git,
  source   => '[email protected]:mcandre/a-private-repo.git',
  user     => 'vagrant',
  owner    => 'vagrant',
  group    => 'vagrant',
  require  => Exec['ssh know github']
}

Solution 3

I know you are using Puppet but I got it up and running using this bash script (provisioners/shell/application.setup.sh):

#!/bin/bash

local_user=vagrant

if [ ! -n "$(grep "^bitbucket.org " /home/$local_user/.ssh/known_hosts)" ]; then 
    ssh-keyscan bitbucket.org >> ~/.ssh/known_hosts 2>/dev/null;
fi

if [[ ! -d "/home/$local_user/app" ]]; then
    git clone [email protected]:czerasz/some-app.git /home/$local_user/app

    chown -R $local_user:$local_user /home/$local_user/app

    su - $local_user -c "source /usr/local/bin/virtualenvwrapper.sh && mkvirtualenv some-env && workon some-env && pip install -r /home/$local_user/app/requirements.txt"
fi

One could easily convert it to a puppet manifest...

Together with this Vagrantfile

config.vm.define "web1", primary: true do |web1_config|
    web1_config.ssh.forward_agent = true

    # Create a private network, which allows host-only access to the machine
    web1_config.vm.network "private_network", ip: "192.168.11.10"
    web1_config.vm.hostname = "web1.#{domain}"

    web1_config.vm.provision "shell", path: "provisioners/shell/python.setup.sh"
    web1_config.vm.provision "shell", path: "provisioners/shell/application.setup.sh"
end

The key point for me was that when I executed:

su - $local_user -c "ssh-keyscan bitbucket.org >> ~/.ssh/known_hosts 2>/dev/null;"
su - $local_user -c "git clone [email protected]:czerasz/some-app.git /home/$local_user/app"

It didn't work out. As if the keys were not passed using su. So I cloned the repo as root and then changed ownership afterwards.

This post was very helpful.

Comments

[Chase Sandmann]

There is an error that causes this to not work on Windows in most cases.

[mastazi]

@ChaseSandmann can you give more info about the error? Do you have a link to the github issue? I have found this one but I think it's not the one because it seems related to VirtualBox 5: github.com/mitchellh/vagrant/issues/6225

[Monkpit]

bootstrap.sh needs to be bootstrap.rb for this to work.

[Eric Hodonsky]

Mate..... I found a bunch of fairly unhelpful posts around this until yours. Cheers!!!