How can sbt pull dependency artifacts from git?

Question

I've heard (and I know I've seen examples too, if only I can remember where) that sbt can obtain dependencies from a git repo.

I am looking to obtain the dependency harrah/up from github. The repository does not provide any artifact JAR files, only a source tree which is set up to be built using sbt. The process that I am imagining is that sbt will download the source repo, build it, and then use that as the dependency artifact.

I may be imagining that sbt can in fact do something like this. Can it? And if so, how?

Solution 1

Yes indeed. You can give your Project a dependency with the dependsOn operator, and you can reference a Github project by its URI, for example RootProject(uri("git://github.com/dragos/dupcheck.git")). Alternatively, you can git clone the project, and then reference your local copy with RootProject(file(...)). See "Full Configuration" on the SBT wiki for details and examples.

Solution 2

You can import unpackaged dependencies into your project from GitHub by treating them as project dependencies, using the dependsOn operator. (This is distinct from the way that precompiled library dependencies are included).

Note that you can specify which branch to pull using # notation. Here's some Scala SBT code that is working well for me:

object V {
  val depProject = "master"
  // Other library versions
}

object Projects {
  lazy val depProject = RootProject(uri("git://github.com/me/dep-project.git#%s".format(V.depProject)))
}

// Library dependencies
lazy val myProject = Project("my-project", file("."))
.settings(myProjectSettings: _*)
.dependsOn(Projects.depProject)
.settings(
  libraryDependencies ++= Seq(...

Note that if you have multiple SBT projects dependending on the same external project, it's worth setting up a central sbt.boot.directory to avoid unnecessary recompilations (see instructions here).

Solution 3

Since I had problems getting the dependencies of my library resolved (using the suggested RootProject) I'd like to point out to the object called ProjectRef. Thus, if one need to depend on a library residing in git, I suggest to do so as follows:

import sbt.Keys._
import sbt._

object MyBuild extends Build {

  lazy val root = Project("root", file("."))
    .dependsOn(myLibraryinGit)
    .settings(
      ...,
      libraryDependencies ++= Seq(...))

  lazy val myLibraryinGit = ProjectRef(uri("git://[email protected]:user/repo.git#branch"), "repo-name")

}

Source: http://blog.xebia.com/git-subproject-compile-time-dependencies-in-sbt/

Solution 4

I wanted to add an answer for sbt 0.13+. Just put something like this to your build.sbt on project root folder (not Build.scala):

lazy val root = (project in file(".")).dependsOn(playJongo)

lazy val playJongo = RootProject(uri("https://github.com/bekce/play-jongo.git"))

View more solutions

Comments

[Owen]

Do you know if it is possible to include this in build.sbt?

[Kipton Barros]

@Owen, I'm not sure, but I suspect not. Doesn't build.sbt need to contain newline separated Settings? Someone correct me if I'm wrong.

[Owen]

Yes; that's correct (though I didn't know what when I asked). So the question reduces to whether there is a Setting for that -- it's possible that projectDependencies is such a setting; I'm not sure.

[retronym]

You do need a .scala build for this.

[axel22]

Great insight. But if that project changes, how do you force sbt to recompile it? clean does not seem to help.

[Arne]

can sbt also keep the local git updated with "git pull"? Or do I need to do that manually?

[Danyel]

Is it possible to set "my-project" dynamically, using the name := "MyProject" setting from build.sbt in project root?

[Sebastian Graf]

git:// protocol syntax doesn't seem to work for me in play/sbt 13.0. https:// works though. This might also be caused by a git specific setting.

[Jimmy Luong]

Hi Axel22, did you find any solution to your problem?

[Martin Tapp]

Using sbt 0.13.9 and sbt updates the git dependency correctly using a shallow clone (git clone --depth 1).

[acjay]

How can you configure this in situations where the git repo is private? Is there a way to configure SBT to send a deploy key, which could be set in a dotfile or environment variable?

[Neil]

The branch only works if set to master, otherwise I get: [error] No project 'project_name' in 'ssh://[email protected]/user/project.git#branch'

[Waldemar Wosiński]

In case of git clone error try with https uri version.

[lisak]

I'd also like to know how to make sbt pull changes after pushing new commit to the github repository, if I point it to #master, reload, it clones it, then I push new commit to master and sbt still works with the originally cloned repository without noticing a new commit. At this point one have to either delete it from /.sbt/0.13/staging/ and have sbt clone it again or going there and doing git pull

[John Smith]

@Neil you've referenced the subproject as project_name in your top level project but in the subproject build.sbt you've likely referenced it as something else. The val name determines the project name.