How to build spark application using Scala IDE and Maven?

Question

I'm new to Scala, Spark and Maven and would like to build spark application described here. It uses the Mahout library.

I have Scala IDE install and would like to use Maven to build the dependencies (which are the Mahout library as well as Spark lib). I couldn't find a good tutorial to start. Could someone help me figure it out?

Solution

First try compiling simple application with Maven in Scala IDE. The key of Maven project is directory structure and pom.xml. Although I don't use Scala IDE, this document seems helpful. http://scala-ide.org/docs/tutorials/m2eclipse/

Next step is to add dependency on Spark in pom.xml you can follow this document. http://blog.cloudera.com/blog/2014/04/how-to-run-a-simple-apache-spark-app-in-cdh-5/

For latest version of Spark and Mahout artifacts you can check them here: http://mvnrepository.com/artifact/org.apache.spark http://mvnrepository.com/artifact/org.apache.mahout

Hope this helps.