(run-main-0) java.lang.NoSuchMethodError
Solution 1
I met with the same error when I used scala-library-2.11 jar But when I replaced it with scala-library-2.10 jar . It runs fine
Solution 2
It is probably caused by using incompatible versions of Scala. When I downgraded from Scala 2.11 to 2.10, I forgot to modify one package version (so one package used 2.11, the rest 2.10), resulting in having the same error.
Note: I only had this problem when using IntelliJ.
Solution 3
If you are getting the error and here because you cannot run Jupiter notebooks with Spark 2.1 and Scala 2.11 below is how I was able to make it work. Assumes you installed Jupiter and toree
Pre-req - Make sure Docker is running else Make fails. Make sure gpg is installed else Make fails.
Build steps -
export SPARK_HOME=/Users/<path>/spark-2.1.0-hadoop2.7/
git clone https://github.com/apache/incubator-toree.git
cd incubator-toree
make clean release APACHE_SPARK_VERSION=2.1.0
pip install --upgrade ./dist/toree-pip/toree-0.2.0.dev1.tar.gz
pip freeze |grep toree
jupyter toree install --spark_home=$SPARK_HOME
========================================================================
To Start the notebook - SPARK_OPTS='--master=local[4]' jupyter notebook
Solution 4
I used these versions and everything works now.
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.11.6</version>
</dependency>
<dependency>
<groupId>com.typesafe.akka</groupId>
<artifactId>akka-actor_2.11</artifactId>
<version>2.3.11</version>
</dependency>
Solution 5
Check whether the scala version you are using corresponds to the precompiled spark version.
Related videos on Youtube
zhang
Updated on July 09, 2022Comments
-
zhang almost 2 years
I got a problem when I used sbt to run a spark job, I have finish compile, but when I run the command
run
, I got the problem below[error] (run-main-0) java.lang.NoSuchMethodError: scala.collection.immutable.HashSet$.empty()Lscala/collection/immutable/HashSet; java.lang.NoSuchMethodError: scala.collection.immutable.HashSet$.empty()Lscala/collection/immutable/HashSet; at akka.actor.ActorCell$.<init>(ActorCell.scala:305) at akka.actor.ActorCell$.<clinit>(ActorCell.scala) at akka.actor.RootActorPath.$div(ActorPath.scala:152) at akka.actor.LocalActorRefProvider.<init>(ActorRefProvider.scala:465) at akka.remote.RemoteActorRefProvider.<init>(RemoteActorRefProvider.scala:124) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:422) at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$2.apply(DynamicAccess.scala:78) at scala.util.Try$.apply(Try.scala:191)
Anyone knows what should I do?
-
Justin Pihony about 9 yearsWhats your version of spark and scala? My initial guess would be that the scala versions are mismatching.
-
zhang about 9 years@JustinPihony Thanks for you help, my scala version is 2.10.5 and spark version is 1.3.0,can you tell me how to judge whether the scala match or not?
-
-
ostrokach almost 7 yearsLink to Jira issue: Toree not working with Apache Spark 2.0.0