How do I get around type erasure on Scala? Or, why can't I get the type parameter of my collections?
Solution 1
This answer uses the
Manifest
-API, which is deprecated as of Scala 2.10. Please see answers below for more current solutions.
Scala was defined with Type Erasure because the Java Virtual Machine (JVM), unlike Java, did not get generics. This means that, at run time, only the class exists, not its type parameters. In the example, JVM knows it is handling a scala.collection.immutable.List
, but not that this list is parameterized with Int
.
Fortunately, there's a feature in Scala that lets you get around that. It’s the Manifest. A Manifest is class whose instances are objects representing types. Since these instances are objects, you can pass them around, store them, and generally call methods on them. With the support of implicit parameters, it becomes a very powerful tool. Take the following example, for instance:
object Registry {
import scala.reflect.Manifest
private var map= Map.empty[Any,(Manifest[_], Any)]
def register[T](name: Any, item: T)(implicit m: Manifest[T]) {
map = map.updated(name, m -> item)
}
def get[T](key:Any)(implicit m : Manifest[T]): Option[T] = {
map get key flatMap {
case (om, s) => if (om <:< m) Some(s.asInstanceOf[T]) else None
}
}
}
scala> Registry.register("a", List(1,2,3))
scala> Registry.get[List[Int]]("a")
res6: Option[List[Int]] = Some(List(1, 2, 3))
scala> Registry.get[List[String]]("a")
res7: Option[List[String]] = None
When storing an element, we store a "Manifest" of it too. A Manifest is a class whose instances represent Scala types. These objects have more information than JVM does, which enable us to test for the full, parameterized type.
Note, however, that a Manifest
is still an evolving feature. As an example of its limitations, it presently doesn't know anything about variance, and assumes everything is co-variant. I expect it will get more stable and solid once the Scala reflection library, presently under development, gets finished.
Solution 2
You can do this using TypeTags (as Daniel already mentions, but I'll just spell it out explicitly):
import scala.reflect.runtime.universe._
def matchList[A: TypeTag](list: List[A]) = list match {
case strlist: List[String @unchecked] if typeOf[A] =:= typeOf[String] => println("A list of strings!")
case intlist: List[Int @unchecked] if typeOf[A] =:= typeOf[Int] => println("A list of ints!")
}
You can also do this using ClassTags (which saves you from having to depend on scala-reflect):
import scala.reflect.{ClassTag, classTag}
def matchList2[A : ClassTag](list: List[A]) = list match {
case strlist: List[String @unchecked] if classTag[A] == classTag[String] => println("A List of strings!")
case intlist: List[Int @unchecked] if classTag[A] == classTag[Int] => println("A list of ints!")
}
ClassTags can be used so long as you don't expect the type parameter A
to itself be a generic type.
Unfortunately it's a little verbose and you need the @unchecked annotation to suppress a compiler warning. The TypeTag may be incorporated into the pattern match automatically by the compiler in the future: https://issues.scala-lang.org/browse/SI-6517
Solution 3
You can use the Typeable
type class from shapeless to get the result you're after,
Sample REPL session,
scala> import shapeless.syntax.typeable._
import shapeless.syntax.typeable._
scala> val l1 : Any = List(1,2,3)
l1: Any = List(1, 2, 3)
scala> l1.cast[List[String]]
res0: Option[List[String]] = None
scala> l1.cast[List[Int]]
res1: Option[List[Int]] = Some(List(1, 2, 3))
The cast
operation will be as precise wrt erasure as possible given the in-scope Typeable
instances available.
Solution 4
I came up with a relatively simple solution that would suffice in limited-use situations, essentially wrapping parameterized types that would suffer from the type erasure problem in wrapper classes that can be used in a match statement.
case class StringListHolder(list:List[String])
StringListHolder(List("str1","str2")) match {
case holder: StringListHolder => holder.list foreach println
}
This has the expected output and limits the contents of our case class to the desired type, String Lists.
More details here: http://www.scalafied.com/?p=60
Solution 5
There is a way to overcome the type erasure issue in Scala. In Overcoming Type Erasure in matching 1 and Overcoming Type Erasure in Matching 2 (Variance) are some explanation of how to code some helpers to wrap the types, including Variance, for matching.
Daniel C. Sobral
I have been programming for more than 20 years now, starting with 8 bits computers, assembler and BASIC. My passion for languages meant that, by the time I entered college, I had already programmed for fun or profit in more than 20 languages, including odd ones like Forth, MUMPS and APL, as well as theoretically important ones like Lisp and Prolog. Some of my code ended up in FreeBSD, of which I was a committer for some years, while I got my masters degree in the field of distributed algorithms. I also contributed to Scala, with small amounts of code, some reasonable amount of documentation, and a couple of years of a lot of attention to the Scala questions on Stack Overflow.
Updated on July 08, 2022Comments
-
Daniel C. Sobral almost 2 years
It's a sad fact of life on Scala that if you instantiate a List[Int], you can verify that your instance is a List, and you can verify that any individual element of it is an Int, but not that it is a List[Int], as can be easily verified:
scala> List(1,2,3) match { | case l : List[String] => println("A list of strings?!") | case _ => println("Ok") | } warning: there were unchecked warnings; re-run with -unchecked for details A list of strings?!
The -unchecked option puts the blame squarely on type erasure:
scala> List(1,2,3) match { | case l : List[String] => println("A list of strings?!") | case _ => println("Ok") | } <console>:6: warning: non variable type-argument String in type pattern is unchecked since it is eliminated by erasure case l : List[String] => println("A list of strings?!") ^ A list of strings?!
Why is that, and how do I get around it?
-
Tristan Juricek almost 15 yearsI agree that answering these kinds of questions is a good idea : I had read about this somewhere before, but it is much easier to find on stack overflow.
-
Aaron Novstrup over 13 yearsThe
get
method can be defined asfor ((om, v) <- _map get key if om <:< m) yield v.asInstanceOf[T]
. -
Daniel C. Sobral over 13 years@Aaron Very good suggestion, but I fear it might obscure the code for people relatively new to Scala. I wasn't very experience with Scala myself when I wrote that code, which was sometime before I put it in this question/answer.
-
sullivan- almost 13 yearsDoesn't work with List(1, "a", "b"), which has type List[Any]
-
Daniel C. Sobral over 12 years@MatthieuF Good question. It is no longer experimental, but it isn't complete yet either. Right now Odersky himself is working on a Scala reflection library. I expect manifests will evolve some out of that effort. I edited the answer to reflect (pun intended) the present status.
-
Kim Stebel over 11 yearsLooking forward to a new
TypeTag
answer ;) -
Daniel C. Sobral over 11 years@KimStebel You know that
TypeTag
are actually automatically used on pattern matching? Cool, eh? -
Kim Stebel over 11 yearsCool! Maybe you should add that to the answer.
-
KajMagnus over 11 yearsWhen you call
Registry.register(...)
, you neither pass in theimplicit m: Manifest
, nor creates anyimplicit
local variable. — I take it that the compiler generates and "appends" that additional parameter? -
KajMagnus over 11 yearsTo answer my own question just above: Yes, the compiler generates the
Manifest
param itself, see: stackoverflow.com/a/11495793/694469 "the [manifest/type-tag] instance [...] is being created implicitly by the compiler" -
Seth over 11 yearsAlthough sullivan's point is correct and there are related problems with inheritance, I still found this useful.
-
Nick over 11 yearsThis is supposed to be easier in Scala 2.10. I would like to see a comparison of both 2.10 and < 2.10.
-
Carl G about 11 years"limitation of the otherwise awesome language" it's less a limitation of Scala and more a limitation of the JVM. Perhaps Scala could have been designed to include type information as it ran on the JVM, but I don't think a design like that would have preserved interoperability with Java (i.e., as designed, you can call Scala from Java.)
-
Carl G about 11 yearsAs a followup, support for reified generics for Scala in .NET/CLR is an ongoing possibility.
-
user48956 over 10 yearsThis doesn't overcome type erasure. In his example, doing val x:Any = List(1,2,3); x match { case IntList(l) => println( s"Match ${l(1)}" ); case _ => println( s"No match" ) } produces "No match"
-
Nader Ghanbari over 9 yearsWhat about removing unnecessary
[List String @unchecked]
as it does not add anything to this pattern match (Just usingcase strlist if typeOf[A] =:= typeOf[String] =>
will do it, or evencase _ if typeOf[A] =:= typeOf[String] =>
if the bound variable is not needed in body of thecase
). -
tksfz over 9 yearsI guess that would work for the given example but I think most real usages would benefit from having the type of the elements.
-
Toby over 8 yearsIn the examples above, doesn't the unchecked part in front of the guard condition do a cast? Wouldn't you get a class cast exception when going through the matches on the first object that cant' be cast to a string?
-
tksfz over 8 yearsHm no I believe there is no cast before applying the guard - the unchecked bit is sort of a no-op until the code to the right of the
=>
is executed. (And when the code on the rhs is executed, the guards provide a static guarantee on the type of the elements. There might be a cast there, but it's safe.) -
stanislav.chetvertkov over 8 yearsDoes this solution produce significant runtime overhead?
-
tksfz over 8 yearsI'm going to venture the short answer is "no" (the long answer is probably "it depends" on what you mean by significant runtime overhead). Notably there is no reflection (which would probably count as significant runtime overhead). The classtag's are immediately provided by the compiler. So there's just the
instanceof
check forList
(which happens to be redundant in this case) and the classTag equality check, which I imagine is fast. -
Dominique Unruh over 7 yearsIt should be noted that the "cast" operation will recursively go through the whole collection and its subcollections and check whether all involved value are of the right type. (I.e.,
l1.cast[List[String]]
does roughlyfor (x<-l1) assert(x.isInstanceOf[String]
) For large datastructures or if the casts happen very often, this may be an inacceptable overhead. -
Dominique Unruh over 7 yearsThe reason why this one will not work is that
isInstanceOf
does a runtime check based on the type information available to the JVM. And that runtime information will not contain the type argument toList
(because of type erasure). -
Andrew Norman almost 7 yearsthe problem I see here is that if I want to call this definition with a List of an unknown type (aka List[_]) I get an "No ClassTag available for _" error.
-
tksfz almost 7 years@AndrewNorman Right. The OP asks why we can't write code that knows the type of
A
inList[A]
when we clearly know its type at compile-time. ForList[_]
we're in a different situation: We don't know the type at compile-time. In that case, the best you can do is get an element, if one exists, and then do a runtime type check usingisInstanceOf
or pattern matchcase x: String => ... case y: Int => ...
etc