"Dead Letters encountered" error while running AKKA remote actors

Question

I am trying to run remote actors using AKKA, on my localhost, but each time I get this error. It says dead letters encountered. I searched on internet and found out that this error comes when actors receive a message after its thread is stopped. So I am looking for a way to keep the actors alive on remote machines. I am using akka actors and not the scala actors.

[INFO] [09/16/2013 18:44:51.426] [run-main] [Remoting] Starting remoting

[INFO] [09/16/2013 18:44:51.688] [run-main] [Remoting] Remoting started; listening on      addresses :[akka.tcp://actorSystem1@localhost:2209]

[INFO] [09/16/2013 18:44:51.759] [actorSystem2-akka.actor.default-dispatcher-5] [akka://actorSystem2/deadLetters] Message [java.lang.String] from 

Actor[akka://actorSystem2/deadLetters] to Actor[akka://actorSystem2/deadLetters] was not delivered. [1] **dead letters encountered**. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.   

Following is the code.

import akka.actor.{Actor, Props, ActorSystem}
import com.typesafe.config.ConfigFactory
import akka.remote._

object MyApp extends App {
 val actorSystem1 = ActorSystem("actorSystem1", ConfigFactory.parseString("""
    akka {
       actor {
           provider = "akka.remote.RemoteActorRefProvider"
             }
       remote {
           transport = ["akka.remote.netty.tcp"]
       netty.tcp {
           hostname = "localhost"
           port = 2209
                 }
             }
        }
   """))


  val actorSystem2 = ActorSystem("actorSystem2")


actorSystem1.actorOf(Props(new Actor {
def receive = {
  case x: String =>
    Thread.sleep(1000)
    println("RECEIVED MESSAGE: " + x)
               } }), "simplisticActor")



    val remoteActor = actorSystem2.actorFor("akka://actorSystem1@localhost:2209/user/simplisticActor")

  remoteActor ! "TEST 1"
  remoteActor ! "TEST 2"



  Thread.sleep(1000)

  actorSystem1.shutdown()
  actorSystem2.shutdown()
 }

Thanks in advance.

Solution

I think I see a few issues with your code that might be leading to deadletters. First, if your intention is to lookup an actor on a remote system from actorSystem2 into actorSystem1, then you will still need to set up remoting properties for actorSystem1, most specifically, you need to make sure it's using the RemoteActorRefProvider. If you don't do this, system 2 will not be able to lookup a remote actor in system 1. Once you make this change, I would also change your remote actor lookup from:

val remoteActor = actorSystem2.actorFor("akka://actorSystem1@localhost:2209/user/simplisticActor")

to:

val remoteActor = actorSystem2.actorSelection("akka.tcp://actorSystem1@localhost:2209/user/simplisticActor")

The actorFor method has been deprecated and also I think you left off the .tcp part of the akka protocol for looking up the remote actor.

Make these changes and then see if things work for you.

Comments

[Endre Varga]

Of course the above advice depends on the exact version of Akka used, which is not specified in the question (although from the logs it seems to be the 2.2.x series). Also, not all deadLetters are errors, they can happen under normal operation, and they are very likely to happen during shutdown.

[cmbaxter]

@EndreVarga, agreed that not all dead letters signify a problem, but in this specific case, there is an issue and one that can be pretty easily fixed.

[Admin]

I had the same problem due to missing the akka.actor.provider config for akka.remote.RemoteActorRefProvider. It has been fixed thanks to this answer.