Send typed objects through tcp or sockets

Question

I’m having trouble creating a network interface for a very simple game I’ve made in Xna. I would simply need to send objects through a TCP client / Socket. Ex: I have a class named “Player”. In every Player, there’s a field name “Info”, of type “PlayerInfo”. In the client / server, I would need to send every players’ info to every client except the one who sent it (obviously). This is a simple example, but I would need to do that with around 5 – 10 objects, plus sending the player updates (positions, actions …) Is there a simple way to do that with TCP / Sock? Note : I would rate my knowledge in C# and programming as 6/10, so you don’t need to explain everything if you have a solution (Ex : what’s the difference between a variable and a field). I also know about interfaces, libraries and so on… Thanks in advance!

Solution 1

I have one approach I would recommend and two lesser ones which are dependent on many things.

The first one implies you already know how to use the Socket class but have a lot of classes that you need to send through it.

From a transport point of view you should create / take into consideration just one very simple class. Let's call this class MyMessage:

public class MyMessage {
  public byte[] Data { get; set; }
}

Ok. From a TCP point of view all you need to do is make sure you're able to pass instances of this class around (from clients to the server and back). I will not dive into the details of doing that but I will point out that if you manage to do this you transform the nature of the TCP/IP connection from "byte-stream" to "message-stream". What that means is that normally, TCP/IP does not guarantee that chunks of data you send through a connection arrive at the destination in the same formations (they might get joined together or be split apart). The only thing it guarantees is that the bytes of all the chunks will eventually arrive in the same order at the other end of the connection (always).

Now that you have a message-stream up and running you could use .NET good old serialization to encapsulate any class instance inside the Data property. What that does is it serializes object graphs into bytes and vice-versa.

The way you do that (most commonly) is to use the standard library class: System.Runtime.Serialization.Formatters.Binary.BinaryFormatter which can be found in mscorlib.dll like so:

public static class Foo {

  public static Message Serialize(object anySerializableObject) {
    using (var memoryStream = new MemoryStream()) {
      (new BinaryFormatter()).Serialize(memoryStream, anySerializableObject);
      return new Message { Data = memoryStream.ToArray() };
    }
  }

  public static object Deserialize(Message message) {
    using (var memoryStream = new MemoryStream(message.Data))
      return (new BinaryFormatter()).Deserialize(memoryStream);
  }

}

The BinaryFormatter class is able to traverse the tree/graph of objects starting from the root / sentinel provided as the second argument of the Serialize (Stream, object) method and write all of the primitive values plus type information and relative position information to the provided stream. It is also able to do the exact reverse and deserialize an entire object graph as long as the provided stream is positioned accordingly to the place of a former object graph serialization.

There are a few catches here though: you will need to annotate all of your classes with [SerializableAttribute]. If your classes contain fields that are of other classes written by you, and you said they do:

[SerializableAttribute]
public class Player {
  public PlayerInfo Info; 
  //... etc 

then you need to annotate those with [SerializableAttribute] too:

[SerializableAttribute]
public class PlayerInfo { //... etc

If your classes contain fields that are of types written by others (say Microsoft) then those would better be already annotated with the attribute. Most of those which could be serialized already are. Primitive types are naturally serializable. Things that shouldn't be serialized are: FileStreams, Threads, Sockets, etc.

After making sure you have serializable classes all you need to do is serialize their instances, send them, receive them and deserialize them:

class Client {

  public static void SendMovement(Movement movement) {
    Message message = Foo.Serialize(movement);

    socketHelper.SendMessage(message);
  }
  public static void SendPlayer(Player player) {
    Message message = Foo.Serialize(player);

    socketHelper.SendMessage(message);
  }
  // .. etc

  public static void OnMessageReceivedFromServer(Message message) {
    object obj = Foo.Deserialize(message);
    if (obj is Movement)
      Client.ProcessOtherPlayersMovement(obj as Movement);
    else if (obj is Player)
      Client.ProcessOtherPlayersStatusUpdates(obj as Player);
    // .. etc
  }

  public static void ProcessOtherPlayersMovement(Movement movement) {
    //...
  }
  // .. etc

}

While on the server side:

class Server {

  public static void OnMessageReceived(Message message, SocketHelper from, SocketHelper[] all) {
    object obj = Foo.Deserialize( message );
    if (obj is Movement)
      Server.ProcessMovement( obj as Movement );
    else if (obj is Player)
      Server.ProcessPlayer( obj as Player );
    // .. etc

    foreach (var socketHelper in all)
      if (socketHelper != from)
        socketHelper.SendMessage( message );
  }
}

You will need a common assembly project (class library) to be referenced by both executable projects (client and server).

All your classes that need to be passed around will have to be written in that assembly so as that both the server and the client know how to understand each other at this very detailed level.

If the server needs not understand what is being said between the clients and only pass around messages (broadcasting one message to the other N-1 clients) then forget what I said about the common assembly. In that particular case, the server sees only bytes, while the clients have a deeper understanding of the actual messages being sent back and forth.

I said I had three approaches.

The second one involves .NET Remoting which can take a lot of work off your shoulders but which is hard to live with if you don't fully understand it. You can read about it on MSDN, here: http://msdn.microsoft.com/en-us/library/kwdt6w2k(v=vs.100).aspx

The third one would be better only if (now, or in the future) by XNA you mean Windows Phone or another implementation of XNA which does not support the BinaryFormatter class (ExEn with MonoTouch, or others). In that case you would have a hard time if you needed your server (a full blown, good old fashioned .NET application) to reference the common assembly I talked about and also have the game project (which would not be a good old fashioned .NET app but have a rather exotic nature) reference the exact same assembly.

In that case we would need to use and alternate form of serializing and deserializing your objects. You would also need to identically implement two sets of classes in the two worlds (.NET and WP7 or WP8). You could use some form of XML serializers which you would need to map to your classes explicitly (not as powerful as the BinaryFormatter class but more versatile in what the nature of the runtime that host your classes could be).

You can read about the XmlSerializer class on MSDN, here: http://msdn.microsoft.com/en-us/library/system.xml.serialization.xmlserializer.aspx

Solution 2

My personal quick-and-clean solution, using JSON.NET:

class JMessage
{
    public Type Type { get; set; }
    public JToken Value { get; set; }

    public static JMessage FromValue<T>(T value)
    {
        return new JMessage { Type = typeof(T), Value = JToken.FromObject(value) };
    }

    public static string Serialize(JMessage message)
    {
        return JToken.FromObject(message).ToString();
    }

    public static JMessage Deserialize(string data)
    {
        return JToken.Parse(data).ToObject<JMessage>();
    }
}

Now you can serialize your objects like so:

Player player = ...;
Enemy enemy = ...;
string data1 = JMessage.Serialize(JMessage.FromValue(player));
string data2 = JMessage.Serialize(JMessage.FromValue(enemy));

Send that data across the wire and then on the other end you can do something like:

string data = ...;
JMessage message = JMessage.Deserialize(data);
if (message.Type == typeof(Player))
{
    Player player = message.Value.ToObject<Player>();
}
else if (message.Type == typeof(Enemy))
{
    Enemy enemy = message.Value.ToObject<Enemy>();
}
//etc...

Solution 3

You could create your own solution using the various classes provided in the .net framework. You would want to checkout WCF or the Sockets namepsace, specifically the TcpClient and TcpListener classes, see MSDN. There are loads of great tutorials if you do a search related to using these. You would also need to consider how to turn your typed objects into byte arrays, similar to this question.

An alternative approach would be to use a network library. There are low-level libraries and high-level libraries. Given your level of programming experience and specific end goal I would suggest a high-level library. An example of such a network library would be lidgren. I'm the developer of another network library networkComms.net and a quick example of how you can send typed objects using this library follows:

Shared Base (defines Player object):

[ProtoContract]
class Player
{
    [ProtoMember(1)]
    public string Name { get; private set; }
    [ProtoMember(2)]
    public int Ammo { get; private set; }
    [ProtoMember(3)]
    public string Position { get; private set; }

    private Player() { }

    public Player(string name, int ammo, string position)
    {
        this.Name = name;
        this.Ammo = ammo;
        this.Position = position;
    }
}

Client (sends a single Player object):

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;

using NetworkCommsDotNet;
using ProtoBuf;

namespace Client
{
    class Program
    {
        static void Main(string[] args)
        {
            Player player = new Player("MarcF", 100, "09.09N,21.12W");

            //Could also use UDPConnection.GetConnection...
            TCPConnection.GetConnection(new ConnectionInfo("127.0.0.1", 10000)).SendObject("PlayerData", player);

            Console.WriteLine("Send completed. Press any key to exit client.");
            Console.ReadKey(true);
            NetworkComms.Shutdown();
        }
    }
}

Server:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;

using NetworkCommsDotNet;
using ProtoBuf;

namespace Server
{
    class Program
    {
        static void Main(string[] args)
        {
            // Convert incoming data to a <Player> object and run this method when an incoming packet is received.
            NetworkComms.AppendGlobalIncomingPacketHandler<Player>("PlayerData", (packetHeader, connection, incomingPlayer) =>
            {
                Console.WriteLine("Received player data. Player name was " + incomingPlayer.Name);
                //Do anything else with the player object here
                //e.g. UpdatePlayerPosition(incomingPlayer);
            });

            //Listen for incoming connections
            TCPConnection.StartListening(true);

            Console.WriteLine("Server ready. Press any key to shutdown server.");
            Console.ReadKey(true);
            NetworkComms.Shutdown();
        }
    }
}

The above is a modified version of this tutorial. You will obviously need to download the NetworkCommsDotNet DLL from the website so that you can add it in the 'using NetworkCommsDotNet' reference. Also see the server IP address in the client example is currently "127.0.0.1", this should work if you run both the server and client on the same machine.

Solution 4

After more than 2 years, I've found new ways of solving this issue, and I thought sharing it might be useful to someone. Please note the accepted answer is still valid.

A simplest way to serialize typed objects I could find is by using the json converter in Json.NET. There's a settings object that allow you to store the type in the json as a value named $type. Here's how to do it and the resulting json:

JsonSerializerSettings settings = new JsonSerializerSettings
{
    TypeNameHandling = TypeNameHandling.All
};

JsonConvert.SerializeObject(myObject, settings);

Json result:

{
    "$type" : "Testing.MyType, Testing",
    "ExampleProperty" : "Hello world!"
}

When deserializing, if the same setting is used, an object of the correct type will be deserialized. Exactly what I needed! Hope this helps.

View more solutions

Comments

[Machinarius]

I'd say you would need to craft your own solution, a high performance serialization mechanism. Something along the lines of bit packing would be fast enough for small pieces of info.

[John Saunders]

Does XNA support WCF? If so, then that would be the way to go.

[Philippe Paré]

All these solutions are very interesting, but the only problem I can think of is How to find out what type it is on the receiver side? If I deserialize something, will I be able to retrieve the type for a cast? Thanks for all the answers! Very helpful.

[Philippe Paré]

All these solutions are very interesting, but the only problem I can think of is How to find out what type it is on the receiver side? If I deserialize something, will I be able to retrieve the type for a cast? Thanks for all the answers! Very helpful.

[Timothy Shields]

The method I outlined solves exactly your problem, namely how to determine the type. See how I say message.Type == typeof(Player) or message.Type == typeof(Enemy) in the example? This is how you "retrieve the type for a cast."

[Eduard Dumitru]

If you use the BinaryFormatter approach, somehow it will be as if the two counterparts were in the same process. In short: you have a clasic problem of casting a bunch of classes to object and then trying to figure out what the objects really are (Player, Movement, Interaction?). So if you use something like if (obj is Player) { Player asPlayer = obj as Player; /* do specific Player processing */ you'll solve your question. The deserialize method already knows (at runtime) what you need only to dispatch (it's a player or not, it'a a move, etc).

[Eduard Dumitru]

It's just that logically speaking, Microsoft couldn't've guessed that you, in feb 2013, would write a player class and need it deserialized. So at syntactical, written, desing time level that method returns object.

[Eduard Dumitru]

So the shorter answer is yes. You can do all of the things you do in regular single process oop: is, as, (DestType)casted, even .GetType().Name or .GetType() == typeof(Player)

[KADEM Mohammed]

@EduardDumitru : i had the same problem as Philippe, "Server not guessing what deserializer use" and than i thought about using a two levels serialization : one for the object it self, and the sec is "Code name for the type + XML of the object) (1 for game, 2 for Player...) and than use that Code name to find the type, and with switch-case i find the right serializer ! can you please tell what you think about that approach ?

[Eduard Dumitru]

@CarterNolan: I'll try to keep this short. 1st of all, I don't understand what the two levels of serialization are: One for the object itself.. okay, but then why does the second one have an XML of the object. I'm presuming you chose XML as a form of serialisation. In that case, what does "the object itself" mean ? You know that you can't actually send objects through sockets, just bytes, and the art of serialization is that of transforming objects into bytes (and vice-versa)... Also, you're not sending XML either. It's just easier for XML to be turned into bytes...

[Eduard Dumitru]

...And that's because XML has a linear representation (you can instantly decide what it's length is,or navigate to it's Nth element i.e a character,and each element poses no further mystery by itself) whereas graphs of objects are complicated mathematical structures.Now,regarding your question I can quickly say that since there are no perfect recipes that work everywhere,and since I can't understand your exact situation in 1 comment, I'd suggest: Do It ! It's okay. It doesn't sound awful, and every experiment is helpful :) I also suggest you write a question thoroughly describing your question

[KADEM Mohammed]

@EduardDumitru Sorry i tried to paste the code as comment, but i look like a mess (not formatted) anyway what i am doing is working, if anybody needs, just request it and i will post it. Thank you.

[pookie]

I always get an exception when I deserialize: An unhandled exception of type 'System.Runtime.Serialization.SerializationException' occurred in mscorlib.dll Additional information: Binary stream '0' does not contain a valid BinaryHeader. Possible causes are invalid stream or object version change between serialization and deserialization.