protobuf-net: Serializing an empty List

Question

we have some problems with serializing an empty list. here some code in .NET using CF 2.0

//Generating the protobuf-msg
ProtoBufMessage msg = new ProtoBufMessage();
msg.list = new List<AnotherProtobufMessage>();
// Serializing and sending throw HTTP-POST
MemoryStream stream = new MemoryStream();
Serializer.Serialize(stream, msg);
byte[] bytes = stream.ToArray();
HttpWebRequest request = createRequest();
request.ContentLength = bytes.Length ;

using (Stream httpStream = request.GetRequestStream())
{              
      httpStream.Write(bytes, 0, bytes.Length);
}

we got a exception, when we try to write on the stream (bytes.length out of range). But a type with an empty List should not be 0 bytes, right (type-information?)?

We need this type of sending, because in the Response are the messages from the Server for our client.

Solution 1

The wire format (defined by google - not inside my control!) only sends data for items. It makes no distinction between an empty list and a null list. So if there is no data to send - yes, the length is 0 (it is a very frugal format ;-p).

Protocol buffers do not include any type metadata on the wire.

Another common gotcha here is that you might assume your list property is automatically instantiated as empty, but it won't be (unless your code does it, perhaps in a field initializer or constructor).

Here's a workable hack:

[ProtoContract]
class SomeType {

    [ProtoMember(1)]
    public List<SomeOtherType> Items {get;set;}

    [DefaultValue(false), ProtoMember(2)]
    private bool IsEmptyList {
        get { return Items != null && Items.Count == 0; }
        set { if(value) {Items = new List<SomeOtherType>();}}
    }
}

Hacky maybe, but it should work. You could also lose the Items "set" if you want and just drop the bool:

    [ProtoMember(1)]
    public List<SomeOtherType> Items {get {return items;}}
    private readonly List<SomeOtherType> items = new List<SomeOtherType>();

    [DefaultValue(false), ProtoMember(2)]
    private bool IsEmptyList {
        get { return items.Count == 0; }
        set { }
    }

Solution 2

As @Marc said, the wire format only sends data for items, so in order to know if the list was empty or null, you have to add that bit of information to the stream.
Adding extra property to indicate whether the original collection was empty or not is easy but if you don't want to modify the original type definition you have another two options:

Serialize Using Surrogate

The surrogate type will have the extra property (keeping your original type untouched) and will restore the original state of the list: null, with items or empty.

    [TestMethod]
    public void SerializeEmptyCollectionUsingSurrogate_RemainEmpty()
    {
        var instance = new SomeType { Items = new List<int>() };

        // set the surrogate
        RuntimeTypeModel.Default.Add(typeof(SomeType), true).SetSurrogate(typeof(SomeTypeSurrogate));

        // serialize-deserialize using cloning
        var clone = Serializer.DeepClone(instance);

        // clone is not null and empty
        Assert.IsNotNull(clone.Items);
        Assert.AreEqual(0, clone.Items.Count);
    }

    [ProtoContract]
    public class SomeType
    {
        [ProtoMember(1)]
        public List<int> Items { get; set; }
    }

    [ProtoContract]
    public class SomeTypeSurrogate
    {
        [ProtoMember(1)]
        public List<int> Items { get; set; }

        [ProtoMember(2)]
        public bool ItemsIsEmpty { get; set; }

        public static implicit operator SomeTypeSurrogate(SomeType value)
        {
            return value != null
                ? new SomeTypeSurrogate { Items = value.Items, ItemsIsEmpty = value.Items != null && value.Items.Count == 0 }
                : null;
        }

        public static implicit operator SomeType(SomeTypeSurrogate value)
        {
            return value != null
                ? new SomeType { Items = value.ItemsIsEmpty ? new List<int>() : value.Items }
                : null;
        }
    }

Make Your Types Extensible

protobuf-net suggest the IExtensible interface which allow you to extend types so that fields can be added to a message without anything breaking (read more here). In order to use protobuf-net extension you can inherit the Extensible class or implement the IExtensible interface to avoid inheritance constraint.
Now that your type is "extensible" you define [OnSerializing] and [OnDeserialized] methods to add the new indicators that will be serialized to the stream and deserialized from it when reconstructing the object with its original state.
The pros is that you don't need to define new properties nor new types as surrogates, the cons is that IExtensible isn't supported if your type have sub types defined in your type model.

    [TestMethod]
    public void SerializeEmptyCollectionInExtensibleType_RemainEmpty()
    {
        var instance = new Store { Products = new List<string>() };

        // serialize-deserialize using cloning
        var clone = Serializer.DeepClone(instance);

        // clone is not null and empty
        Assert.IsNotNull(clone.Products);
        Assert.AreEqual(0, clone.Products.Count);
    }

    [ProtoContract]
    public class Store : Extensible
    {
        [ProtoMember(1)]
        public List<string> Products { get; set; }

        [OnSerializing]
        public void OnDeserializing()
        {
            var productsListIsEmpty = this.Products != null && this.Products.Count == 0;
            Extensible.AppendValue(this, 101, productsListIsEmpty);
        }

        [OnDeserialized]
        public void OnDeserialized()
        {
            var productsListIsEmpty = Extensible.GetValue<bool>(this, 101);
            if (productsListIsEmpty)
                this.Products = new List<string>();
        }
    }