Copying byte array to various fields in class/struct in C#

Question

In the example C# code below, I have a byte array which has been read from a socket. I want to parse the data into the various fields of 'exampleClass' (first 8 bytes into the 64-bit variable 'field1', next 4 bytes into 32-bit variable 'field2', etc.)

using System;
namespace CsByteCopy
{
  class Program
  {
    class ExampleClass
    {
      public UInt64 field1;
      public UInt32 field2;
      public UInt16 field3;
      public byte[] field4 = new byte[18];
    }

    static void Main(string[] args)
    {
      byte[] exampleData =
      {
        // These 8 bytes should go in 'field1'
        0x00,0x01,0x02,0x03,0x04,0x05,0x06,0x07,
        // These 4 bytes should go in 'field2'
        0x08,0x09,0x0A,0x0B,
        // These 2 bytes should go in 'field3'
        0x0C,0x0D,
        // These 18 * 1 bytes should go in 'field4'
        0x0E,0x0F,0x10,0x11,0x12,0x13,0x14,0x15,0x16,0x17,0x18,0x19,0x1A,0x1B,0x1C,0x1D,0x1E,0x1F,
      };

      ExampleClass exampleClass = new ExampleClass();
      // Perform copy
    }
  }
}

It's been a long time since I last used C, but if I recall correctly, I might have been able to get away with a single memcpy() call to populate all the fields in the class. What's the most efficient way of populating the fields of 'exampleClass' in C#?

Solution 1

You have a lot of options, the one that turns out the best usually depends on what your program needs/can handle and exactly how much speed you want. There are a lot of articles that explain the different ways you can populate a class or a struct with data.

Binary Serialization is one way. That requires you write a custom serializer, which is fairly easy. An MSDN article regarding that shows it's not too difficult.

You can write a private constructor that takes in the byte array and uses a BinaryReader or the BitConverter class to read from the byte array (you'd have to wrap it in a MemoryStream for BinaryReader) or to simply convert sections of the byte array to the values you need (BitConverter). In the case of BitConverter you would also need to use Buffer.BlockCopy to copy the remaining data of the byte array to the byte array field in your class

A third way, also generally the fastest way, is to convert your class to a struct and use unsafe code to cast the byte array as that struct. Something like this:

unsafe struct ExampleClass
{
   public ulong field1;
   public uint field2
   public ushort field3
   public fixed byte field4[18];

   public static ExampleClass ReadStruct(byte[] data)
   {
       fixed (byte* pb = &data[0])
       {
           return *(ExampleClass*)pb;
       }
   }
}

Of course, the above code is only valid if you can use unsafe code. Furthermore, converting the class to a struct may also not be what you're looking for. In most cases, structs are passed by value to functions, so that calling methods copies the entire struct (in this case 32 bytes) instead of passing a reference (4 or 8 bytes, depending on CPU architecture) and can reduce the efficiency of your program. There are exceptions to structs being passed by value, and you can use your favorite search engine for that.

If you can't use unsafe code, there is also Marshal.PtrToStructure which will do the same as the above code, but about 10 times slower. You would also need to use the MarshalAs attribute to specify the size of the array, instead of using the fixed keyword (which is unsafe). At that point, you might as well use the BinaryReader/BitConverter, as it will be faster than the marshal class.

Solution 2

I think you want this:

Here by changing the buffer in field4 you also change the other 3 which are specific types.

[StructLayout(LayoutKind.Explicit, Size=8)]
struct UValue
{
   [FieldOffset(0)]
   public fixed byte field4[18]; // 18 bytes long

   [FieldOffset(0)]
   public uint64 field1;

   [FieldOffset(8)]
   public Uint32 field2

   [FieldOffset(12)]
   public Uint16 field3

}

You might want this:

Here by changing the buffer in fieldload you change the others as above but there is also another byte buffer at the end.

[StructLayout(LayoutKind.Explicit, Size=8)]
struct UValue
{
   [FieldOffset(0)]
   public fixed byte fieldload[38]; // modify this to modify others

   [FieldOffset(0)]
   public uint64 field1;

   [FieldOffset(8)]
   public Uint32 field2

   [FieldOffset(12)]
   public Uint16 field3

   [FieldOffset(14)]
   public fixed byte field4[18]; // 18 bytes long
}

Solution 3

Another option, if you can use a struct, is to Marshal the byte array directly into the structure.

struct ExampleStruct
{
    public UInt64 field1;
    public UInt32 field2;
    public UInt16 field3;
    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 18)]
    public byte[] field4;
}

And to get a struct for that:

var handle = GCHandle.Alloc(exampleData, GCHandleType.Pinned);
var structure = Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof (ExampleStruct));
handle.Free();

And structure will contain your data.

Solution 4

Forget the efficiency thing and make your code maintainable and readable first. If necessary, profile and improve.

Whats wrong with using the BitConverter class.

field1 = BitConverter.ToInt64(exampleData, 0)
field2 = BitConverter.ToInt32(exampleData, 8)
field3 = BitConverter.ToInt16(exampleData, 12)
Array.Copy(exampleData, 14, field4, 0, 18)

Solution 5

And now for something completely different ...

This does not really answer the OP's question; instead it is something I cooked up to provide a way of mapping a C# struct on top of a byte array, and allow the individual fields in the underlying byte array to be both read and written. It uses unsafe code, and gets a pointer to the byte array and then casts it to a pointer to the struct that maps the fields.

This may not be all that efficient, but has the advantage of providing a symbolic mapping of the fields without the use of "magic numbers" for the lengths and offsets of the fields. But note that this will not work for data interchange with a system that uses big endian instead of little endian data representation.

The field access methods are implemented as extension methods. See TestMethod() for an example of how to use them.

This was inspired by the answer by Christopher Currens - if you find this useful please give him an upvote.

using System;
using System.Runtime.InteropServices;

namespace ConsoleApplication1
{
   [StructLayout(LayoutKind.Sequential, Pack = 1)]
   unsafe struct ExampleStruct
   {
      internal const int CField4Length = 18;

      public UInt64 Field1;
      public UInt32 Field2;
      public UInt16 Field3;
      public fixed byte Field4[CField4Length];
   }

   static unsafe class ExampleStructExtensionMethods
   {
      public static UInt64 GetField1(this byte[] byteArray)
      {
         fixed (byte* byteArrayPointer = &byteArray[0])
         {
            return (*(ExampleStruct*)byteArrayPointer).Field1;
         }
      }

      public static UInt32 GetField2(this byte[] byteArray)
      {
         fixed (byte* byteArrayPointer = &byteArray[0])
         {
            return (*(ExampleStruct*)byteArrayPointer).Field2;
         }
      }

      public static UInt16 GetField3(this byte[] byteArray)
      {
         fixed (byte* byteArrayPointer = &byteArray[0])
         {
            return (*(ExampleStruct*)byteArrayPointer).Field3;
         }
      }

      public static byte[] GetField4(this byte[] byteArray)
      {
         fixed (byte* byteArrayPointer = &byteArray[0])
         {
            byte[] field4 = new byte[ExampleStruct.CField4Length];
            for (int i = 0; i < ExampleStruct.CField4Length; i++)
               field4[i] = (*(ExampleStruct*)byteArrayPointer).Field4[i];

            return field4;
         }
      }

      public static void SetField1(this byte[] byteArray, UInt64 field1)
      {
         fixed (byte* byteArrayPointer = &byteArray[0])
         {
            (*(ExampleStruct*)byteArrayPointer).Field1 = field1;
         }
      }

      public static void SetField2(this byte[] byteArray, UInt32 field2)
      {
         fixed (byte* byteArrayPointer = &byteArray[0])
         {
            (*(ExampleStruct*)byteArrayPointer).Field2 = field2;
         }
      }

      public static void SetField3(this byte[] byteArray, UInt16 field3)
      {
         fixed (byte* byteArrayPointer = &byteArray[0])
         {
            (*(ExampleStruct*)byteArrayPointer).Field3 = field3;
         }
      }

      public static void SetField4(this byte[] byteArray, byte[] field4)
      {
         if (field4.Length != ExampleStruct.CField4Length)
            throw new ArgumentException("Byte array must have length 18", "field4");

         fixed (byte* byteArrayPointer = &byteArray[0])
         {
            for (int i = 0; i < ExampleStruct.CField4Length; i++)
               (*(ExampleStruct*)byteArrayPointer).Field4[i] = field4[i];
         }
      }
   }

   class TestProgram
   {
      byte[] exampleData =
      {
        // These 8 bytes should go in 'field1'
        0x00,0x01,0x02,0x03,0x04,0x05,0x06,0x07,
        // These 4 bytes should go in 'field2'
        0x08,0x09,0x0A,0x0B,
        // These 2 bytes should go in 'field3'
        0x0C,0x0D,
        // These 18 * 1 bytes should go in 'field4'
        0x0E,0x0F,0x10,0x11,0x12,0x13,0x14,0x15,0x16,0x17,0x18,0x19,0x1A,0x1B,0x1C,0x1D,0x1E,0x1F,
      };

      public void TestMethod()
      {
         UInt64 field1 = exampleData.GetField1();
         UInt32 field2 = exampleData.GetField2();
         UInt16 field3 = exampleData.GetField3();
         byte[] field4 = exampleData.GetField4();

         exampleData.SetField1(++field1);
         exampleData.SetField2(++field2);
         exampleData.SetField3(++field3);
         exampleData.SetField4(new byte[ExampleStruct.CField4Length] 
           { 0x42,0x42,0x42,0x42,0x42,0x42,0x42,0x42,0x42,0x42,0x42,0x42,0x42,0x42,0x42,0x42,0x42,0x42 });
      }
   }
}

View more solutions

Comments

[Hogan]

Neat --- so to copy the data in you copy to handle before you free it? (can you give an example of that?)

[Hogan]

+1, Good answer, how do these ways compare to using LayoutKind.Explicit as in my example? (or is there a reason my way won't work?)

[Christopher Currens]

Yours won't compile, because byte[any number] varName isn't valid C# code. Also, StructLayout is meant to let the marshaller know the order and/or offsets of the fields in the structure. I noticed you've specified the same offset for two values, and I think the marshaller will throw an error for that. If you want to use Attributes to specify array sizes, you can use the MarshalAs attribute and have it marshal as an UnmangedType.ByValArray with SizeConst as the array size.

[Hogan]

Same offset is valid, see microsoft documentation here: msdn.microsoft.com/en-us/library/aa288471(v=vs.71).aspx

[vcsjones]

@Hogan: The purpose of the GCHandle is to pin the byte[] so that the CLR / GC doesn't move it while Marshal.PtrToStructure is operating, and also so we can get the address where the byte[] lives. The Free is for the GCHandle itself. Alloc doesn't copy the byte[] contents.

[Christopher Currens]

@Hogan, You're right. That's what I get for going from memory :) Either way, the important thing, is that using the MarshalAs attribute would be better than explicitly specifying the layout in this case. Specifying the layout won't allow the marshaller to account for the array size, unless you're using unsafe code.

[Christopher Currens]

It's important to know that the fixed keyword can only be used in an unsafe context. Fixed literally means that the field is actually of type byte* instead of a byte[].

[Hogan]

@Christopher - this is correct, it might even need an unsafe modifier for the stuct decl. I did not try and compile this code.

[MrHIDEn]

I like this unsafe struct ExampleClass idea. This works but I need to convert it back to bytes. Is there similar way to do that?

[MrHIDEn]

fixed (byte* pb = &data[0]) == fixed (byte* pb = data)

[RenniePet]

"make your code maintainable and readable first" - well, you've demonstrated exactly why your suggested solution is not very good. It's not readable (it contains lots of "magic numbers"), and it's definitely not maintainable. If I wanted to add a new field between field1 and field2 there's a 90% chance that I'd do it wrong.

[RenniePet]

I think you mean "typeof (ExampleStruct)".

[RenniePet]

"because structs are immutable" - no, that's a (debatable) usage recommendation only.

[Christopher Currens]

@RenniePet You're right, it's not an accurate statement. I wasn't intending to say that they are immutable (obviously public fields are not immutable), I was poorly trying to say that they are passed by value and thus copied (as the latter half of the sentence indicates). Thanks for the comment. I'll update the wording.

[RenniePet]

Thanks again for this answer - it made me aware of how unsafe code could be used to map a struct to a byte array. I've now taken the concept one step further, because I wanted to both read and write the mapped fields in the byte array. See the "answer" I've posted below.