Efficiently convert byte array to Decimal

Question

If I have a byte array and want to convert a contiguous 16 byte block of that array, containing .net's representation of a Decimal, into a proper Decimal struct, what is the most efficient way to do it?

Here's the code that showed up in my profiler as the biggest CPU consumer in a case that I'm optimizing.

public static decimal ByteArrayToDecimal(byte[] src, int offset)
{
    using (MemoryStream stream = new MemoryStream(src))
    {
        stream.Position = offset;
        using (BinaryReader reader = new BinaryReader(stream))
            return reader.ReadDecimal();
    }
}

To get rid of MemoryStream and BinaryReader, I thought feeding an array of BitConverter.ToInt32(src, offset + x)s into the Decimal(Int32[]) constructor would be faster than the solution I present below, but the version below is, strangely enough, twice as fast.

const byte DecimalSignBit = 128;
public static decimal ByteArrayToDecimal(byte[] src, int offset)
{
    return new decimal(
        BitConverter.ToInt32(src, offset),
        BitConverter.ToInt32(src, offset + 4),
        BitConverter.ToInt32(src, offset + 8),
        src[offset + 15] == DecimalSignBit,
        src[offset + 14]);
}

This is 10 times as fast as the MemoryStream/BinaryReader combo, and I tested it with a bunch of extreme values to make sure it works, but the decimal representation is not as straightforward as that of other primitive types, so I'm not yet convinced it works for 100% of the possible decimal values.

In theory however, there could be a way to copy those 16 contiguous byte to some other place in memory and declare that to be a Decimal, without any checks. Is anyone aware of a method to do this?

(There's only one problem: Although decimals are represented as 16 bytes, some of the possible values do not constitute valid decimals, so doing an uncheckedmemcpy could potentially break things...)

Or is there any other faster way?

Solution 1

Even though this is an old question, I was a bit intrigued, so decided to run some experiments. Let's start with the experiment code.

static void Main(string[] args)
{
    byte[] serialized = new byte[16 * 10000000];

    Stopwatch sw = Stopwatch.StartNew();
    for (int i = 0; i < 10000000; ++i)
    {
        decimal d = i;

        // Serialize
        using (var ms = new MemoryStream(serialized))
        {
            ms.Position = (i * 16);
            using (var bw = new BinaryWriter(ms))
            {
                bw.Write(d);
            }
        }
    }
    var ser = sw.Elapsed.TotalSeconds;

    sw = Stopwatch.StartNew();
    decimal total = 0;
    for (int i = 0; i < 10000000; ++i)
    {
        // Deserialize
        using (var ms = new MemoryStream(serialized))
        {
            ms.Position = (i * 16);
            using (var br = new BinaryReader(ms))
            {
                total += br.ReadDecimal();
            }
        }
    }
    var dser = sw.Elapsed.TotalSeconds;

    Console.WriteLine("Time: {0:0.00}s serialization, {1:0.00}s deserialization", ser, dser);
    Console.ReadLine();
}

Result: Time: 1.68s serialization, 1.81s deserialization. This is our baseline. I also tried Buffer.BlockCopy to an int[4], which gives us 0.42s for deserialization. Using the method described in the question, deserialization goes down to 0.29s.

In theory however, there could be a way to copy those 16 contiguous byte to some other place in memory and declare that to be a Decimal, without any checks. Is anyone aware of a method to do this?

Well yes, the fastest way to do this is to use unsafe code, which is okay here because decimals are value types:

static unsafe void Main(string[] args)
{
    byte[] serialized = new byte[16 * 10000000];

    Stopwatch sw = Stopwatch.StartNew();
    for (int i = 0; i < 10000000; ++i)
    {
        decimal d = i;

        fixed (byte* sp = serialized)
        {
            *(decimal*)(sp + i * 16) = d;
        }
    }
    var ser = sw.Elapsed.TotalSeconds;

    sw = Stopwatch.StartNew();
    decimal total = 0;
    for (int i = 0; i < 10000000; ++i)
    {
        // Deserialize
        decimal d;
        fixed (byte* sp = serialized)
        {
            d = *(decimal*)(sp + i * 16);
        }

        total += d;
    }
    var dser = sw.Elapsed.TotalSeconds;

    Console.WriteLine("Time: {0:0.00}s serialization, {1:0.00}s deserialization", ser, dser);

    Console.ReadLine();
}

At this point, our result is: Time: 0.07s serialization, 0.16s deserialization. Pretty sure that's the fastest this is going to get... still, you have to accept unsafe here, and I assume stuff is written the same way as it's read.

Solution 2

@Eugene Beresovksy read from a stream is very costly. MemoryStream is certainly a powerful and versatile tool, but it has a pretty high cost to a direct reading a binary array. Perhaps because of this the second method performs better.

I have a 3rd solution for you, but before I write it, it is necessary to say that I haven't tested the performance of it.

public static decimal ByteArrayToDecimal(byte[] src, int offset)
{
    var i1 = BitConverter.ToInt32(src, offset);
    var i2 = BitConverter.ToInt32(src, offset + 4);
    var i3 = BitConverter.ToInt32(src, offset + 8);
    var i4 = BitConverter.ToInt32(src, offset + 12);

    return new decimal(new int[] { i1, i2, i3, i4 });
}

This is a way to make the building based on a binary without worrying about the canonical of System.Decimal. It is the inverse of the default .net bit extraction method:

System.Int32[] bits = Decimal.GetBits((decimal)10);

EDITED:

This solution perhaps don't peform better but also don't have this problem: "(There's only one problem: Although decimals are represented as 16 bytes, some of the possible values do not constitute valid decimals, so doing an uncheckedmemcpy could potentially break things...)".

Comments

[Evgeniy Berezovsky]

Although your solution is the more straightforward one, it is only half as fast as mine, strangely enough. Read the text between the 2 code snippets in my question and you'll find that I did already try this, without spelling it out in code, as it did not perform. In terms of correctness, it is no better or worse than my solution, unless there's a bug in it (e.g. a range check that needs to be added), which should be fixable and would incur a cost in performance that I doubt would make it as "slow" as the new Decimal(Int32[]) solution.

[atlaste]

@EugeneBeresovsky I know this is an old post, but I was wondering if you tried the variant int[] tmp = new int[4]; Buffer.BlockCopy(src, offset, tmp, 0, 16); return new decimal(tmp);. BitConverter is quite slow, so that might tip the answer to this solution a bit.