Java vs .NET performance

Question

I got myself very surprised after I wrote this small code to compare .NET 4.5 and Java 8 performance in my computer:

class ArrayTest
{
    public int[][] jagged;

    public ArrayTest(int width, int height)
    {
        Height = height;
        Width = width;
        Random rng = new Random();
        jagged = new int[width][];
        for (int i = 0; i < height; i++)
        {
            jagged[i] = new int[width];
            for (int j = 0; j < jagged[i][j]; j++)
            {
                jagged[i][j] = rng.Next(2048);
            }
        }
    }
    public int this[int i, int j]
    {
        get
        {
            return jagged[i][j];
        }
        set
        {
            jagged[i][j] = value;
        }
    }

    public void DoMath(ArrayTest a)
    {
        for (int i = 0; i < Height; i++)
        {
            for (int j = 0; j < Width; j++)
            {
                this[i, j] *= a[i, j];
            }
        }
    }

    public int Height { get; private set; }

    public int Width { get; private set; }
}



class Program
{
    static void Main(string[] args)
    {
        Random rng = new Random();
        const int loop = 10;
        int width = 10000,
            height = 10000;

        ArrayTest a1 = new ArrayTest(width, height),
            a2 = new ArrayTest(width, height);


        Stopwatch sw = new Stopwatch();
        sw.Start();
        for (int i = 0; i < loop; i++)
        {
            a1.DoMath(a2);
        }
        sw.Stop();

        Console.WriteLine("Time taken: " + sw.ElapsedMilliseconds);

        Console.ReadKey();
    }
}

Here is the Java version:

    public class ArrayTest {
    private int width, height;
    private int[][] array;

    public ArrayTest(int width, int height) {
        this.width = width;
        this.height = height;
        array = new int[height][width];
        Random rng = new Random();
        for (int i = 0; i < height; i++) {
            for (int j = 0; j < width; j++) {
                array[i][j] = rng.nextInt(2048);
            }
        }
    }

    public int getWidth() {
        return width;
    }
    public void setWidth(int width) {
        this.width = width;
    }
    public int getHeight() {
        return height;
    }
    public void setHeight(int height) {
        this.height = height;
    }
    public int[][] getArray() {
        return array;
    }

    public void doMath(ArrayTest a) {
        for (int i = 0; i < height; i++) {
            for (int j = 0; j < width; j++) {
                array[i][j] *= a.getArray()[i][j];
            }
        }
    }

}

public class Main {

    public static void main(String[] args) {
        // TODO Auto-generated method stub
        final int loops = 10;
        int width = 10000, height = 10000;
        ArrayTest a1 = new ArrayTest(width, height),
                a2 = new ArrayTest(width, height);

        long start, end;


        start = java.lang.System.currentTimeMillis();
        for (int i = 0; i < loops; i++) {
            a1.doMath(a2);
        }
        end = java.lang.System.currentTimeMillis();
        System.out.println("Elapsed time: " + (float)(end - start));
    }

}

In my computer this C# code is taking about 5200ms to run, and the Java version is taking about 2800ms(!!!). I was actually expecting that the .NET version would run faster (or at least close to) than Java, but got very surprised with this result.

Note: I ran the .NET version compiled in release mode, outside Visual Studio.

Can somebody explain this result? Is this really right? How could I rewrite this code so that the C#.NET version gets closer to the Java one in execution speed?

[edit]

Well, I know this is not a really valid benchmarking or a fair comparison, but how could I rewrite this code so that the C#.NET version gets closer to the Java one in execution speed? Tried in any forms (manipulating the jagged array directly, via a getter, etc), but the test ran even slower.

[edit 2]

Edited the test so that I have width and height = 500, and loop = 5000. Now I get about 6300ms for .NET version, and about 3700ms for Java version. ran the test multiple times for each version, of course.

[edit 3]

Just wrote a similar test using flat arrays instead of 2D arrays, and this time the .NET version runs equally or even faster than Java. So is that it? C#.NET's jagged arrays are just slower than Java's multidimensional arrays?

Solution 1

Any time you do any kind of benchmarking or performance analysis, you need to ask a lot of questions, and take any particular result with (to paraphrase Tanenbaum), "a metric ton of salt".

HOWEVER ....

I copied pasted your code, and this is what I got:

Compiler      Version   Timing
--------      -------   ------
MSVS 2012     C# 5      4448.0
Eclipse Luna  JRE 1.7    977.0

So the Java program ran 4.5x faster than the C# program.

I also ran the MSVS "Profiling Wizard":

https://msdn.microsoft.com/en-us/library/dd264959.aspx

Large picture

As you might be able to see from the screenshot, the "Big Pig" in this profile was ArrayTest.get_item(int32, int32), which consumed NEARLY HALF of the execution time:

Large picture

Solution 2

You can't judge the performance based on single instance. You need to do it for multiple instance. If you want concrete answers on performance I would suggest you to read quora, blog or SO.