Why is my Azure VM (D3) Disk I/O crazy slow?

Question

I have setup a D3 VM running Windows 10 on my Azure account. Unfortunately the Disk speed, on both the C (200GB) and D (100GB Temp) are EXTREMELY low. Less than 1MB/s when viewed through Task Manager while copying over a large folder structure containing about 16000 items.

To be honest I'm not even sure what to try...I guess my plan will be to create a new vhd drive, attach it, and see how it performs but I'm not real hopeful.

I read through this post --> Slow disk performance on Azure Virtual Machine

and

this post --> http://azure.microsoft.com/blog/2014/10/06/d-series-performance-expectations/

They both reference "throttling" which I understand but what I am seeing must be something different as I never approach the level it states of 96MB/s write.

Is this just common knowledge or is it possible I have something configured wrong? A Windows 10 VM bug maybe? I have verified that I did, in fact, chose D3 for my VM size which includes a 200GB local SSD.

Any suggestions on what I can check or try next?

---

EDIT

Some further info. Here is a screenshot of my Disk Read/Write from the Azure Portal for the last 24 hours.

Solution

Some problems here, but besides the possible azure issue you lack a basic understanding of computer storage.

opying over a large folder structure containing about 16000 items.

that is a LOT of files. Likely not a lot of too large files. Lots of folders, lots of metadata.

what I am seeing must be something different as I never approach the level it states of 96MB/s write.

which is totally irrelevant if you waste all your time doing linear IO (which explorer does) with a ton of metadata updates. MB is not IOPS.

Now, I agree you have an issue -because I also see 12k IOPS given - but as administrator you should know that:

• Copy operations of a lot of files cause a lot of metadata updates which slow things up and

• Windows Explorer is really slow in that it does things linearly. Split the copy job into multiple smaller jobs that run in parallel and you may see a speed up.

The last one can be significant - Explorer etc. do one file at a time, waiting for certain things to finish. All the while nothing happens. Multiple parallel jobs may simply be - faster.

Comments

[Refracted Paladin]

Sure, I'll research how to do that and post back here once I have figured it out. I apologize for taking your time. Thanks for the responses.

[TomTom]

What we are interested in is the IO counter and the seconds per read / write - which indicate latency.

[Refracted Paladin]

That makes sense I just am not seeing how to do that in the Portal. I'll 'Bingle' it for a while so I can provide that.

[Refracted Paladin]

I found iometer.org and will try and use that to get some helpful data.

[Stephen Punwasi]

During benchmarks I found that in general Azure's SSD I/O is about 1/20th the speed of Digital Ocean. They must be doing something wonky with the storage.

[Refracted Paladin]

@StephenP. Interesting, may I ask what you used to benchmark?

[TomTom]

@RefractedPaladin IOMETER? What about using the tools of the operating system? As a programmer you should know the basics of available API's for instrumenting programs. For example.... the whole performance counter framework where you can just look at the disc performance numbers.

[Refracted Paladin]

Yup, I am a developer who is exploring using Azure VM's. Not a System Admin. I do understand that 16k is a lot of items. I should also have known better than to look at Task Manager for my speed info. I will post a picture of what Azure Portal is reporting for Read/Write. I don't know how to monitor IOPS through the Portal. This all started because I noticed everything running really slow, especially Visual Studio so I narrowed it down to Disk Performance I just don't understand why.