Why does Azure deployment take so long?

Question

I'm trying to understand why it can take from 20-60min to deploy a small application to Azure (using the configuration/package upload method, not from within VS).

I've read through this situation and this one but I'm still a little unclear - is there a weird non-technology ritual that occurs while the instances are distributing, like somebody over at Microsoft lighting a candle or doing a dance?

Solution 1

As a fellow Azure user, I share your pain - deploying isn't "quick"/"painless" - and this hurts especially when you're in a development cycle and want to test dev iterations on Azure. However, in general deployments should take much less than 60 minutes - and less than 20 minutes too.

Steve Marx provided a brief overview of the steps involved in deployment: http://blog.smarx.com/posts/what-happens-when-you-deploy-on-windows-azure

And he references a deeper level explanation at: http://channel9.msdn.com/blogs/pdc2008/es19

Solution 2

There's a lot that goes on behind the scenes when you deploy an application to the Azure cloud. I don't have any special insight into what's going on behind the curtain, but having worked on the VS tools to upload projects to the Azure cloud, these are my impressions as an outsider looking in:

Among other things:

1. Hardware must be allocated from the available pool of servers
2. The VHD of the core OS must be uploaded to the machine
3. A VM instance must be initialized and booted off that VHD image
4. Your application package must be copied to the VM and installed
5. The VM monitor must wait for your service to start up, or fail
6. The data center load balancer and firewall must be made aware of your application's service endpoints
7. Once all of that has synchronized, your app is accessible from the web.

The VHD image is probably gigabytes in size, much larger than your app upload. Even on a superfast datacenter network, it takes time to move that much stuff into the VM, unpack it, and boot from it. Also, the load balancer and firewall are probably optimized to make routing requests the highest priority. Reconfiguring the firewall and load balancer is lower priority, and has to be done without interrupting traffic flow.

Also note that all this work only has to be done for a new deployment. Updating an existing deployment rolls out much faster - 2 to 3 minutes instead of 20 to 30 minutes.

Solution 3

Check out this PDC10 video by Mark Russinovich. He goes into great detail on what's going on inside Azure with some insights into the (admittedly slow) deployment process.

Original link is no longer working. Here's another link to a version of the same presentation: https://channel9.msdn.com/events/Build/BUILD2011/SAC-853T

Comments

[stepanian]

This is exactly the reason I still use EC2 even though I love the abstraction away from the OS administration that Azure provides.

[staccata]

In the meantime we are almost 3 years further and still very slow.

[Casey]

@staccata Web sites are way faster to deploy so use them if you can.

[rks]

Tough to fathom that 5 years have gone by but the start-up time is still the same.

[hoangpx]

now is end of 2017 and this is still super slow. Sometimes I think about unboxing a computer then install os then deploy app manually is faster than this.

[sr9yar]

End of 2018. I've just registered yesterday, and I could resize the pool in 5 minutes. Today I'm resizing to 1 dedicated node on a custom image, it's been over 30minutes already. :( This does seem like some voodoo is going on here.

[Berkant İpek]

2019 is nearly over and even a simple Windows 10 virtual machine provision still takes 8-10 minutes.

[João dos Reis]

At the end of 2019, provisioning a new instance of APIM has just crossed the 35 minute mark.

[Nathan Basanese]

// , It's a business decision. They're not selling to devs.

[frackham]

coming towards end of 2020, az cognitive search service taking >1hr. Always completes.

[Jeroen Wiert Pluimers]

Great link, but no +1 yet: the video is more than one hour, so it would have been tremendously handy if you had posted a slight summary of the hightlights. If you do: +1 from me.

[Jamie Keeling]

To be fair BrentDaCodeMonkey did say he goes into "great detail".. a good video nonetheless!

[Andre Pena]

Wow.. This video is a Windows Azure deployment long.

[BrentDaCodeMonkey]

Admittedly, it was a fairly old (in cloud terms) recording. I've updated with a new link to the same session by Russinovich from a different event.

[afr0]

Interesting points, but considering we have the fastest of every options available these days. Shouldn't this allow us do all this in seconds if not ms?

[dthorpe]

"These days" is now 5 years after when my original comment was written. I'm sure Azure provisioning performance has improved since 2011. :>

[afr0]

still not in seconds I can confirm that

[philippeback]

definitely not.

[Fls'Zen]

Yeah, still takes like 10 minutes for a new deployment. Honestly, I'm a little surprised Azure doesn't keep some VMs waiting at step 4 in this list to improve their customer experience.

[Gravy]

Is this answer still appropriate for 2018? I am trying to deploy a redis cache on Azure. My first service on azure - been waiting for 25 mins now and can't believe that this is normal / acceptable.

[Cocowalla]

@Gravy I've been using a lot for years now, and yes, deployments (especially of VMs) can at times be painfully slow. Generally you're looking between 10 and 60 minutes.

[Nick Turner]

@BrentDaCodeMonkey awesome answer! no guessing