Azure VM can't access internet

Question

I've set up some windows VMs and and a single linux VM on azure and none of them can access the internet.

None of the VMs have a public IP addresses. I can RDP into the windows boxes/ssh into the linux box due to inbound NAT rules on a load balancer.

I've applied Network Security Groups to the NI of each VM (there is no NSG on the VNet itself). These NSGs allow outbound internet traffic. To test this I've used Network Watcher's "IP Flow Verify" test which shows that the traffic isn't being blocked by the NSG rules. I've also used the Network Watcher's "Connectivity Troubleshoot" which shows that the connection form any of the VMs to an internet IP address is "unreachable" but shows no issues as a cause.

I can resolve domain names (using "host" on linux, for example) so I don't believe this to be a DNS issue.

Why can't I access the internet from these VMs? Thanks for looking.

---- Edit 1 ----

This is the output I see from Network Watcher's "Connection Troubleshoot":

---- Edit 2 ----

Removing the NSG of the NI on the VM in question doesn't help. I've double checked that I have no NSG on the VM's subnet.

The inbound/outbound rules of the NSG are

The subnet in question has "NAT gateway" set to "none" (and, indeed, cannot be set to anything else).

Network Watcher's "IP Flow Verify" claims the NSG isn't to blame:

Solution 1

The answer is contained in this LB outbound connection MS doc page.

Outbound connectivity, for vnets with a standard* SKU load balancer does not exist automatically: the VMs requiring outbound access must either be given an external facing IP address, or be put into a backend pool of the load balancer AND the load balancer must have an outbound rule defined for the required outbound traffic on that backend pool (even if the rule merely accepts the default options).

* For a basic SKU load balancer this isn't necessary.

Solution 2

All VM's without a public IP has connectivity to internet, even when you haven't associated a NSG to subnet/NIC. Once you have associated it, you can test through Network Watcher to make sure everything is "green" like images below, please confirm you did these tests like so.

In your case you said "Connection troubleshoot" showed unreachable. You can have a look effective routes of VM's NIC. Make sure default route 0.0.0.0/0 points to internet and there is no wrong route.

Connection troubleshoot shows Reachable

Here Next hop shows internet

And finally IP flow shows Access Allowed

Comments

[eff]

Hi Taguada, thanks for the response. The "next hop", "ip flow verify", and "Effective routes" are similar to yours - they show access is allowed and the route to the internet is seen. However, as in the original question, the route is still "unreachable" according to the Network Watcher's "Connection troubleshoot". Any idea how I can go about investigating that further?

[Taguada]

Just in case, remove temporally your current NSG from subnet and NIC where this VM is running and then create a new NSG and associate it to your VM's NIC, keep it as default rules, but default it has AllowVnetOutBound. I can try to reproduce your issue on my side, please let me know what is your address space of VNET and VM's private IP. There is a Azure NAT gateway that allows outgoing internet for a private VNET (docs.microsoft.com/en-us/azure/virtual-network/…), but your scenario should work without it. I'm running some tests here and update your soon.

[eff]

Swapping to a new, clean, NSG doesn't change the situation, sadly, but thanks for the suggestion. The new NSG had both AllowVnetOutBound and AllowInternetOutBound rules. My VNet is 10.0.0.0/16, with the subnet containing the VMs being 10.0.0.0/20. The VM's ip is 10.0.0.23. Many thanks for your efforts, but I would have assumed it was something that I've done that I haven't thought of (and so haven't mentioned here) that's blocking this - as you say, the typical defaults allow internet outbound (at present I can't apt-get update nor windows update).

[Taguada]

This case is very interesting :) The only way to block traffic from your VM to internet is to use NSG, of course if you have done something inside VM it can also block. I cannot reproduce it on my side. Can you update your question with your NSG rules (hide any critical information)? Are you sure you have only one NSG? For example you have a NSG associated to your NIC and other one associate to SUBNET. Remove all NSG of this VM (subnet / NIC), VM by default is able to access internet like this.

[Taguada]

I created the same scenario, nat rule, load balacing, nsg, vnet, ip, vm and son. Everything worked fine. My last shoot for this case, here you can find many tests which can be done (docs.microsoft.com/en-us/azure/virtual-network/…). In case that anything works, you should open a ticket to Microsoft. For more test, I should access your VM, Azure, run tcpdump, capture packet and many other option, but it is not the case. I suggest you to try to figure out how to make sure that packets are flowing out from your VM to the destination.

[eff]

I had a standard LB which did inbound NAT but wasn't aware the outbound traffic required backend pools and outbound rules and, as such, hadn't defined them. Adding VMs to backend pools and defining outbound rules fixed the issue.

[eff]

Hi @Taguada, as in my answer below, the issue was that I hadn't placed my VMs in a backend pool of my LB, nor defined an outbound rule for said (non-existant) pool. This is only an issue for standard SKU LBs, so if you'd set up with a basic LB you wouldn't have seen it. I'm a little surprised the behaviour changes from basic->standard but mostly relieved to have it sorted! Many thanks for your investigations.