Best way to block Windows Workstations from receiving Microsoft updates

Question

I have a network with 10,000 workstations that are all currently trying to download SP3 and are flooding the network (this was do to an errant AD policy change). What is the best way to block this traffic at the network level with either firewalls, Web Content Filtering or inline IPS? Is there a good list of IP addresses out there that is current for Microsoft update servers?

The trick is that we have to do this quickly since the network is flooded. We can't wait for a policy change...

Solution 1

Have users on a VLAN that doesn't route directly to the internet and use ISA or your open source caching server of choice.

Faster less intrusive options:

• Update AD so that computers don't auto update
• Temporary update to the dns server so that their download doesn't resolve
• Temporary update to kill requests to the windows update server at the firewall

Fastest to slowest

1. Firewall rule
2. DNS change
3. Update AD settings so that updates don't self install
4. Isolate users from internet with routing so that they have to use a proxy server.

Personally I like day to day workings of environment 4, with 3 handling the updates but getting them all from something like WSUS. 2 and 1 are hack fixes not meant for more than trying to bandaid an outage cause until it is properly fixed.

Possible domains to try and block

• windowsupdate.microsoft.com
• windowsupdate.microsoft.com
• v4.windowsupdate
• www.windowsupdate.com
• download.windowsupdate
• wustat.windows.com
• officeupdate.microsoft.com
• office.microsoft.com
• crl.microsoft.com
• download.microsoft.com

That list is probably not complete, I'd recommend you do a port mirror of a trunk or user vlan into a laptop running wireshark and you filter for DNS requests or the filename in question.

Get the IP addresses associated with the dns records and block them at the firewall. This will probably cause issues with some peoples work, but it will let your WAN/Inet connection be usable again for the many that don't need those sites. If you have a firewall that supports blocking by DNS instead of IP all the better. Your IPS may support responding to the file requests on a per domain basis, if so that would also work.

This is definitely NOT something I would ever suggest someone do when not requiring network triage, ever.

Solution 2

Sounds like you don't test your group policy changes in a lab first, eh?

Create a GPO with settings the "Configure Automatic Updates" setting in the "Windows Update" node of the "Windows Components" node of "Administrative Templates" set to "Disabled". As long as you haven't done something to disable periodic policy refresh, which happens every 90 to 120 minutes, by default, the clients will pick up the change w/o needing a reboot.

If you have disabled background policy refresh or if you can't wait, break out a copy of "psexec" and start running "gpupdate /force" on clients after you make the above change. (Blocking background policy refresh seems like a really bad idea...)

Blocking this at layer 3 is going to be difficult because the Windows Update service is DNS load-balanced. I don't know that you can easily get a list of IP addresses.

You might get by with creating DNS zones "windowsupdate.com" and "update.microsoft.com" in the DNS servers your clients use with no records in it. Clients with cached DNS lookup results will be unaffected.

Alternatively, break out a copy of "psexec" and execute a little script on each client to stop the "wuauserv" service and mark it "Disabled". That'll stop it, too.

You really should be using WSUS, BTW.

---

Assuming a list of computer names in "computers.txt" (which you can get by exporting the results of a "Find..." operation from Active Directory Users and Computers to a file and cleaning it up w/ Notepad):

@echo off
for /f %%i in (computers.txt) do (
 start sc \\%%i stop wuauserv
 start sc \\%%i config wuauserv start= disabled
)

This is a persisent change to PCs, but you can undo it with "Services" settings in a GPO later.

With that many PCs, one would assume that they're broken down into many geographic areas. You should probably distribute running such a script across servers that are located in the geographic areas closest to groups of computers.

If you wanted to get fancy you could use dsquery or Joe Richards' (http://www.joeware.net) adfind to export the lists of computers from AD. (You could also export lists of computers out of DHCP servers using the Support Tool dhcpcmd, if that was a better way to get geographic lists...)

Solution 3

If AD policies still work, you can either point workstations to WSUS or block windows or auto updates through GPO.

Also, MS has Windows Service Pack Blocker Tool Kit which can block SPs downloading.

If that is not an option - start with blocking *.windowsupdate.microsoft.com at the firewall. Do not remember all the addresses, not too many - 3 or 5.

Solution 4

above script tweaked when using

net view > c:\temp\comps.txt

for /f "skip=3 tokens=1 delims= " %%i in (c:\temp\comps.txt) do ( start sc %%i stop wuauserv start sc %%i config wuauserv start= disabled )

you could even use .... %%i in ('net view') ( ...

and it will work. ( i use file pipe method rather as i usually edit out a few stations )

we used this to temporarily stop WSUS server connections while troubleshooting.

ran here on a 250 station network and it took about 30 seconds.

good luck

ps. Now if Adobe would move to a service based update process i would die happy.. the registry hacking to stop adobe from updating is silly.

View more solutions

Comments

[joiner]

No good, the systems are flooding the network now and workstations will not get a policy update until they are rebooted.

[DanBig]

Why the downvote?

[joiner]

I like the DNs server idea. The firewall is what I am really looking for though...but mores specifically a complete listing up update servers that we can block (we have to use IPs, no DNS blocking allowed).

[Dave Klotz]

Down vote seemed silly, especially when you can tell users there will be a lunch time reboot.

[Oskar Duveborn]

Why not start with just blocking all outbound traffic that is not coming from important servers that may need it? Users surfing the net should be the least of your worries I figure ^^

[DanBig]

+1 agree on all points.

[joiner]

you guys are right about the downvote so I changed it....

[Oskar Duveborn]

The IP addresses are most likely changing often due to caching, load balancing services and so on...

[joiner]

Thanks, and just so I dont look like an idiot, this was a customer who called me for help. They do have WSUS and have been doing this correectly for years now. I don't know how they FOOBARRED it, but as you can imagine an org with 10,000 workstations would not have grown to that size without AD policies and WSUS in place.

[Dave Klotz]

Your goal is triage at the point of 10k users trying to download, kill it with fire. Block all outgoing from non-servers like oskar said, then slowly open things back up. Some of our juniper firewalls I know have done filtering based on a DNS name, but I know that isn't commonly supported and may have just been a feature of the management software used.

[Spence]

@TrevJen: I'd like to see that I haven't seen organiations nearly that large who don't use WSUS and use Group Policy for almost nothing, but I have. It doesn't shock me anymore at all. Large organizations are just as likely to make bone-headed decisions as small ones (if not more so, because they get all wrapped up in being "enterprisey" and "corporate" and having meetings about having meetings). Vast numbers of Windows admins seem to know next to nothing about Group Policy and how automation can save them effort. The "state of the art" in Windows network administration is really quite bad.

[Dave Klotz]

I knew there was a scripted way to do it but couldn't remember if you had to dig around in the registry or not. Powershell of vbs should take care of that decently quickly. The idea of 10k+ users and no WSUS is scary.

[Spence]

@TrevJen: Lastly, I forgot to say that it's spelled "FUBAR". It's an acronym for "...-up beyond all recognition". I'll leave it to you to figure out the rest...

[Dave Klotz]

@Oskar - even with load balanced IPs changing you should still be able to monitor for requests and dig the records up from the servers to know what they are. Although I've heard you end up blocking 5-6 class Cs worth of IPs when trying to do this. Someone who scripts a lot will fix it with a script, someone who isn't confident in scripting will probably kill it as quickly as they can with their firewall and start to fix the problems in AD.

[MrGigu]

Testing in a lab is nice, but when you have your superiors screaming for blood, bad things can happen in live environments.

[Spence]

@Farseeker: Some VM-based Windows XP machines in an OU all to themselves is better than nothing, and shouldn't cost a lot of money or time to implement. Given the potential costs of a screwup, it only makes sense. Unreasonable demands from management are just that, and I suppose that's yet one more reason why I work strictly as a contractor. If I don't like the Customer's demands, I walk.