RamMap "Empty standby list" drastically speeds up PC. Is there a better way to do this?

Question

Many people are educated on how Windows Memory manager caching works by reading Microsoft white papers, etc. However, there are relatively few documents that describe real-world performance issues (sluggishness) related to very large/growing mapped files and standby lists.

I have a Win7 64bit (8Gb RAM) PC which acts as a server. I can't add any more memory to it. It has a bunch of TV tuners which record/write several simultaneous gigantic mpeg2 files (at least 3-8GBs each) to a HDD 24/7. Depending on how busy my PC is recordings TV shows, after 1-3 days, my PC becomes sluggish.

By just looking at Windows Task Manager, it appears that I don't have very many large applications open; and, plenty of "available memory". Even if I close all active processes, the PC remains sluggish. However, when using RamMap, I can see that these large mpeg2 files become mapped files in the active standby list; which over time make my PC very sluggish. If I select "Empty standby list" in RamMap, my PC immediately becomes extremely responsive; as if I just rebooted it. There is nothing else I can do to address sluggishness. In fact, the PC will never recover from this sluggishness unless I empty the standby list or reboot.

I have a command-line utility, EmptyStandbyList.exe, which I found that quietly purges the standby list instead of having to do that in RamMap's GUI. However, unfortunately it also purges smaller items in the standby list that are very useful for performance purposes to stay in the standby list.

While searching on Google, I've found other people who have almost the same exact issue as me. However, most of their questions go unanswered; with no real explanation to why this happens; and, no definitive way to fix the issue other than the "Empty standby list" technique.

I've seen mentions of a MS utility/service called "Dynamic Cache"; but not sure if it cures specifically what I want. Also, the Win7/Win2008 R2 version of it seems to be only available via MS support. I also saw a couple of utilities which set the "min/max NT cache limits"; but those look like they're specific to Working sets for processes, not standby lists.

I'm hoping someone who can think outside-the-box can think of a more graceful cure for me instead of me having to use brutally empty the entire standby list regularly; and, hopefully have a definitive explanation to the symptom me (and other people) are seeing. The reason I mentioned "outside-the-box" is because there are already plenty of people who like to just recite how mapped files and standby list priorities "should" work on paper; which isn't very helpful for me.

Solution 1

The Standby list is the Superfetch cache and the "Dynamic Cache" fixes a different issue. The service fixes an issue where the NTFS Metadata cache is too high. But Standbylist and this Cache are 2 different things. The NTFS Cache is shown as used Memory, while the Standby-Cache is contains data, but can be emptied very fast if you really new that Memory to store data in it.

On Sysinternals Forum the user wj32, the author of Process Hacker, created a command line tool which does what you want:

http://forum.sysinternals.com/rammap-empty-standby-list_topic27297_post132769.html#132769 (download here)

Try it if it "fixes" your issue.

Solution 2

Could it be that the sluggishness comes from Windows forcing stuff to be loaded/saved from the Swap drive, that with every video taped, the standby list grows and with a big Swap drive other (background) programs now get pushed to the Swap drive, slowing down the machine.

If this PC is only used for taping TV shows, maybe a solution could be to make the Swap drive as small as Windows allows it to be set, which is I believe 16MB. Do not turn the Swap drive totally off, because then Windows will make a Swap drive in secret.

Some how I can not imagine that Windows would get sluggish, if it was just reading data from RAM, while no new programs get loaded from Disk.

Qoute from MS document http://download.microsoft.com/download/7/E/7/7E7662CF-CBEA-470B-A97E-CE7CE0D98DC2/MemorySizingGuidanceWin7.docx :

"Windows 7 SuperFetch™ works with the memory manager to set priorities for pages on the Standby list. Pages that have recently been placed on the Standby list start with a high priority, which slowly decreases over time.

If the system has too little memory to handle the workload, only a small percentage of the pages on the Standby list have a low priority. By monitoring the number of pages at different priorities, you can determine whether the system would benefit from increasing the amount of installed RAM."

Overall, the goal is to ensure that the system can maintain recently referenced pages in memory and still have enough available memory to satisfy any immediate memory needs."

Maybe also the TV recording program, as it keeps running and doesn't stop has every Page that goes to the Standby List marked as highest priority pushing other stuff out, that would normally speed up Windows. You would think though that stuff gets released when memory gets low, and/or when there is no Swap file that Windows can't secretly start to heavily use the Swap drive slowing things down. This is on the assumption that data in the Standby List will not be loaded to the Swap drive.

Comments

[I say Reinstate Monica]

I realize this is an old post and that this suggestion could be difficult to test, but if someday you get the opportunity you may want to try your setup with a server edition of Windows. Perhaps there's yet more undocumented under-the-hood differences in how a server OS handles memory that would work to your benefit.

[Karan]

Isn't that command line tool the exact same EmptyStandbyList.exe that the OP mentioned in his post, and for which he wants a better, perhaps more focussed alternative?

[magicandre1981]

yes, but you can run it from a scheduled task or via shortcut from the desktop.

[Karan]

Uhh... Did you read the line that starts "However, unfortunately...", and the fact that he wants "a more graceful cure"?

[user22667]

Its nice to know there people who take the time to read the OP a bit more carefully before posting. However, I did learn a couple things I didn't know before. It sounds like what's actually causing the performance issue is related to Superfetch cache; not NTFS Metadata cache. RamMap shows both. I know under Windows vista, I had to completely turn off the Superfetch service because it didn't know how to handle TV recordings (from my DVR app); and, was causing excessive use of my HDD. I currently have the Win7 superfetch service to enhance bootup and applications I use in memory.

[user22667]

I'm using an SSD for my boot partition; which is only 256GB's in size. I'm curious if there are any settings in superfetch would would cure this issue; such as, suppressing certain file types from superfetch? Hopefully, someone who knows how this all works make explain why this is happening and offer an fix thats better than what I have.

[user22667]

BTW: It looks like the Superfetch service is responsible for both writing prefetch data to disk as well as the keeping data in memory. My Superfetch service was set to manual and not started. I'm curious if that's related to the symptom I'm seeing.

[magicandre1981]

if the service is set not started you should only have a small standbylist. Which issues do you have when you have a large standbylist?

[user22667]

I'm testing now. What I noticed immediately is the C:\Windows\Prefetch directory is now actively getting populated with files. Responsiveness of apps loading into memory is better and Windows bootup is faster. Hopefully, the problem I had was caused by this service being set to manual (which would never start).

[magicandre1981]

ok, so the issue is that Superfetch was disabled. This is expected. A disabled service and an emptied standby list slows down Windows.

[user22667]

Just to be clear: Whether disabling the Superfetch service or disabling via: [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\PrefetchParameters] "EnableSuperfetch"=0 does not suppress Standby list functionality. Only manually purging the standby list cures the problem.

[magicandre1981]

yes, the standbylist is still used,but not so much. But again, I still don't understand why this causes any issues for you.

[user22667]

Yes, I dont know either. All I know is there's an associated general system sluggishness symptom after large file standbylist activity; especially files that grow from very small to very large over time (such as video encodings, video DVR recordings, etc). I dont know of a tool that can expose what resource might be depleted due to this. Emptying the standby list completely cures the problem instantly. There are other people who can reproduce this exact behavior; and, end up using the same cure as me; without knowing why.

[magicandre1981]

ok, use the commandline tool I linked and create a scheduled task which runs every hour (adjust the time to your needs).

[BeowulfNode42]

emptying the standby list without changing the ratios/limit settings of the memory manager, simply prompts windows to start filling up the now free memory. This also gets rid of the actual useful stuff that's in the standby list as well and both of these things reduce the speed of your machine from what it is capable of doing.