Caching and the WebBrowser control in .Net

Question

I'm using the WebBrowser control in .Net to execute some 3rd party affiliate marketing conversions.

I have a queue table in a database with all the scripts/images to execute. I loop through all these in a WinForms app with the WebBrowser control. After I've executed a script/image I Dispose the WebBrowser control, set it to null, and renews it with a new WebBrowser control instance.

Consider this URL: http://renderserver/RenderScript.aspx?id=1

RenderScript.aspx displays an Image with a URL of e.g.: http://3rdparty/img.ashx?id=9343

I use Fiddler to see all requests and responses, and when the same URL is executed twice, it uses some kind of cache. That cache exists underneath the WebBrowser control itself.

This cache means that the img.ashx is not called.

I tried using Internet Explorer to request the URL: http://renderserver/RenderScript.aspx?id=1 and hit F5. Then it is requested perfectly.

But if I click the address bar and hits Enter to navigate to the same URL again - it is not requested. When I use Firefox is will request the page and image everytime no matter if I use F5 or navigate from the address bar.

I found some Win32 API calls (http://support.microsoft.com/kb/326201) that was able to clear the cache. It worked on my local machine. Then the app was deployed to a server running Windows Server 2003 Standard x64 (my own machine is Vista x86).

And now the API calls to clear the cache doesn't work.

Any ideas on why the API calls doesn't work on Windows Server, but works on Vista? Both machines running IE8.

Solution

I had the same problem (quite) a while back. Microsoft has a page that was very helpful with this:

http://support.microsoft.com/default.aspx?scid=http://support.microsoft.com:80/support/kb/articles/q326/2/01.asp&NoWebContent=1

I created a class out of Microsoft's sample, however I also had to add a couple if statements to stop processing when there are no more items; it's been a while now, but I'm pretty sure it would throw an error (see ERROR_NO_MORE_ITEMS in the code below).

I hope its helpful!

 using System;
 using System.Runtime.InteropServices;

 // copied from: http://support.microsoft.com/default.aspx?scid=http://support.microsoft.com:80/support/kb/articles/q326/2/01.asp&NoWebContent=1

 namespace PowerCode
 {
     public class IECache
     {
         // For PInvoke: Contains information about an entry in the Internet cache
         [StructLayout(LayoutKind.Explicit, Size = 80)]
         public struct INTERNET_CACHE_ENTRY_INFOA
         {
             [FieldOffset(0)]
             public uint dwStructSize;
             [FieldOffset(4)]
             public IntPtr lpszSourceUrlName;
             [FieldOffset(8)]
             public IntPtr lpszLocalFileName;
             [FieldOffset(12)]
             public uint CacheEntryType;
             [FieldOffset(16)]
             public uint dwUseCount;
             [FieldOffset(20)]
             public uint dwHitRate;
             [FieldOffset(24)]
             public uint dwSizeLow;
             [FieldOffset(28)]
             public uint dwSizeHigh;
             [FieldOffset(32)]
             public FILETIME LastModifiedTime;
             [FieldOffset(40)]
             public FILETIME ExpireTime;
             [FieldOffset(48)]
             public FILETIME LastAccessTime;
             [FieldOffset(56)]
             public FILETIME LastSyncTime;
             [FieldOffset(64)]
             public IntPtr lpHeaderInfo;
             [FieldOffset(68)]
             public uint dwHeaderInfoSize;
             [FieldOffset(72)]
             public IntPtr lpszFileExtension;
             [FieldOffset(76)]
             public uint dwReserved;
             [FieldOffset(76)]
             public uint dwExemptDelta;
         }

         // For PInvoke: Initiates the enumeration of the cache groups in the Internet cache
         [DllImport(@"wininet", SetLastError = true, CharSet = CharSet.Auto, EntryPoint = "FindFirstUrlCacheGroup", CallingConvention = CallingConvention.StdCall)]
         public static extern IntPtr FindFirstUrlCacheGroup( int dwFlags, int dwFilter, IntPtr lpSearchCondition, int dwSearchCondition, ref long lpGroupId, IntPtr lpReserved );

         // For PInvoke: Retrieves the next cache group in a cache group enumeration
         [DllImport(@"wininet", SetLastError = true, CharSet = CharSet.Auto, EntryPoint = "FindNextUrlCacheGroup", CallingConvention = CallingConvention.StdCall)]
         public static extern bool FindNextUrlCacheGroup( IntPtr hFind, ref long lpGroupId, IntPtr lpReserved );

         // For PInvoke: Releases the specified GROUPID and any associated state in the cache index file
         [DllImport(@"wininet", SetLastError = true, CharSet = CharSet.Auto, EntryPoint = "DeleteUrlCacheGroup", CallingConvention = CallingConvention.StdCall)]
         public static extern bool DeleteUrlCacheGroup( long GroupId, int dwFlags, IntPtr lpReserved );

         // For PInvoke: Begins the enumeration of the Internet cache
         [DllImport(@"wininet", SetLastError = true, CharSet = CharSet.Auto, EntryPoint = "FindFirstUrlCacheEntryA", CallingConvention = CallingConvention.StdCall)]
         public static extern IntPtr FindFirstUrlCacheEntry( [MarshalAs(UnmanagedType.LPTStr)] string lpszUrlSearchPattern, IntPtr lpFirstCacheEntryInfo, ref int lpdwFirstCacheEntryInfoBufferSize );

         // For PInvoke: Retrieves the next entry in the Internet cache
         [DllImport(@"wininet", SetLastError = true, CharSet = CharSet.Auto, EntryPoint = "FindNextUrlCacheEntryA", CallingConvention = CallingConvention.StdCall)]
         public static extern bool FindNextUrlCacheEntry( IntPtr hFind, IntPtr lpNextCacheEntryInfo, ref int lpdwNextCacheEntryInfoBufferSize );

         // For PInvoke: Removes the file that is associated with the source name from the cache, if the file exists
         [DllImport(@"wininet", SetLastError = true, CharSet = CharSet.Auto, EntryPoint = "DeleteUrlCacheEntryA", CallingConvention = CallingConvention.StdCall)]
         public static extern bool DeleteUrlCacheEntry( IntPtr lpszUrlName );

         public static void ClearCache()
         {
             // Indicates that all of the cache groups in the user's system should be enumerated
             const int CACHEGROUP_SEARCH_ALL = 0x0;
             // Indicates that all the cache entries that are associated with the cache group
             // should be deleted, unless the entry belongs to another cache group.
             const int CACHEGROUP_FLAG_FLUSHURL_ONDELETE = 0x2;
             // File not found.
             const int ERROR_FILE_NOT_FOUND = 0x2;
             // No more items have been found.
             const int ERROR_NO_MORE_ITEMS = 259;
             // Pointer to a GROUPID variable
             long groupId = 0;

             // Local variables
             int cacheEntryInfoBufferSizeInitial = 0;
             int cacheEntryInfoBufferSize = 0;
             IntPtr cacheEntryInfoBuffer = IntPtr.Zero;
             INTERNET_CACHE_ENTRY_INFOA internetCacheEntry;
             IntPtr enumHandle = IntPtr.Zero;
             bool returnValue = false;

             // Delete the groups first.
             // Groups may not always exist on the system.
             // For more information, visit the following Microsoft Web site:
             // http://msdn.microsoft.com/library/?url=/workshop/networking/wininet/overview/cache.asp            
             // By default, a URL does not belong to any group. Therefore, that cache may become
             // empty even when the CacheGroup APIs are not used because the existing URL does not belong to any group.            
             enumHandle = FindFirstUrlCacheGroup(0, CACHEGROUP_SEARCH_ALL, IntPtr.Zero, 0, ref groupId, IntPtr.Zero);

             // If there are no items in the Cache, you are finished.
             if (enumHandle != IntPtr.Zero && ERROR_NO_MORE_ITEMS == Marshal.GetLastWin32Error()) {
                 return;
             }

             // Loop through Cache Group, and then delete entries.
             while (true) {
                 if (ERROR_NO_MORE_ITEMS == Marshal.GetLastWin32Error() || ERROR_FILE_NOT_FOUND == Marshal.GetLastWin32Error()) {
                     break;
                 }

                 // Delete a particular Cache Group.
                 returnValue = DeleteUrlCacheGroup(groupId, CACHEGROUP_FLAG_FLUSHURL_ONDELETE, IntPtr.Zero);
                 if (!returnValue && ERROR_FILE_NOT_FOUND == Marshal.GetLastWin32Error()) {
                     returnValue = FindNextUrlCacheGroup(enumHandle, ref groupId, IntPtr.Zero);
                 }

                 if (!returnValue && (ERROR_NO_MORE_ITEMS == Marshal.GetLastWin32Error() || ERROR_FILE_NOT_FOUND == Marshal.GetLastWin32Error())) {
                     break;
                 }
             }

             // Start to delete URLs that do not belong to any group.
             enumHandle = FindFirstUrlCacheEntry(null, IntPtr.Zero, ref cacheEntryInfoBufferSizeInitial);
             if (enumHandle != IntPtr.Zero && ERROR_NO_MORE_ITEMS == Marshal.GetLastWin32Error()) {
                 return;
             }

             cacheEntryInfoBufferSize = cacheEntryInfoBufferSizeInitial;
             cacheEntryInfoBuffer = Marshal.AllocHGlobal(cacheEntryInfoBufferSize);
             enumHandle = FindFirstUrlCacheEntry(null, cacheEntryInfoBuffer, ref cacheEntryInfoBufferSizeInitial);

             while (true) {
                 internetCacheEntry = (INTERNET_CACHE_ENTRY_INFOA)Marshal.PtrToStructure(cacheEntryInfoBuffer, typeof(INTERNET_CACHE_ENTRY_INFOA));

                 if (ERROR_NO_MORE_ITEMS == Marshal.GetLastWin32Error()) {
                     break;
                 }

                 cacheEntryInfoBufferSizeInitial = cacheEntryInfoBufferSize;
                 returnValue = DeleteUrlCacheEntry(internetCacheEntry.lpszSourceUrlName);
                 if (!returnValue) {
                     returnValue = FindNextUrlCacheEntry(enumHandle, cacheEntryInfoBuffer, ref cacheEntryInfoBufferSizeInitial);
                 }
                 if (!returnValue && ERROR_NO_MORE_ITEMS == Marshal.GetLastWin32Error()) {
                     break;
                 }
                 if (!returnValue && cacheEntryInfoBufferSizeInitial > cacheEntryInfoBufferSize) {
                     cacheEntryInfoBufferSize = cacheEntryInfoBufferSizeInitial;
                     cacheEntryInfoBuffer = Marshal.ReAllocHGlobal(cacheEntryInfoBuffer, (IntPtr)cacheEntryInfoBufferSize);
                     returnValue = FindNextUrlCacheEntry(enumHandle, cacheEntryInfoBuffer, ref cacheEntryInfoBufferSizeInitial);
                 }
             }
             Marshal.FreeHGlobal(cacheEntryInfoBuffer);
         }
     }
 }

To use it in your code, just call:

IECache.ClearCache()

before calling the navigate methods.

Comments

[David van Dugteren]

Negative. It has nothing to do with cache headers. They're already set correctly to avoid caching. And other browsers are not caching, as I pointed out.

[EricLaw]

Sorry, but you are mistaken. WinINET/IE/WebOCs will not reuse cached responses if the appropriate response headers are set. What are the exact headers the ASHX sends? Can you send me a Network Capture (www.fiddlercap.com)?

[Nate]

Where is Utilities.Web.WebBrowserHelper.WebBrowserHelper.ClearCache()‌​; defined?

[NetMage]

Using IIS 7.5 delivering pdf files, with no cache control, WinINET set to automtically definitely doesn't re-request the pdf file when it is newer, and has been broken since IE 8 at least.

[EricLaw]

@NetMage-- you're confused. If no cache-controlling headers are present to forbid it, the standard specifies that the response may be cached and reused.

[Mohammad Fathi MiMFa]

It made a deadlock and I think should change !returnValue to returnValue at the line 'if (!returnValue && ERROR_FILE_NOT_FOUND == Marshal.GetLastWin32Error())'.