Add a customized header to Webbrowser control for all requests in C#?

Question

I'm using C# and .NET 3.5. As you know, we can add custom header to Navigate() on the web browser control like this:

var myUrl = "http://example.com/mypage.htm";
System.Uri uri = new Uri(myUrl);

byte[] authData = System.Text.UnicodeEncoding.UTF8.GetBytes("user:password");
string authHeader = 
    "Authorization: Basic " + Convert.ToBase64String(authData) + "\r\n" +
    "User-Agent: MyUserAgent\r\n";

webTDW8961nd.Navigate(uri, "", null, authHeader);

In the example above we set a Basic Authorization header for a single navigation.

Now let talk about redirection. If we want to execute javascript which will redirect to another page, the Basic Authorization header won't be included.

What is your solution? How can I add a header which works for all of the requests and not only once?

Solution 1

The problem is that while both WinForm's and WPF's WebBrowser are nothing else but a relatively thin wrapper around the ActiveX IE control, they don't expose all the events of interest to us (and the second provides even less than the first). There are two ways to solve this: first, to subclass the WF browser control and add what you need or to use the WPF one and add the hooks there. I found the second approach to be more convenient in a WPF application.

You only need the relevant interfaces. The easiest way is to add a reference to Microsoft Internet Controls (you'll find this under the COM heading is VS). This opens up a namespace called SHDocVw that contains all we need (if, for any reason, you want to get rid of this dependency, you can simply copy the P/Invoke interfaces used into your own code).

You can get the underlying browser using reflection. It will return null if you call it too early, so I put it into the WebBrowser.Navigating handler:

using SHDocVw;
var ActiveXInstance = (IWebBrowser2)Browser.GetType().InvokeMember("ActiveXInstance", BindingFlags.GetProperty | BindingFlags.Instance | BindingFlags.NonPublic, null, Browser, new object[] { });

As soon as you have it, you can do nice things with the browser. For instance, you can use various properties and methods not exposed directly:

ActiveXInstance.Silent = true; // suppresses script error dialogs

and add the missing event hooks:

var SetupEvents2 = (DWebBrowserEvents2_Event)ActiveXInstance;
SetupEvents2.BeforeNavigate2 += OnBeforeNavigate2;

There are two event interfaces, the 2 variant contains the newer events. You can look all this up on MSDN.

And, back to the headers: the BeforeNavigate2 event allows you to put your extra headers into the provided object:

private void OnBeforeNavigate2(object pDisp, ref object URL, ref object Flags, ref object TargetFrameName, ref object PostData, ref object Headers, ref bool Cancel) {
  Headers = $"Accept-Language: XX;en\r\n";
}

Solution 2

To add custom header for each request you can implement extension method:

public static class WebBrowserExtensions
{
    public static void NavigateWithAuthorization(this WebBrowser browser, Uri uri)
    {
        byte[] authData = System.Text.Encoding.UTF8.GetBytes("user:password");
        string authHeader = "Authorization: Basic " + Convert.ToBase64String(authData) + "\r\n" + "User-Agent: MyUserAgent\r\n";
        browser.Navigate(uri, "", null, authHeader);
    }
}

And then call it instead of standard method:

//browser.Navigate(uri, "", null, authHeader);
browser.NavigateWithAuthorization(uri);

The second question is about redirecting. But your scenario will not work in a simple browser and fiddler. It is the feature of web protocol, when you redirect to another Uri you initiate new request with new properties. You can compose your request in js code.

Comments

[asDca21]

Shouldn't it be "WebBrowserExtensions : WebBrowser"? How should I use it?

[Vadim Martynov]

No it should not because I suggest to use extension methods - I add the whole code you need. 1. Add new class with code from answer. 2. In a place where you want to use new method add using derictive with namespace of your new class. 3. Just call method as in the example. See also msdn.microsoft.com/en-us/library/bb383977.aspx

[asDca21]

I used it and it didn't work for JavaScript based redirector for example when the script submits a form.

[Vadim Martynov]

Do you want add this behavior for user interaction requests too? As I understood from the last comment - yes

[asDca21]

The page is completely fixed. There is a script in the page and by executing that, a form get submitted automatically. User has nothing to do with Webbrowser.

[Vadim Martynov]

Well as we can see in .net sources there is no way to override standard navigation pipeline. But there is a hardcore way with implementing your own WebBrowserBase child class and paste code from .net. Are you sure it is not X-Y problem? What is your scenario that you want to implement?

[asDca21]

I just want to get into modems UI (192.168.1.1) using Basic Authorization (here is no problem) and submit a form which actions is a page that require Basic Authorization too (and here is the problem!).

[Vadim Martynov]

But in a normal way (when you use chrome or opera or IE) your browser doesn't generate request request with authorization but using cookies. Why don't you want to use it?

[asDca21]

O.o using cookie as Basic Authorization? I have sniffed packets to the UI and found out that each time browser requests a page or submit something, it includes Authorization header. Even with cookies, doesn't Webbrowser send them automatically?

[Vadim Martynov]

If you are already authoricated then yes, you can read more on wiki. Because the BA field has to be sent in the header of each HTTP request, the web browser needs to cache credentials for a reasonable period of time to avoid constantly prompting the user for their username and password. Caching policy differs between browsers.

[asDca21]

I used HttpWebRequest and response to get cookies but server didn't give me any cookie! (there were no problems with the codes)

[Vadim Martynov]

What is your current goal? You said that each time browser requests a page or submit something, it includes Authorization header. Is it about your app browser or not?

[GantTheWanderer]

This wonderful answer led me to a solution for my problem. The only difference is that modifying Headers did not work for me, the headers just never made it to the actual request. As a workaround, I checked the URL and Headers to see if the specific header I needed was missing to the call to a specific URL I needed it to be passed to. if (!(Headers as string).Contains("X-MyHeader") && (URL as string).EndsWith("/UrlThatNeedsTheHeader")) If the header was missing, I used Browser.Stop() to end the navigation. Then recalled Browser.Navigate passing in my headers again.

[Reza Aghaei]

For those who want to set user agent, there is a better solution. You can use the solution which is shared at bottom of this post: Winform Webbrowser being recognized as a mobile device. It works well even when you refresh the browser.