WebClient generates (401) Unauthorized error

Question

I have the following code running in a windows service:

WebClient webClient = new WebClient();
webClient.Credentials = new NetworkCredential("me", "12345", "evilcorp.com");
webClient.DownloadFile(downloadUrl, filePath);

Each time, I get the following exception

{"The remote server returned an error: (401) Unauthorized."}

With the following inner exception:

{"The function requested is not supported"}

I know for sure the credentials are valid, in fact, if I go to downloadUrl in my web browser and put in my credentials as evilcorp.com\me with password 12345, it downloads fine.

What is weird though is that if I specify my credentials as [email protected] with 12345, it appears to fail.

Is there a way to format credentials?

Solution 1

webClient.UseDefaultCredentials = true; resolved my issue.

Solution 2

For me, 'webClient.UseDefaultCredentials = true;' solves it only on local, not in the web app on the server connecting to another server. I couldn't add the needed credential into Windows as a user, but I found later some programming way - I won't test it as I already made own solution. And also I don't want to mangle with the web server's registry, even if I have the needed admin rights. All these problems are because of the Windows internal handling of the NTLM authentication ("Windows Domain") and all of libraries and frameworks built over that (e.g. .NET).

So the solution for me was quite simple in idea - create a proxy app in a multiplatform technology with a multiplatform NTLM library where the NTLM communication is created by hand according to the public specs, not by running the built-in code in Windows. I myself chose Node.js and the httpntlm library, because it's about only one single source file with few lines and calling it from .NET as a program returning the downloaded file (also I prefer transferring it through the standard output instead of creating a temporary file).

Node.js program as a proxy to download a file behind the NTLM authentication:

var httpntlm = require('httpntlm');         // https://github.com/SamDecrock/node-http-ntlm
//var fs = require('fs');

var login = 'User';
var password = 'Password';
var domain = 'Domain';

var file = process.argv.slice(2);           // file to download as a parameter

httpntlm.get({
    url: 'https://server/folder/proxypage.aspx?filename=' + file,
    username: login,
    password: password,
    workstation: '',
    domain: domain,
    binary: true                            // don't forget for binary files
}, function (err, res/*ponse*/) {
    if (err) { 
        console.log(err);
    } else {
        if (res.headers.location) {         // in my case, the server redirects to a similar URL,
            httpntlm.get({                  // now containing the session ID
                url: 'https://server' + res.headers.location,
                username: login,
                password: password,
                workstation: '',
                domain: domain,
                binary: true                // don't forget for binary files
            }, function (err, res) {
                if (err) { 
                    console.log(err);
                } else {
                      //console.log(res.headers);
                      /*fs.writeFile("434980.png", res.body, function (err) {  // test write
                          if (err)                                             // to binary file
                              return console.log("Error writing file");
                          console.log("434980.png saved");
                      });*/
                      console.log(res.body.toString('base64'));  // didn't find a way to output
                }                                                // binary file, toString('binary')
            });                                                  // is not enough (docs say it's
                                                                 // just 'latin1')...
        } else {           // if there's no redirect
            //console.log(res.headers);                          // ...so I output base64 and
            console.log(res.body.toString('base64'));            // convert it back in the caller 
        }                                                        // code
    }
});

.NET caller code (the web app downloading files from a web app on another server)

public static string ReadAllText(string path)
{
    if (path.StartsWith("http"))
        return System.Text.Encoding.Default.GetString(ReadAllBytes(path));
    else
        return System.IO.File.ReadAllText(path);
}

public static byte[] ReadAllBytes(string path)
{
    if (path.StartsWith("http"))
    {
        ProcessStartInfo psi = new ProcessStartInfo();
        psi.FileName = "node.exe";                     // Node.js installs into the PATH
        psi.Arguments = "MyProxyDownladProgram.js " + 
            path.Replace("the base URL before the file name", "");
        psi.WorkingDirectory = "C:\\Folder\\With My\\Proxy Download Program";
        psi.UseShellExecute = false;
        psi.CreateNoWindow = true;
        psi.RedirectStandardInput = true;
        psi.RedirectStandardOutput = true;
        psi.RedirectStandardError = true;
        Process p = Process.Start(psi);
        byte[] output;
        try
        {
            byte[] buffer = new byte[65536];
            using (var ms = new MemoryStream())
            {
                while (true)
                {
                    int read = p.StandardOutput.BaseStream.Read(buffer, 0, buffer.Length);
                    if (read <= 0)
                        break;
                    ms.Write(buffer, 0, read);
                }
                output = ms.ToArray();
            }
            p.StandardOutput.Close();
            p.WaitForExit(60 * 60 * 1000);             // wait up to 60 minutes 
            if (p.ExitCode != 0)
                throw new Exception("Exit code: " + p.ExitCode);
        }
        finally
        {
            p.Close();
            p.Dispose();
        }
        // convert the outputted base64-encoded string to binary data
        return System.Convert.FromBase64String(System.Text.Encoding.Default.GetString(output));
    }
    else
    {
        return System.IO.File.ReadAllBytes(path);
    }
}

Solution 3

According to the msdn docs the exception could be because the method has been called simultaneously on multiple threads. The DownloadFile method also requires a completely qualified URL such as http://evilcorp.com/.

Comments

[lzlstyle]

Most of cases need this setting are that the server need to get user's identify information, like the ASP.Net website configing the Windows authentication.

[KDT]

Brian if you were closer I swear I would've kissed you right now!! Been battling with this for hours and missed only this setting!? :-o THANKS!!

[FastTrack]

Definitely helped me out too. Thank you Brian

[vwfreak]

Why is this the case? Doesn't this mean that the WebClient will be using the credentials of whoever is running the app? This solution does work for me, but I am writing this console app for someone else, and I want it to work when another user runs it, even though he might not have the correct permissions himself.