C# get or set cookies, to download content from the web using cookies

Question

I need help with cookies. I'm planing use cookies to download web content. To get the content I need to log into a website because only authorized users can download web content or files. I'm using

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();

and then I'm scraping source code, and I need to get link to file, but I can't get because I'm not authorized, so I need to set cookies. I have not worked with cookies before. How do I to do this?

Solution 1

I've created a quick little application that helps with generating web requests for me

public class HttpRequestHandler {
    private CookieContainer cookies;

    public HttpRequestHandler() {
        cookies = new CookieContainer();
    }

    public HttpWebRequest GenerateWebRequest(string url) {
        HttpWebRequest request = (HttpWebRequest)WebRequest.Create(new System.Uri(url));

        request.CookieContainer = cookies;
        request.AllowAutoRedirect = true;
        request.KeepAlive = true;
        request.Referer = HttpUtility.UrlEncode(referer);
        request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.8) Gecko/2009021910 Firefox/3.0.7 (.NET CLR 3.5.30729)";
        request.Headers.Add("Pragma", "no-cache");
        request.Timeout = 40000;

        return request;
    }
}

Your problem is probably related to lack of a CookieContainer. If you create a cookie container you can save/use cookies in your web requests.

Solution 2

You should set the CookieContainer property of the HTTPWebRequest class to an instance of a CookieContainer class. From MSDN, it's stated that:

CookieContainer is null by default. You must assign a CookieContainer object to the property to have cookies returned in the Cookies property of the HttpWebResponse returned by the GetResponse method.

In other words, after you have set the CookieContainer property of the HTTPWebRequest object in your code, you can get the corresponding Cookies in the HTTPWebResponse object in your code. The sample code in the MSDN link shared above should get you started.