httpwebrequest - Login to website, via C#

Question

I'm relatively new to using C#, and have an application that reads parts of the source code on a website. That all works; but the problem is that the page in question requires the user to be logged in to access this source code. What my program needs a way to initially log the user into the website- after that is done, I'll be able to access and read the source code.

The website that needs to be logged into is: mmoinn.com/index.do?PageModule=UsersLogin

I've searched for the entire day about how to do this and tried examples, but have had no luck.

Thanks in advance

Answer 1

You can continue using WebClient to POST (instead of GET, which is the HTTP verb you're currently using with DownloadString), but I think you'll find it easier to work with the (slightly) lower-level classes WebRequest and WebResponse.

There are two parts to this - the first is to post the login form, the second is recovering the "Set-cookie" header and sending that back to the server as "Cookie" along with your GET request. The server will use this cookie to identify you from now on (assuming it's using cookie-based authentication which I'm fairly confident it is as that page returns a Set-cookie header which includes "PHPSESSID").

---

POSTing to the login form

Form posts are easy to simulate, it's just a case of formatting your post data as follows:

field1=value1&field2=value2

Using WebRequest and code I adapted from Scott Hanselman, here's how you'd POST form data to your login form:

string formUrl = "http://www.mmoinn.com/index.do?PageModule=UsersAction&Action=UsersLogin"; // NOTE: This is the URL the form POSTs to, not the URL of the form (you can find this in the "action" attribute of the HTML's form tag
string formParams = string.Format("email_address={0}&password={1}", "your email", "your password");
string cookieHeader;
WebRequest req = WebRequest.Create(formUrl);
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
    os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];

Here's an example of what you should see in the Set-cookie header for your login form:

PHPSESSID=c4812cffcf2c45e0357a5a93c137642e; path=/; domain=.mmoinn.com,wowmine_referer=directenter; path=/; domain=.mmoinn.com,lang=en; path=/;domain=.mmoinn.com,adt_usertype=other,adt_host=-

---

GETting the page behind the login form

Now you can perform your GET request to a page that you need to be logged in for.

string pageSource;
string getUrl = "the url of the page behind the login";
WebRequest getRequest = WebRequest.Create(getUrl);
getRequest.Headers.Add("Cookie", cookieHeader);
WebResponse getResponse = getRequest.GetResponse();
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
    pageSource = sr.ReadToEnd();
}

EDIT:

If you need to view the results of the first POST, you can recover the HTML it returned with:

using (StreamReader sr = new StreamReader(resp.GetResponseStream()))
{
    pageSource = sr.ReadToEnd();
}

Place this directly below cookieHeader = resp.Headers["Set-cookie"]; and then inspect the string held in pageSource.