Could not successfully simulate the form login using c# programming code - httpwebrequest

Currently I'm trying to simulate the Login using c# code (HttpWebRequest/HttpWebResponse), however I ended up with the login.aspx webpage's html text rather than the html text after successful login.
the returned variable 'html' is exactly the same with the login.aspx webpage itself, seems it did not post the data at all. please help. thank you. dave. here is the code i used
var LOGIN_URL = "http://altech.com.au/login.aspx";
HttpWebRequest webRequest = WebRequest.Create(LOGIN_URL) as HttpWebRequest;
StreamReader responseReader = new StreamReader(
webRequest.GetResponse().GetResponseStream());
string responseData = responseReader.ReadToEnd();
responseReader.Close();
string postData =String.Format(
"ctl00$ContentPlaceHolderBodyMain$txtEmail {0}&ctl00$ContentPlaceHolderBodyMain$txtPassword={1}&btnLogin=Login","myemail", "mypassword");
//have a cookie container ready to receive the forms auth cookie
CookieContainer cookies = new CookieContainer();
// now post to the login form
webRequest = WebRequest.Create(LOGIN_URL) as HttpWebRequest;
webRequest.Method = "POST";
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.CookieContainer = cookies;
webRequest.UserAgent = "Mozilla/5.0 (Windows NT 5.2; WOW64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1";
// write the form values into the request message
StreamWriter requestWriter = new StreamWriter(webRequest.GetRequestStream());
requestWriter.Write(postData);
requestWriter.Close();
HttpWebResponse response2 = (HttpWebResponse)webRequest.GetResponse();
StreamReader sr2 = new StreamReader(response2.GetResponseStream(), Encoding.Default);
string html = sr2.ReadToEnd();

You need to URL encode your data or it will not be accepted.
Dictionary<string, string> FormData = new Dictionary<string, string>();
//Add all of your name/value pairs to the dictionary
FormData.Add(
"ctl00$ScriptManager1",
"ctl00$ContentPlaceHolderBodyMain...");
...then to format it correctly...
public string GetUrlEncodedPostData(
Dictionary<string, string> FormData)
{
StringBuilder builder = new StringBuilder();
for (int i = 0; i < FormData.Count(); ++i)
{
//Better to use string.join, but this is a bit more clear ^_^
builder.AppendFormat(
"{0}={1}&",
WebUtility.UrlEncode(InputNameValue.ElementAt(i).Key),
WebUtility.UrlEncode(InputNameValue.ElementAt(i).Value));
}
//Remove trailing &
builder.Remove(builder.Length - 1, 1);
return builder.ToString();
}

Related

Attaching files to Azure DevOps work item

I am trying to attach files (screenshots) to an Azure DevOps work item via a C# desktop app. I have managed to attach files, but they're not valid image files, which leads me to believe that I'm doing something wrong in uploading them.
From the documentation DevOps Create Attachment below is the section on the Request body of the API call, which is rather vague.
From a GitHub discussion this answer seems to suggest that I just upload the binary content directly, which is what I'm doing.
My code is as follows
var img = File.ReadAllBytes(fname);
string query = #"/_apis/wit/attachments?fileName=" + fname + #"&api-version=6.0"
string response = AzureUtils.AttachFile(query, img, "POST", false, "application/octet-stream");
Is it correct that I literally pass in the byte array which is read from the file (variable img) as the body?
Why is it not a valid file when I look at it in DevOps?
The code for AttachFile is
public static string AttachFile(string query, byte[] data = null, string method = "GET",
bool dontUseBaseURL = false, string contentType = "application/json-patch+json")
{
try
{
HttpWebRequest request = WebRequest.Create(query) as HttpWebRequest;
request.ContentType = contentType;
request.Method = method;
request.Proxy.Credentials = CredentialCache.DefaultCredentials;
request.Headers.Add("Authorization", "Basic " +
Convert.ToBase64String(Encoding.ASCII.GetBytes(string.Format("{1}", ["AzurePAT"]))));
if (data != null)
{
using (StreamWriter writer = new StreamWriter(request.GetRequestStream()))
{
writer.Write(data);
}
}
HttpWebResponse response = request.GetResponse() as HttpWebResponse;
string result = string.Empty;
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
result = reader.ReadToEnd();
}
request = null;
response = null;
return result;
}

login to aspx page from another site using httprequest

Using C# and ASP.NET I want to programmatically fill login detail on a web page (form) and then 'POST' those values. How do I do this?currently i am using below code but it not work it take me on same page.
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("https://pcs.d2h.com/");
req.Method = "GET";
StreamReader responseReader = new StreamReader(req.GetResponse().GetResponseStream());
string responseData = responseReader.ReadToEnd();
responseReader.Close();
// extract the viewstate value and build out POST data
string viewState = ExtractViewState(responseData);
string postData =
String.Format(
"__VIEWSTATE={0}&CenterCPH_tbUsername={1}&CenterCPH_tbPassword={2}&CenterCPH_btnSubmit=Login",
viewState, "9115308123", "12345678#A"
);
// have a cookie container ready to receive the forms auth cookie
CookieContainer cookies = new CookieContainer();
HttpWebRequest req2 = (HttpWebRequest)WebRequest.Create("https://pcs.d2h.com/login.aspx");
req2.Method = "POST";
req2.ContentType = "application/x-www-form-urlencoded";
req2.ContentLength = postData.Length;
req2.KeepAlive = true;
req2.CookieContainer = cookies;
req2.AllowAutoRedirect = false;
StreamWriter sout = new StreamWriter(req2.GetRequestStream());
sout.Write(postData);
sout.Flush();
sout.Close();
HttpWebResponse res = (HttpWebResponse)req2.GetResponse();
StreamReader sr = new StreamReader(res.GetResponseStream());
string returnvalue = sr.ReadToEnd();
HttpWebRequest req3 = (HttpWebRequest)WebRequest.Create("https://pcs.d2h.com/Terminals/Default.aspx");
req3.CookieContainer = cookies;
res = (HttpWebResponse) req3.GetResponse();
sr = new StreamReader(res.GetResponseStream());
returnvalue = sr.ReadToEnd();
sr.Close();
return returnvalue;

Automate download data from web site using vb.net

I have an assignment, I want to download the data from some specific web site, I checked the topic of httpwebrequest and httpwebresponse but still I m not able to understand how we can fill the text boxes on that web site and press the login button to go inside the web site, login into the web site is the first step.
I want to use VB.net for this task, if anybody can help us.
Thanks In Advance.
I had written an article on this using ASP.NET MVC, it should help you : Introduction to Web Scraping with HttpWebRequest using ASP.NET MVC 3
Web Scraping means parsing those pages to extract pieces of information in a structured way. It also refers to creating a programmatic interface, an API, that interacts with a site through an HTML interface meant for humans.
Please dont think that your program will type in username and password in to thier respective textboxes and then the program will press the login button for you to go inside the website.
Web Scraping does not happen in this manner.
You need to study how the website POST the Login username and password and then using your code you need to do the same thing to get the cookie. At each step, rather on each page you ll need to see how the website works using firebug or chrome dev tool and then send the POST or GET data accordingly to get what you want.
Below is what I had written to scrape data from my WordPress website, I have added comments wherever applicable to make the code easier to read.
Below are the constant that you need to define, ‘UserName’ and ‘Pwd’ are the login details to my WordPress account, ‘Url’ stand for the login page url and ‘ProfileUrl’ is the address of the page where the profile details are shown.
const string Url = "http://yassershaikh.com/wp-login.php";
const string UserName = "guest";
const string Pwd = ".netrocks!!"; // n this not my real pwd :P
const string ProfileUrl = "http://yassershaikh.com/wp-admin/profile.php";
public ActionResult Index()
{
string postData = Crawler.PreparePostData(UserName, Pwd, Url);
byte[] data = Crawler.GetEncodedData(postData);
string cookieValue = Crawler.GetCookie(Url, data);
var model = Crawler.GetUserProfile(ProfileUrl, cookieValue);
return View(model);
}
I had created a static class called “Crawler”, here’s the code for it.
// preparing post data
public static string PreparePostData(string userName, string pwd, string url)
{
var postData = new StringBuilder();
postData.Append("log=" + userName);
postData.Append("&");
postData.Append("pwd=" + pwd);
postData.Append("&");
postData.Append("wp-submit=Log+In");
postData.Append("&");
postData.Append("redirect_to=" + url);
postData.Append("&");
postData.Append("testcookie=1");
return postData.ToString();
}
public static byte[] GetEncodedData(string postData)
{
var encoding = new ASCIIEncoding();
byte[] data = encoding.GetBytes(postData);
return data;
}
public static string GetCookie(string url, byte[] data)
{
var webRequest = (HttpWebRequest)WebRequest.Create(url);
webRequest.Method = "POST";
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.2) Gecko/20100101 Firefox/10.0.2";
webRequest.AllowAutoRedirect = false;
Stream requestStream = webRequest.GetRequestStream();
requestStream.Write(data, 0, data.Length);
requestStream.Close();
var webResponse = (HttpWebResponse)webRequest.GetResponse();
string cookievalue = string.Empty;
if (webResponse.Headers != null && webResponse.Headers["Set-Cookie"] != null)
{
cookievalue = webResponse.Headers["Set-Cookie"];
// Modify CookieValue
cookievalue = GenerateActualCookieValue(cookievalue);
}
return cookievalue;
}
public static string GenerateActualCookieValue(string cookievalue)
{
var seperators = new char[] { ';', ',' };
var oldCookieValues = cookievalue.Split(seperators);
string newCookie = oldCookieValues[2] + ";" + oldCookieValues[0] + ";" + oldCookieValues[8] + ";" + "wp-settings-time-2=1345705901";
return newCookie;
}
public static List<string> GetUserProfile(string profileUrl, string cookieValue)
{
var webRequest = (HttpWebRequest)WebRequest.Create(profileUrl);
webRequest.Method = "GET";
webRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.2) Gecko/20100101 Firefox/10.0.2";
webRequest.AllowAutoRedirect = false;
webRequest.Headers.Add("Cookie", cookieValue);
var responseCsv = (HttpWebResponse)webRequest.GetResponse();
Stream response = responseCsv.GetResponseStream();
var htmlDocument = new HtmlDocument();
htmlDocument.Load(response);
var responseList = new List<string>();
// reading all input tags in the page
var inputs = htmlDocument.DocumentNode.Descendants("input");
foreach (var input in inputs)
{
if (input.Attributes != null)
{
if (input.Attributes["id"] != null && input.Attributes["value"] != null)
{
responseList.Add(input.Attributes["id"].Value + " = " + input.Attributes["value"].Value);
}
}
}
return responseList;
}
Hope this helps.

Auto login by using HttpWebRequest

I'm writing a window application by C# to auto login & post topic to a website. I've tried to pass username & password as the method following but the responseText did not changed. I meant if I passed the incorrect username or password the responseText would be included the "Error text such: wrong password or username" or if I passed to right username & password the responseText would be included "Login successfully" but the responseText I received was like a original login page
Do you have any idea? Thank you so much
private string Login()
{
HttpWebRequest request;
CookieContainer cookieContainer = new CookieContainer();
string cookieHeader = string.Empty;
byte[] byteArray = Encoding.UTF8.GetBytes("username=abc#gmail.com&password=123");
request = (HttpWebRequest)WebRequest.Create("http://www.vietnamworks.com/dang-nhap");
request.ContentType = "application/x-www-form-urlencoded";
request.UserAgent = userAgent;
request.Method = "POST";
request.CookieContainer = cookieContainer;
request.ContentLength = byteArray.Length;
using (Stream os = request.GetRequestStream())
{
os.Write(byteArray, 0, byteArray.Length);
}
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
string pageSource;
using (StreamReader sr = new StreamReader(response.GetResponseStream()))
{
pageSource = sr.ReadToEnd();
}
return pageSource;
}

failed to Login to a remote website

I spent too much hours to overcome this, till now with no success
From my site which is developed in MVC im trying to send a login request to a remote site, for example, facebook.
From fiddler It seems that the following inputs are required charset_test, lsd, locale,email, pass.
lsd key seems to be the unique token
Here is my code
CookieContainer cookies = new CookieContainer()
private string GetToken()
{
Uri uri = new Uri("http://www.facebook.com");
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(uri);
request.CookieContainer = cookies;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream stream = response.GetResponseStream();
StreamReader reader = new StreamReader(stream);
string htmlText = reader.ReadToEnd();
HtmlDocument document = new HtmlDocument(); --> HtmlAgilePack object
document.LoadHtml(htmlText);
//Need to check xpath doesn't return null object
HtmlNode node = document.DocumentNode.SelectNodes("//input[#name='lsd']").First();
return node.Attributes["Value"].Value;
}
private void Login()
{
string postData = string.Format("charset_test=fixcharset_test&lsd={0} &locale=en_US&email=myemail&pass=mypass", HttpUtility.UrlEncode(GetToken()));
Uri uri = new Uri("http://www.facebook.com/login.php");
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(uri);
request.CookieContainer = cookies;
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = postData.Length;
request.Method = "POST";
request.AllowAutoRedirect = true;
StreamWriter requestWriter = new StreamWriter(request.GetRequestStream());
requestWriter.Write(postData);
requestWriter.Close();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
}
//After login to facebook suppose to
public string void AddFriend()
{
Uri uri = new Uri("http://www.facebook.com");
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(uri);
request.CookieContainer = cookies;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
string status = response.StatusDescription;
StreamReader responseReader = new StreamReader(response.GetResponseStream());
string responseData = responseReader.ReadToEnd();
responseReader.Close();
return responseDat
}
Keep in mind, that facebook is just an example, Im aware of its API, Im trying to login to site with no API.
sadly, the response return a facebook page with a sorry message for unrecognizing my browser.
Any help will be more than appreciate.
Thanks,
Are you setting HTTP_USER_AGENT on your request?