Jsoup: login without id or name for submit button - authentication

I've tried to login into website using Jsoup, but unfortunatelly I've forced with some problems- don't know how should I pass submit button into because there's no id or name for it. Could you take a look how should my code look like?
<form action="http://www.abcde.com/index.php?app=core&module=global&section=login&do=process" method="post" id="login">
<input type="hidden" name="auth_key" value="auth_key">
<input type="hidden" name="referer" value="http://www.abcde.com/">
<h3>Login</h3>
<div class="ipsForm ipsForm_horizontal">
<fieldset>
<ul>
<li class="ipsField ipsField_primary">
<label for="ips_username" class="ipsField_title">Username</label>
<div class="ipsField_content">
<input id="ips_username" type="text" class="input_text" name="ips_username" size="30" tabindex="0">
</div>
</li>
<li class="ipsField ipsField_primary">
<label for="ips_password" class="ipsField_title">Password</label>
<div class="ipsField_content">
<input id="ips_password" type="password" class="input_text" name="ips_password" size="30" tabindex="0"><br>
</div>
</li>
</ul>
</fieldset>
<div class="ipsForm_submit ipsForm_center">
<input type="submit" class="ipsButton" value="Login" tabindex="0">
</div>
</div>
</form>
I've started:
Connection.Respose loginForm = Jsoup.connect("http://www.abcde.com/").method(Connection.Method.GET)
.execute();
Document document = Jsoup
.connect("http://www.abcde.com/)
.data("cookieexists", "false")
.data("ips_username", "username", "ips_password",
"password").cookies(loginForm.cookies()).post();

First you must send a GET request to the server -
Document doc1 = Jsoup.connect("www.forumowisko.pl")
.userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0")
.get();
Then you extract the auth_key value -
Element e = doc.select("input[id=auth_key]").first();
String authKey = e.attr("value");
And now you can send the POST request -
Document doc2 = Jsoup.connect("http://www.forumowisko.pl/index.php?app=core&module=global&section=login&do=process")
.userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0")
.data("auth_key", authKey)
.data("ips_username", "MyUsername")
.data("ips_password, "MyPassword")
.data("rememberMe", "1")
.data("referer", "http://www.forumowisko.pl/")
.cookies(doc1.cookies())
.post();
Notice that the POST request has a different URL from the GET request.

Related

Generating a valid __RequestVerificationToken from C#

One of the most popular books on ASP.NET Core is "Pro ASP.NET Core 3" by Adam Freeman.
In chapters 7-11, he builds an example application, SportsStore.
As you can see, each product in the listing gets its own 'Add To Cart' button:
If we do 'view source' on this page, we'll see the following HTML for that item in the product list:
<div class="card card-outline-primary m-1 p-1">
<div class="bg-faded p-1">
<h4>
Kayak
<span class="badge badge-pill badge-primary" style="float:right">
<small>$275.00</small>
</span>
</h4>
</div>
<form id="1" method="post" action="/Cart">
<input type="hidden" data-val="true" data-val-required="The ID field is required." id="ID" name="ID" value="1" />
<input type="hidden" name="returnUrl" value="/" />
<span class="card-text p-1">
A boat for one person
<button type="submit" class="btn btn-success btn-sm pull-right" style="float:right">
Add To Cart
</button>
</span>
<input name="__RequestVerificationToken" type="hidden" value="CfDJ8KKqNOS0gwdMvC0-bdjTwWlvCcBJldeidwIX5b2f24gYblS9X1sqCwJWIEsKKOSf8kut0SQsQRLF3R1XBSYZkPGnta9YzRK4tcQl8dq_0uWmjeUhm8yMe90fWDt_x0smmAD1lmb9-BxQF8y_7-IQSz4" /></form>
</div>
Note the input tag towards the bottom:
<input name="__RequestVerificationToken" type="hidden" value="CfDJ8KKqNOS0gwdMvC0-bdjTwWlvCcBJldeidwIX5b2f24gYblS9X1sqCwJWIEsKKOSf8kut0SQsQRLF3R1XBSYZkPGnta9YzRK4tcQl8dq_0uWmjeUhm8yMe90fWDt_x0smmAD1lmb9-BxQF8y_7-IQSz4" />
If we look at the Views\Shared\ProductSummary.cshtml file in the SportsStore project, we'll see the code that is involved with generating these listing items:
#model Product
<div class="card card-outline-primary m-1 p-1">
<div class="bg-faded p-1">
<h4>
#Model.Name
<span class="badge badge-pill badge-primary" style="float:right">
<small>#Model.Price.ToString("c")</small>
</span>
</h4>
</div>
<form id="#Model.ID" asp-page="/Cart" method="post">
<input type="hidden" asp-for="ID" />
<input type="hidden" name="returnUrl" value="#ViewContext.HttpContext.Request.PathAndQuery()" />
<span class="card-text p-1">
#Model.Description
<button type="submit" class="btn btn-success btn-sm pull-right" style="float:right">
Add To Cart
</button>
</span>
</form>
</div>
As you can see, the form element in this case doesn't have an explicit inclusion of the input tag with the __RequestVerificationToken value. This form thus appears to be a tag helper which takes care of generting the input tag with the __RequestVerificationToken token.
As an experiment, let's suppose I have added the following method to Controllers\HomeController:
[HttpGet]
public ContentResult ButtonExample()
{
var token = "...";
return new ContentResult()
{
ContentType = "text/html",
StatusCode = (int)HttpStatusCode.OK,
Content =
String.Format(
#"<!DOCTYPE html>
<html>
<body>
<form id=""1"" method=""post"" action=""/Cart"">
<input type=""hidden"" data-val=""true"" id=""ID"" name=""ID"" value=""1"" />
<button type=""submit"">Add to Cart</button>
</form>
<input name=""__RequestVerificationToken"" type=""hidden"" value=""{0}"" />
</body>
</html>",
token)
};
}
As you can see, this generates a very simple page with a single button which is intended to add the product with ID value 1 (i.e. the Kayak) to the cart.
I of course need to pass an appropriate value for the __RequestVerificationToken.
My question is, is there a way to get this value from C# so that I can include it in the method above?
The idea as shown above would be to set the token value here:
var token = "...";
This is then interpolated into the string that generates the HTML using String.Format.
UPDATE
This page mentions the following:
To generate the anti-XSRF tokens, call the #Html.AntiForgeryToken method from an MVC view or #AntiForgery.GetHtml() from a Razor page.
So I guess the question is, how do we do the equivalent from C# directly instead of from an MVC view or Razor page?
You can add the below code to your form which will generate the __RequestVerificationToken. It is used to prevent CSRF attacks Prevent XSRF/CSRF attacks.
<form action="/" method="post">
#Html.AntiForgeryToken()
</form>

vue.js + Jest : how to test a form post?

Until now I gave been using Avoriaz, but I would like to use Jest now ...
found some tuts... but could not get any hint on testing my contact view component sending POST to an external urk...
<form id="contactForm" action="https://formspree.io/mysite.com" method="POST">
<div class="form-group">
<input type="hidden" name="_next" v-model="next" />
<input type="hidden" name="_language" v-model="language" />
<input type="hidden" name="_subject" value="Contact from my site" />
<input v-model="sendername" ...>
<input v-model="email" ...>
</div>
<div class="form-group">
<div class="uk-margin">
<textarea v-model="message" ...></textarea>
</div>
</div>
<button type="submit" class="btn btn-primary btn-gradient submit">Send</button>
</form>
As the external post URL is only valid for production... I would like mock it and use the _next property as a callback page url...
any useful links to put me on first tracks ?? thanks a lot for feedback
You should prevent submit and post data using axios for example and then mock axios.
<!-- the submit event will no longer reload the page -->
<form v-on:submit.prevent="onSubmit"></form>
It's an easier way to unit-test that.

Using Scrapy for Crawling Pages Post Authentication

Taking cues and ideas from a previous post , I tried to come up with my own code.
However, using my code I've noticed that it doesn't really scrape anything and probably doesn't go beyond the authentication level at all. I say this because I don't see any Error Logs even when I put an incorrect password.
My best guess is that the HTML for the authentication fields is not contained in a "Form" tag and hence the formdata is possibly overlooking it. Might be wrong.
My Code so far:
class LoginSpider(BaseSpider):
name = 'auth1'
start_urls = ['http://www.example.com/administration']
def parse(self, response):
return [FormRequest.from_response(response,
formdata={'employee[email]': 'xyz#abc.com', 'employee[password]': 'XYZ'},
formxpath='//div[#class="form-row"]',
callback=self.after_login)]
def after_login(self, response):
if "authentication failed" in response.body:
self.log("Login failed", level=log.ERROR)
return
# We've successfully authenticated, let's have some fun!
else:
return Request(url="http://www.liveyoursport.com/administration/customers",
callback=self.parse_tastypage)
def parse_tastypage(self, response):
sel = Selector(response)
item = Item()
item ["Test"] = sel.xpath("//h1/text()").extract()
yield item
Here's the HTML section:
<div class="content-row">
<div class="special-header-title span_full">
<h3><span class="blue-text">Sign </span>In</h3>
</div>
</div>
<div class="content-row">
<div class="form-section checkout-address-edit span_80" id="sign-in-form" >
<form accept-charset="UTF-8" action="/employees/sign_in" class="new_employee" id="new_employee" method="post"><div style="margin:0;padding:0;display:inline"><input name="utf8" type="hidden" value="✓" /><input name="authenticity_token" type="hidden" value="HQYZa0hNZ2Y+UvtbIk9OxI48Hlsnt+MiYOeV9ql2yWo=" /></div>
<div>
<div class="form-row">
<div class="form-col-1"><label for="employee_email">Email</label></div>
<div class="form-col-2">
<input id="employee_email" name="employee[email]" size="30" type="email" value="" />
</div>
</div>
<div class="form-row">
<div class="form-col-1"><label for="employee_password">Password</label></div>
<div class="form-col-2">
<input id="employee_password" name="employee[password]" size="30" type="password" />
</div>
</div>
</div>
<div class="form-row form-row-controls">
<div class="form-col-1"></div>
<div class="form-col-2">
<input class="sign-in-button f-right" name="commit" type="submit" value="Sign in" />
</div>
</div>
</form> <br>
Forgot your password?<br />
Didn't receive unlock instructions?<br />
</div>
From the docs:
formxpath (string) – if given, the first form that matches the xpath
will be used.
but it seems that you are not matching the form, rather the parent div.
Try it like this:
return [FormRequest.from_response(response,
formdata={'employee[email]': 'xyz#abc.com', 'employee[password]': 'XYZ'},
formxpath='//form[#id="new_employee"]',
callback=self.after_login)]
Also, if you only have one form element on the page, you don't need to define formxpath.

HTMLInputElemet or webbrowser.document on Windows Phone App Login

Hey I'm try to make a Windows Phone app that logs people in to a webpage, but I'm having trouble getting the app to post the log-in info to the webbrowser.
on the desktop I would have used
HTMLInputElement nameBox = (HTMLInputElement) myDoc.all.item("name", 0);
HTMLInputElement passBox = (HTMLInputElement) myDoc.all.item("pass", 0);
nameBox.value = "textBox1.text";
passBox.value = "textBox2.text;
but the HTMLInputElemet don't exist in Windows Phone.
I also tried using webBrowser1.document but that don't work either on Windows Phone.
It is a log-in using a form and not a query string.
The server side code look like this:
<div class="form-item textfield edit-name">
<div class="label">
<label >E-mail-adresse</label>
</div>
<input type="text" maxlength="60" placeholder="" name="name" id="edit-name" size="60" value="" class="form-text required" /> </div>
<div class="form-item password edit-pass">
<div class="label">
<label >Adgangskode</label>
</div>
<input type="password" name="pass" id="edit-pass" maxlength="128" size="60" class="form-text required" /> </div>
<input type="hidden" name="form_build_id" id="form-b6cb750ff3a79c34d408df98d730a39e" value="form-b6cb750ff3a79c34d408df98d730a39e" />
<input type="hidden" name="form_id" id="edit-user-login" value="user_login" />
<div class="form-actions"><input type="submit" name="op" id="edit-submit" value="Log ind" class="form-submit" />
It is easy in the desktop but I just cant figure out how to do this on the Phone, hope some one can help.
One way to access the fields is using Javascript, thanks to the InvokeScript method:
webBrowser1.InvokeScript("eval", string.Format("document.getElementsByName('name')[0].value='{0}'", "PutTheLoginHere"));

Logging into a website using Mechanize and Nokogiri?

I am having some difficulty with one of our service providers login forms. The other sites are working fine but for some reason I can't get past their login form.
The website login for is like this:
<form accept-charset="UTF-8" action="/sessions" class="new_user_session" id="new_user_session" method="post"><div style="margin:0;padding:0;display:inline"><input name="utf8" type="hidden" value="✓" /><input name="authenticity_token" type="hidden" value="kaLEkPesQfeheronzGTdfnVAzpuUiC+VmjVXBu540n8=" /></div>
<fieldset class="big">
<div class="form-row">
<div class="form-label">
<label for="user_session_email">Email</label>
</div>
<div class="form-field">
<input id="user_session_email" name="user_session[email]" size="30" type="text" />
</div>
</div>
<div class="form-row">
<div class="form-label">
<label for="user_session_password">Password</label>
</div>
<div class="form-field">
<input id="user_session_password" name="user_session[password]" size="30" type="password" />
</div>
<div class="form-comment"><p>Forgot your password?</p></div>
</div>
<div class="form-row optional">
<div class="form-field">
<label for="user_session_remember_me"><input name="user_session[remember_me]" type="hidden" value="0" /><input id="user_session_remember_me" name="user_session[remember_me]" type="checkbox" value="1" /> Remember me for 2 weeks</label>
</div>
</div>
</fieldset>
I have tried to login using the same code as other the other sites but it doesn't work.
# Create a new mechanize object
agent = Mechanize.new
# Load the dial9 website
page = agent.get("http://webapplication.co.uk")
# Select the first form
form = agent.page.forms.first
form.username = 'username
form.password = 'password'
# Submit the form
page = form.submit form.buttons.first
I have also tried a different way of logging in as suggested in other SO questions/answers:
email = 'user#domain.com'
password = 'password
# Create a new mechanize object
agent = Mechanize.new
# Load the postmarkapp website
page = agent.get("https://domain.com")
# Select the first form
form = agent.page.forms.first
form.field_with(:email => "user_session_email").value = email
form.field_with(:password => "user_session_password").value = password
# Submit the form
page = form.submit form.buttons.first
Using this method of authentication I get the following output when running the rake task:
undefined method `email' for [hidden:0x3fef2ab2b994 type: hidden name: utf8 value: ✓]:Mechanize::Form::Hidden
Upon closer inspection the above error seems to be due to the fact that there is a field immediately after the form is started:
<form accept-charset="UTF-8" action="/sessions" class="new_user_session" id="new_user_session" method="post"><div style="margin:0;padding:0;display:inline"><input name="utf8" type="hidden" value="✓" /><input name="authenticity_token" type="hidden" value="kaLEkPesQfeheronzGTdfnVAzpuUiC+VmjVXBu540n8=" /></div>
Am I missing something? If so, what? Any pointers are appreciated!
Try changing
form.field_with(:email => "user_session_email").value = email
form.field_with(:password => "user_session_password").value = password
to
form.field_with(:name => "user_session[email]").value = email
form.field_with(:name => "user_session[password]").value = password