When I open the URL with driver.get(url), how can I get the response content of the page? Please refer to the image for more information.
In a separate post I saw this answer. As per it there is a ticket opened for Selenium.
I'm using Python and Django, but it's actually simple to get the response. I'm using a StaticLiveServerTestCase as my base test for the test. The .get() method on self.client actually returns the response itself. For example:
response = self.client.get(url)
However, it looks like what you're really trying to get is the cookie based on what you're pointing to in the picture. I use Django and the Django test suite to authenticate a user session to be used in the test.
def create_pre_authenticated_session(self, username, url="/"):
user = User.objects.create(username=username)
session = SessionStore()
session[SESSION_KEY] = user.pk
session[BACKEND_SESSION_KEY] = settings.AUTHENTICATION_BACKENDS[0]
session[HASH_SESSION_KEY] = user.get_session_auth_hash()
session.save()
# to set a cookie we need to first visit the domain.
# 404 pages load the quickest!
self.browser.get(self.live_server_url + '/404_no_such_url/')
self.browser.add_cookie(dict(
name=settings.SESSION_COOKIE_NAME,
value=session.session_key,
secure=False,
path='/',
))
self.browser.get(self.live_server_url + url)
return user
This has some other stuff in it that I borrowed from Percival's Test-Driven Development with Python, but I hope that it can provide some guidance on what you're trying to accomplish.
Related
I'd like to automate my log in to my bank to automatically fetch my transactions to stay up-to-date with spendings and earnings, but I am stuck.
The bank's login webpage is: https://login.bancochile.cl/bancochile-web/persona/login/index.html#/login
I am using python's request module with sessions:
urlLoginPage = 'https://login.bancochile.cl/bancochile-web/persona/login/index.html'
urlLoginSubmit = 'https://login.bancochile.cl/oam/server/auth_cred_submit'
username = '11.111.111-1' # this the format of a Chilean National ID ("RUT")
usernameFormatted = '111111111' # same id but formatted
pw = "password"
payload = [
("username2", usernameFormatted),
("username2", username),
("userpassword", pw),
("request_id", ''),
("ctx", "persona"),
("username", usernameFormatted),
("password", pw),
]
with requests.Session() as session:
login = session.get(urlLoginPage)
postLogin = session.post(
urlLoginSubmit,
data=payload,
allow_redirects=False,
)
redirectUrl = postLogin.headers["Location"]
First I find that the form data has duplicated keys, so I am using the payload as a list of tuples. From Chrome's inspect I find the form data to be like this:
username2=111111111&username2=11.111.111-1&userpassword=password&request_id=&ctx=persona&username=111111111&password=password
I've checked the page's source code to look for the use of a csrf token, but couldn't find any hint of it.
What happens is that the site does a redirect upon submitting the login data. I set allow_redirects=False to catch the redirect url of the post under the Location-header. However, here is the problem. Using the web-browser I know that the redirect url should be https://portalpersonas.bancochile.cl/mibancochile/rest/persona/perfilamiento/home, but I always end up on an error page when using the above method (https://login.bancochile.cl/bancochile-web/contingencia/error404.html). (I am using my own, correct login credentials to try this)
If I submit the payload in a wrong format (e.g. by dropping a key) I am redirected to the same error-page. This tells me that probably something with the payload is incorrect, but I don't know how to find out what may be wrong.
I am kind of stuck and don't know how I can figure out where/how to look for errors and possible solutions. Any suggestions on how to debug this and continue or ideas for other approaches would be very welcome!
Thanks!
With reference to https://docs.pro.coinbase.com/#get-account-history
HTTP REQUEST
GET /accounts//holds
I am struggling to produce python code to get the account holds via API pagination request and I could not find any example of implementing it.
Could you please advise me on how to proceed with this one?
According to Coinbase Pro documentation, pagination works like so (example with the holds endpoint):
import requests
account_id = ...
url = f'https://api.pro.coinbase.com/accounts/{account_id}/holds'
response = requests.get(url)
assert response.ok
first_page = response.json() # here is the first page
cursor = response.headers['CB-AFTER']
response = requests.get(url, params={'after': cursor})
assert response.ok
second_page = response.json() # then the next one
cursor = response.headers['CB-AFTER']
# and so on, repeat until response.json() is an empty list
You should properly wrap this into helper functions or class, or, even better, use an existing library and save dramatic time.
I am trying to log in to a site using requests as follows:
s = requests.Session()
login_data = {"userName":"username", "password":"pass", "loginPath":"/d2l/login"}
resp = requests.post("https://d2l.pima.edu/d2l/login?login=1", login_data)
although I am getting a 200 response, when I say
print(resp.content)
b"<!DOCTYPE html><html><head><meta charset='utf-8' /><script>var hash = window.location.hash;if( hash ) hash = '%23' + hash.substring( 1 );window.location.replace('/d2l/login?sessionExpired=0&target=%2fd2l%2ferror%2f404%2flog%3ftargetUrl%3dhttp%253A%252F%252Fd2l.pima.edu%253A80%252Fd2l%252Flogin%253Flogin%253D1' + hash );</script><title></title></head><body></body></html>"
notice it says session expired.
What I've tried:
logging back out and in in the actual browser, no success.
http basic auth, no success.
I'm thinking maybe I need to authenticate myself to this site using cookies?
If so how do I determine which cookies to send it?
I tried figuring this out by saying
resp.cookies
Out[4]: <RequestsCookieJar[]>
shouldn't this be giving me names of cookies? I'm not sure what to do with such output.
Main Point: HOW DO I FIGURE OUT HOW TO AUTHENTICATE MYSLEF TO THIS WEBSITE?
Help is appreciated.
I would rather not use selenium.
From loading this page https://d2l.pima.edu/d2l/login and viewing its source, you'll notice the POST target path is /d2l/lp/auth/login/login.d2l. Try using that as your POST path. Your other fields look consistent with the form's expectations.
Note: with python requests if you create a session object use it to make your requests:
resp = s.post(<blah blah>, login_data)
The session will hold any cookies set by the login server, and you can continue to use the s object to make requests in the authenticated session.
This question already has an answer here:
Is there a way to update the headers in one feature file and use the Auth token from Karate.config.js?
(1 answer)
Closed 1 year ago.
I'm trying to setup a framework to run Graphql calls and create and E2E environment.
I've got the following setup so far but i can't seem to get the headers part of it working. i have managed to set the auth for each request and it all works but as it logs in for each request it doesn't really work as expected.
I want do the following steps:
run a login Test (different usernames valid/invalid)
run a logout test (Ensure token is removed)
Then login with correct user and extract the "set-cookie" header (to use globally for all future requests)
I was trying to use the following:
Karate-config.js
karate.callSingle('classpath:com/Auth/common-headers.feature', config);
headers.js
function fn() {
var headers = {}
headers["set-cookie"] = sessionAccessId
karate.log('Cookie Value: ', headers)
return headers
}
common-headers.feature
Feature: Login to Application and extract header
Background:
* url serverAuthenticateUri
* header Accept = 'application/json'
Scenario: 'Login to the system given credentials'
Given request { username: '#(username)', password: '#(password)'}
When method post
Then status 200
And match $.success == '#(result)'
And def myResult = response
* def sessionAccessId = responseHeaders['set-cookie'][0]
* configure headers = read('classpath:headers.js')
* print 'headers:', karate.prevRequest.headers
feature-file.feature
Feature: sample test script
Background:
* url serverBaseUri
* def caseResp = call read('classpath:com/E2E/POC/CommonFeatures/CreateCaseRequest.feature')
* def caseReqId = caseResp.response.data.createCaseAndRequest.siblings[0].id
* def caseId = caseResp.response.data.createCaseAndRequest.siblings[0].forensicCaseId
* def graphQlCallsPath = 'classpath:com/E2E/POC/GraphQl/intForensic/'
* def commmonFiles = 'classpath:E2E/CommonFiles/'
Scenario: TC1a - Request Server Details from Config DB (1st Run):
Should handle requesting Server Details Data from Config Database.
* def queryFile = graphQlCallsPath + '20-TC1a_req_req_valid_id.graphql'
* def responseFile = graphQlCallsPath + '20-TC1a_resp_req_valid_id.json'
Given def query = read(queryFile)
And replace query.reqId = caseReqId
And request { query: '#(query)' }
When method post
Then status 200
And json resp = read(responseFile)
And replace resp.reqId = caseReqId
And replace resp.caseID = caseId
And match resp == $
I can log in correctly and i get the set-cookie token but this isn't being passed on the feature-file.feature and i get an error saying "not logged in" in the response.
Any help appreciated! I might be looking at this totally wrong and i have tried to follow the shared-scope as much as i can but can't understand in.
Please make this change and hopefully that works !
headers["set-cookie"] = karate.get('sessionAccessId');
Why is explained here: (read the whole section carefully) https://github.com/intuit/karate#configure-headers
EDIT: one more suggestion:
var temp = karate.callSingle('classpath:com/Auth/common-headers.feature', config);
karate.configure('headers', { 'set-cookie': temp.sessionAccessId });
Some extra suggestions:
If you have just started with Karate - based on your question I would suggest you get one flow working in a single Scenario first without any use of call and with nothing whatsoever in karate-config.js. Hard-code everything and get it working first. Use the header keyword to set any headers you need. I also see you are trying to set a set-cookie header (which may work) but Karate has a special keyword for cookie.
And don't even think about callSingle() to start with :)
Once you get that first "hard-coded" flow working, then attempt to configure headers and then only finally try to do "framework" stuff. You seem to have jumped straight into super-complexity without getting the basics right.
Please read this other answer as well, because I suspect that you or someone in your team is attempting to introduce what I refer to as "too much re-use": https://stackoverflow.com/a/54126724/143475 - try not to do this.
Also note that your question is so complex that I have not been able to follow it, so please ask a simpler or more specifc question next time. If you still are stuck, kindly follow this process: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue
I have a spreadsheet on my Google Drive and I want to download a CSV from another website and put it into my spreadsheet. The problem is that I have to login to the website first, so I need to use some HTTP request to do that.
I have found this site and this. If either of these sites has the answer on it, then I clearly don't understand them enough to figure it out. Could someone help me figure this out? I feel that the second site is especially close to what I need, but I don't understand what it is doing.
To clarify again, I want to login with an HTTP request and then make a call to the same website with a different URL that is the call to get the CSV file.
I have done a lot of this in the past month so I should be able to help you, we are trying to emulate the browsers behaviour here so first you need to use chrome's developer tools(or something similar) and note down the exact things the browser does like the form values posted, the url that is called and so on. The following example shows the general techinique to be used:
The first step is to login to the website and get the session cookie:
var payload =
{
"user_session[email]" : "username",
"user_session[password]" : "password",
};// The actual values of the post variables (like user_session[email]) depends on the site so u need to get it either from the html of the login page or using the developer tools I mentioned.
var options =
{
"method" : "post",
"payload" : payload,
"followRedirects" : false
};
var login = UrlFetchApp.fetch("https://www.website.com/login" , options);
var sessionDetails = login.getAllHeaders()['Set-Cookie'];
We have logged into the website (In order to confirm just log the sessionDetails and match it with the cookies set by chrome). The next step is purely dependent on the website so I will give u a general example
var downloadPayload =
{
"__EVENTTARGET" : 'ctl00$ActionsPlaceHolder$exportDownloadLink1',
};// This is just an example it may or may not be needed, if needed u need to trace the values from the developer tools.
var downloadCsv = UrlFetchApp.fetch("https://www.website.com/",
{"headers" : {"Cookie" : sessionDetails},
"method" : "post",
"payload" : downloadPayload,
});
Logger.log(downloadCsv.getContentText())
The file should now be logged, you can then parse the csv using hte GAS inbuilt function and dump the data in the spreadsheet.
A few points to note:
I have assumed that all form post values are static and can be
hardcoded, in case this is not true then let me know I will give you
a function that can extract values from the html.
Some websites require the browser to send a token value(the value will be present in the html) along with the credentials. In this case you need to extract the values and then post it.