Is there an option to set cookies while using rest client in Pentaho 5.1?
I read a couple of blogs and it wasnt mentioned anywhere.
I have tried using curl using shell job entry. Got the cookie and used it in my next curl to get data.
I need to do a similar process using rest client transformation entry.
Please let me know if there are any leads I can follow.
i dont know if you can do that with this step, but with the http client step you can set you own http request headers. This works because i use this way.
if you can use the http client step instead the rest client do this:
add a new Script step (the javascript step) and add this js code to your trans (these are sample headers, the most interesting for you is the last one):
//set the headers to next step
var header_accept_charset = "utf-8";
var header_cache_control = "max-age=0";
var header_user_agent = "batman browser";
lal = "lalvalue_fooo";
lel = "lelvalue_meeeh";
var cookie = "lol="+ lol +"; lal="+ lal;
Now make sure the vars are passed to the next step, the http client (click on get variables to fill the rows of "fields"), this should works.
The cookie is just another request header, a string simply built with the concatenation of variables and values with semicolons.
PD:Maybe this method works with the Rest Client step, if works also with this step let me know, i am interested to know that.
Related
Objective:
We want few API calls should go to mock-server(https://192.x.x.x:8001) and others should go to an actual downstream application server(https://dev.api.acme.com).
Setup :
On local, mock server is up with standalone jar on port 8001. e.g https://192.x.x.x:8001
In application config file (config.property)downstream system(which need to mock) defined with mockserver IP i.e https://192.x.x.x:8001
Testing scenario and problem:
1.
Scenario: pathMatches('/profile/v1/users/{id}/user')
* karate.proceed('https://dev.api.acme.com')
* def response = read ('findScope.json')
* def responseStatus = 200ˀˀ
* print 'created response is: ' + response
Now, when we hit API request via postman or feature file then it does karate.proceed properly to https://dev.api.acme.com/profile/v1/users/123/user instead of 192.x.x.x. However, in this request, host is referring to https://192.x.x.x:8001 instead of https://dev.api.acme.com which create a problem for us.
How can we override request header in this case? I did try with karate.set and also with header host=https://192.x.x.x:8001 but no luck.
Thanks!
Please see if the 1.0 version works: https://github.com/intuit/karate/wiki/1.0-upgrade-guide
Unfortunately https proxying may not work as mentioned. If you are depending on this, we may need your help (code contribution) to get this working
If the Host header is still not mutable, that also can be considered a feature request, and here also I'd request you to consider contributing code
I am trying to log in to a site using requests as follows:
s = requests.Session()
login_data = {"userName":"username", "password":"pass", "loginPath":"/d2l/login"}
resp = requests.post("https://d2l.pima.edu/d2l/login?login=1", login_data)
although I am getting a 200 response, when I say
print(resp.content)
b"<!DOCTYPE html><html><head><meta charset='utf-8' /><script>var hash = window.location.hash;if( hash ) hash = '%23' + hash.substring( 1 );window.location.replace('/d2l/login?sessionExpired=0&target=%2fd2l%2ferror%2f404%2flog%3ftargetUrl%3dhttp%253A%252F%252Fd2l.pima.edu%253A80%252Fd2l%252Flogin%253Flogin%253D1' + hash );</script><title></title></head><body></body></html>"
notice it says session expired.
What I've tried:
logging back out and in in the actual browser, no success.
http basic auth, no success.
I'm thinking maybe I need to authenticate myself to this site using cookies?
If so how do I determine which cookies to send it?
I tried figuring this out by saying
resp.cookies
Out[4]: <RequestsCookieJar[]>
shouldn't this be giving me names of cookies? I'm not sure what to do with such output.
Main Point: HOW DO I FIGURE OUT HOW TO AUTHENTICATE MYSLEF TO THIS WEBSITE?
Help is appreciated.
I would rather not use selenium.
From loading this page https://d2l.pima.edu/d2l/login and viewing its source, you'll notice the POST target path is /d2l/lp/auth/login/login.d2l. Try using that as your POST path. Your other fields look consistent with the form's expectations.
Note: with python requests if you create a session object use it to make your requests:
resp = s.post(<blah blah>, login_data)
The session will hold any cookies set by the login server, and you can continue to use the s object to make requests in the authenticated session.
I have made an api and I want to access a post call in it. I made the following transformation in kettle:
with a params field in Generate Rows step as:
and REST Client step configuration as:
but I am unable to get any of the parameters in my post call on server side. If I write a simple post call in python as:
import requests
url = "http://10.131.70.73:5000/searchByLatest"
payload = {'search_query': 'donald trump', 'till_date': 'Tuesday, 7 June 2016 at 10:40'}
r = requests.post(url, params=payload)
print(r.text)
print(r.status_code)
I am able to get the parameters by request.args.get("search_query") on the client side in Flask. How can I make an equivalent POST call in kettle?
I found the solution myself eventually. Describe the fields in generate rows as:
and in the parameters tab in REST Client step, we should get the same fields:
Works perfect!
I have a spreadsheet on my Google Drive and I want to download a CSV from another website and put it into my spreadsheet. The problem is that I have to login to the website first, so I need to use some HTTP request to do that.
I have found this site and this. If either of these sites has the answer on it, then I clearly don't understand them enough to figure it out. Could someone help me figure this out? I feel that the second site is especially close to what I need, but I don't understand what it is doing.
To clarify again, I want to login with an HTTP request and then make a call to the same website with a different URL that is the call to get the CSV file.
I have done a lot of this in the past month so I should be able to help you, we are trying to emulate the browsers behaviour here so first you need to use chrome's developer tools(or something similar) and note down the exact things the browser does like the form values posted, the url that is called and so on. The following example shows the general techinique to be used:
The first step is to login to the website and get the session cookie:
var payload =
{
"user_session[email]" : "username",
"user_session[password]" : "password",
};// The actual values of the post variables (like user_session[email]) depends on the site so u need to get it either from the html of the login page or using the developer tools I mentioned.
var options =
{
"method" : "post",
"payload" : payload,
"followRedirects" : false
};
var login = UrlFetchApp.fetch("https://www.website.com/login" , options);
var sessionDetails = login.getAllHeaders()['Set-Cookie'];
We have logged into the website (In order to confirm just log the sessionDetails and match it with the cookies set by chrome). The next step is purely dependent on the website so I will give u a general example
var downloadPayload =
{
"__EVENTTARGET" : 'ctl00$ActionsPlaceHolder$exportDownloadLink1',
};// This is just an example it may or may not be needed, if needed u need to trace the values from the developer tools.
var downloadCsv = UrlFetchApp.fetch("https://www.website.com/",
{"headers" : {"Cookie" : sessionDetails},
"method" : "post",
"payload" : downloadPayload,
});
Logger.log(downloadCsv.getContentText())
The file should now be logged, you can then parse the csv using hte GAS inbuilt function and dump the data in the spreadsheet.
A few points to note:
I have assumed that all form post values are static and can be
hardcoded, in case this is not true then let me know I will give you
a function that can extract values from the html.
Some websites require the browser to send a token value(the value will be present in the html) along with the credentials. In this case you need to extract the values and then post it.
For reasons outlined here I need to review a set values from they querystring or formdata before each request (so I can perform some authentication). The keys are the same each time and should be present in each request, however they will be located in the querystring for GET requests, and in the formdata for POST and others
As this is for authentication purposes, this needs to run before the request; At the moment I am using a MessageHandler.
I can work out whether I should be reading the querystring or formdata based on the method, and when it's a GET I can read the querystring OK using Request.GetQueryNameValuePairs(); however the problem is reading the formdata when it's a POST.
I can get the formdata using Request.Content.ReadAsFormDataAsync(), however formdata can only be read once, and when I read it here it is no longer available for the request (i.e. my controller actions get null models)
What is the most appropriate way to consistently and non-intrusively read querystring and/or formdata from a request before it gets to the request logic?
Regarding your question of which place would be better, in this case i believe the AuthorizationFilters to be better than a message handler, but either way i see that the problem is related to reading the body multiple times.
After doing "Request.Content.ReadAsFormDataAsync()" in your message handler, Can you try doing the following?
Stream requestBufferedStream = Request.Content.ReadAsStreamAsync().Result;
requestBufferedStream.Position = 0; //resetting to 0 as ReadAsFormDataAsync might have read the entire stream and position would be at the end of the stream causing no bytes to be read during parameter binding and you are seeing null values.
note: The ability of a request's content to be read single time only or multiple times depends on the host's buffer policy. By default, the host's buffer policy is set as always Buffered. In this case, you will be able to reset the position back to 0. However, if you explicitly make the policy to be Streamed, then you cannot reset back to 0.
What about using ActionFilterAtrributes?
this code worked well for me
public HttpResponseMessage AddEditCheck(Check check)
{
var request= ((System.Web.HttpContextWrapper)Request.Properties.ToList<KeyValuePair<string, object>>().First().Value).Request;
var i = request.Form["txtCheckDate"];
return Request.CreateResponse(HttpStatusCode.Ok);
}