I am using scrapy to scrape some data, I am wondering how much data of Request the Response stores.
My specific question is shown in the following code:
def parse(self,response):
r = FormRequest(url=url1, formdata={somedata}, callback=parse2)
#is this line necessary if I want the formdata being attached?
r.meta['formdata'] = formdata
yield r
def parse2(self,response):
#can I access to the formdata here without that line of code?
Any suggestion would be appreciated.
Yes, however formdata is already converted into body and won't be the dictionary but a string as far as I remember.
so try r.body also quick reminder you can check the attributes of an object by:
dir(r)
Related
I am trying to create Karate feature file for API Testing.
It is a post request and we have to pass random number for 1 field i.e. order in request payload every time.
Now, I am trying to use this function in the same feature file to pass that random generated value in json payload before sending post request.
Could someone please have a look at my feature file and help.
Thanks
Also, Is there a way if i want to pass this same random value for a json payload created as a separate request.json payload file
Your requestPayload is within double quotes, so it became a string.
Here's an example that should get you going. Just paste it into a new Scenario and run it and see what happens.
* def temp1 = 'bar'
* url 'https://httpbin.org/anything'
* def payload = { foo: '#(temp1)' }
* request payload
* method post
And please read the documentation and examples, it will save a you a lot of time.
enter image description hereI am creating a feature file using Karate framework and as per that i need to pass a json file as key value pair in the request body
eg
Given url
And def textJson = text1
And request{test:'test1.json',test2:'description text'}
when post
then status 200
enter image description here
the json file is read into a variable in another feature file and passed on to this feature file.
As of now i am getting the requested file is missing basically its not reading
Given url
And def textJson = text1
And request{test:'test1.json',test2:'description text'}
So if I understand this correctly, you intend to input the data from a json file into the test key under request. You are currently passing in a string instead of actually reading in a file. To read in a json file and save it as a variable try the following syntax.
And def test1 = read('test1.json')
And request { test1: '#(test1)'}
I am a little confused at some of the other syntax you have included. I can point out a couple other problems in the syntax.
Given url #You need to actually have a url after the url keyword
And def textjson = test1 #This does not seem to point to a reference
And request {test: 'test1.json', test2: 'description text'} # The test is not actually reading json in but the string "test1.json"
I wasn't exactly sure the scope of this question, but I hope that this helps.
I'm trying to send a POST request and format the query string in a specific format. Order doesn't matter aside from the first parameter, but I haven't been successful.
What I need:
localhost/someapp/api/dosomething/5335?save=false&userid=66462
What some of my attempts have spit out:
http://localhost/someapp/api/dosomething/?Id=29455&save=false&userId=797979
http://localhost/someapp/api/dosomething/?save=false&userId=797979
How I formatted the request:
request.AddQueryParameter("Id", "29455");
request.AddQueryParameter("save", "false");
request.AddQueryParameter("user", "4563533245");
If I try AddParameterfor Id it doesn't get appended on the query string (I'm thinking because it's a POST and not a GET), so that won't work. The API isn't expecting a form, it's expecting :
(string id, List<Dictionary<string,string>>)
I could use a StringBuilder, but that feels wrong. I'm not sure if UrlSegment is the best way to go either, since I would basically be hacking the query string. Is there a way to format my request in the format I need using RestSharp's API?
What I ended up using is UrlSegment and then kept the .AddQueryParameter methods, so the final code block looks like :
var url = new RestClient(localhost/someapp/api/dosomething/{id});
var request = new RestRequest(Method.POST);
request.AddParameter("Id", "5335", ParameterType.UrlSegment);
request.AddQueryParameter("save", "true");
request.AddQueryParameter("UserId", "5355234");
Which produced the URI I needed.
The easiest coding process for using RestSharp or any other API client library would be to use Postman to generate if you are unsure of how to code it. Download Postman, do a new request, enter the URL string to send to the API, click on Code, select C# (RestSharp) from the dropdown. Here is the code it generated.
var client = new RestClient("http://localhost/someapp/api/dosomething /5335?save=false&userid=66462");
var request = new RestRequest(Method.POST);
request.AddHeader("Postman-Token", "bd05aa45-f1b9-4665-a3e7-888ad16f2800");
request.AddHeader("cache-control", "no-cache");
IRestResponse response = client.Execute(request);
I am using dotmailer.com ESP and using SoapUI for calling API. I can get summary of the campaign in JSON format but also I want to extract summary of that campaign into a text file but couldn't find any way. Anyone can help?
Thanks.
You can do it using groovy. Simple add a groovy script testStep inside your testCase, there you can get the response from a previous testStep and save in a file. Since there are not many details in your question a generic way to do so could be:
// get the test step
def testRequest = context.testCase.getTestStepByName('Test Request')
// get the response
def response = testRequest.getPropertyValue('Response')
// create a file
def file = new File('C:/temp/response.txt');
// save the response to it
file << response
Hope it helps,
I am trying to HTTP POST an item to an API using Scrapy. In my pipeline I have:
Request( url, method='POST',
body=json.dumps(item),
headers={'Content-Type':'application/json'} )
This does not work. The error is:
{ some JSON } is not JSON serializable
Any idea on what I am doing wrong?
As stated in paul trmbrth's comment, instead of
body=json.dumps(item)
use
body=json.dumps(dict(item))
So your code would become:
Request( url, method='POST',
body=json.dumps(dict(item)),
headers={'Content-Type':'application/json'} )
I have used JsonRequest from from scrapy.http as follows:
JsonRequest(
url=<url>,
method='POST',
# Replace with your item or dict(item)
data=item
)
Behind it calls json.dumps on the dictionary you pass to the data parameter so you first might want to make sure that your item that you're trying to send is json serializable.