I am unable to get specific data and need you guys' advice on how I can retrieve it.
I am using a curl as a post to submit through an Api in postman and in sucess I only get empty string but the response that I need is there in the headers on the headers tab next to body the image is below
api response shows in headers click for image view
I have used getHeaders and many more functions to get but failed to get anything yet so please guide me that should i do.
Thanks
Related
I am trying to get list of products from api. i requested url with get method and response code is 200. but how can i get to see list of products in postman?
https://url/Product/all
Looks like you are getting the response in HTML format; you just need to scroll down in the response body to see them. Change your response body format from HTML to JSON. You should be able to see the JSON.
I'm working on a small-scale instagram scraping project mainly using selenium and the python requests module. I discovered that when the Request Header changes for www.instagram.com, the text that I get from using
requests.get("https://www.instagram.com/<address>/?__a=1")
Returns the HTML code for the webpage. Instead I was expecting it to be the json text containing post details. Currently, it works fine if I change the headers manually.
How do I automatically get the request header using selenium or requests? I'm expecting to get the text labeled in the image attached:
www.instagram.com Request Header
Thank you.
import requests
response = requests.get("https://www.instagram.com/<address>/?__a=1")
print(response.status_code)
print(response.headers)
print(response.json())
As I see in the headers printed out on to console, there is a set-cookie header. If this is something that you are looking for, then you may use this line to extract only this header:
print(resp.headers['Set-Cookie'])
I'm scraping http://www.germandeli.com/Meats/Sausages
I would like to extract the link for every product(or item) from the page. I use scrapy shell to test but it keeps return the empty value [ ].
Here is the code I use:
response.xpath('*//h2[#class="item-cell-name"]/a/#href')
Any helps would be greatly appreciated.
Well unfortunately the item content is rendered through JS. But luckily the URL sends a AJAX request to fetch a JSON of the items. This makes it much easier for us to parse it. You can check the XHR tab in the google chrome console to imitate the request with the required headers.
This URL returns the list of products. The limit and the offset parameters in the URL can be played around with to fetch the next set of data. Also to parse the JSON content you can use json.loads from the standard library.
Hi all please help me out with this issue. while I send following http request ("http://xyz/Scribe/Scribe.xsd"); it does not showing any data in response tab in View Result Tree. So please help me out with that.
Thanks.
It's because your server does not return :
Content-Type header
As JMeter uses it to render the response and does not find it, it does not render it.
See:
https://issues.apache.org/bugzilla/show_bug.cgi?id=54226
Issue has now been fixed, you can use nightly build:
http://jmeter.apache.org/nightly.html
I am using scrapy tool to scrape content from website, i need help from you guys how to scrape the reponse which is dynamically loaded from ajax.
when content loading from ajax at that mean time url not changing it keep remains same but content would be changed so on that event i need to crawl.
thank you,
G.kavirajan
yield FormRequest('http://addons.prestashop.com/en/modules/featureproduct/ajax-homefeatured.php',
formdata={'type':'new','ajax':'1'},
callback=self.your_callback_method)
bellow are the urls that you can easily catch using fiddler or firebug
this is for featured tab http://addons.prestashop.com/en/modules/featureproduct/ajax-homefeatured.php?ajax=1&type=random
this is for new tab http://addons.prestashop.com/en/modules/featureproduct/ajax-homefeatured.php?ajax=1&type=new
you can request on these url directly to get results you required, although website is using POST request to get data for these url, but i tried with parameter GET request is also working properly