I have many rst files with rss urls on each page and I have to fetch rss content and paste it on the respective pages. Any Idea on how to do it?
Suppose its RSS like
https://example.com/feeds/project/project-name 1 date description
Related
I am using a request:
https://www.googleapis.com/youtube/v3/search?part=snippet&type=video&q={search value}&maxResults=3&key={key}`
This way I am getting json with info about video thumbnails and descriptions what i need but I don't have an url to channel picture and don't have infro how many views has a video.
I found the way to get the info about channel URL logo in separate request
https://www.googleapis.com/youtube/v3/search?part=snippet&channelId={id}
but it doesn't look right to make additional separate requests for every single logo of the list and Views statistics.
Also, is it possible to get in the same request for embedded video URL's as well?
You are right, it's impossible from a search query q=${searchValue} to get the associated channel thumbnail URL in a single request.
I am trying to crawl a page that is using lazy loading to get the next set of items. My crawler follows normal links, but this one seems to be different:
The page:
https://www.omegawatches.com/de/vintage-watches
is followed by https://www.omegawatches.com/de/vintage-watches?p=2
But only if you load it within the browser. Scrapy will not follow the link.
Is there a way to make scray follow the pages 1,2,3,4 automatically?
The page follows Virtual scrolling and the api through which it gets data is
https://www.omegawatches.com/de/vintage-watches?p=1&ajax=1
it returns a json data which contains different details including products in html format, and if the next page exist or not in a a tag with class link next
increase the page number till there is no a tag with link next class.
I need to extract the HTML content from different pages and put them in XML file. Is there a way to get complete HTML content from child pages of a group page in Sitefinity?
Get the page nodes you need with the Sitefinity API, loop through them and do a GET request, e.g. by using WebClient() and DownloadString method.
Then do whatever you wish with the html string.
Most of the news/blogs websites include RSS feeds link in their header. For example:
<link rel="alternate" type="application/rss+xml" title="Example Feed" href="http://example.com/feed/" />
I want to know what is the practical use of adding above? Is it to tell the browser that the website has RSS feeds? In past Firefox has the button in the address bar, but now they have remove it. Also if some user want to subscribe the RSS, he needs to enter the feeds url directly. So where it is being used? Thanks
Yes, it is to tell anything consuming the page (e.g. a browser) that there is an alternative form of the content elsewhere.
Most browsers used to all have an RSS button that would light up if it saw this, to allow you to subscribe to the feed.
RSS has (arguably) dropped in popularity, so this is less common today by default in browsers. But extensions and so on will still use it.
Or indeed, if you add a "normal" url to something like Google Reader, it will fetch that page, and look for an rss link in the head tag, to find the final feed url.
In my asp.net 3.5 C# application I had RSS feed in some of my web pages.I am using SyndicationItem object to prepare RSS feed and using using System.ServiceModel.Syndication dll to prepare RSS page.
I want them to be auto discoverable i.e The RSS feed button in IE at the browser level should get highlighted when we go to the page with RSS feed. Let me know how to do this
It depends how you define auto discoverable.
If you are referring to the way a web browser will place an RSS feed icon in the address bar (like on SO) then you need to place the following code into the HEAD section of a web page the user is viewing:
<link rel="alternate" type="application/atom+xml" title="Feed Title" href="http://rss-URL">
The code does not go in to the RSS feed. This approach will also allow search engines to find your feed.
It is also worth passing your feed via the http://validator.w3.org/feed/ as this will check it is valid and give you extra tips to ensure it is formatted in the best way.
You need to put the following meta tag into the header of your HTML page:
<link rel='alternate' type='application/rss+xml' title='RSS' href='/my_rss.html'>