I am trying to scrape webpages reviews.. For example:
https://www.chubbiesshorts.com/products/the-arnolds
Here the reviews are generated by plugin, running its script. I can not render those dynamic content using splash, nor the content is being render in splash UI("http://localhost:8050/info?wait=0.5&images=1&expand=1&timeout=3600.0&url=https%3A%2F%2Fwww.chubbiesshorts.com%2Fproducts%2Fthe-arnolds&lua_source=function+main%28splash%2C+args%29%0D%0A++assert%28splash%3Ago%28args.url%29%29%0D%0A++assert%28splash%3Await%28150%29%29%0D%0A++return+%7B%0D%0A++++html+%3D+splash%3Ahtml%28%29%2C%0D%0A++++png+%3D+splash%3Apng%28%29%2C%0D%0A++++har+%3D+splash%3Ahar%28%29%2C%0D%0A++%7D%0D%0Aend").
Can anyone help me to get those content.
Related
I have a application in Vue.js with a bunch of links in a navbar.
Each of the links need to open a new tab with some html that is specific to the link.
How could I implement this in Vue.js? I am thinking theres a way with vue router, but I am building the page dynamically with whatever html I get back from a server, so I'm not seeing how to do it.
Thank you
I have a dynamic page where I need to make an API request and update the Meta Tags dynamically.
I was able to achieve that by adding API request in App.js and updating the head tag HTML based on the response from API.
When I share the URL on social media, the tags are not showing up as it requires some loading time.
My application has a lot of dynamic pages with many routes. However, only 1 path in routes is important for me from an SEO standpoint.
What should be my approach here to make the Meta tags show up when shared on Social Media?
You can resolve this problem in two ways.
First: You need to turn on SSR Server Side Rendering. In this it will render all html on server side. Second: If you are using vue inside Laravel Blade template then you can handle meta and og tags on Laravel Blade file side. In this way before loading vue your meta and og tags will be loaded from Laravel.
Im building vue spa with vue router and on FAQ page i want to dynamically add meta title, description and permalink for each question (using accordion), so to show up them individually on google serp and on click, page to be auto scrolled on it.
Can you give me some hints how to do it?
A single page application isn't great for search engines. But if you don't want the hassle of moving your current vue project to another framework such as Nuxt.js, id recommend using vue-meta to add the meta tags to each page in your route, and the vue-prerender spa plugin, to build the static html files for search engines to scan the meta data of each page:
vue-meta:
https://www.npmjs.com/package/vue-meta
vue-prerender:
https://github.com/chrisvfritz/prerender-spa-plugin
I have been trying to scrape an Angular Website using Selenium. To my surprise it doesn't let you scrape the html rendered contents as it renders it dynamically using Javascript. I want to locate those tags for the purpose of scraping but I am unable to do so. What is the right way to scrape them? Here is some more context:
They say you can't do it using python.
Some also tried downloading all the html content and then read them. But again this isn't my use case.
But my use case is a lot different:
I want to login to my google account then it redirects me to an angular page where I click a button called reporting and from there I am redirected to a page from where I have to finally click download button to download the report.
I have this device data that are shown in real time using ThingsBoard.
And I have an iframe to show the device data content on a web page.
If I used the same iframe on my ionic app's HTML page, I am getting all the header, submenu and other unnecessary things that are not needed in the app.
What I need is the main content area of the device data section.
Can I do that with the iframe or do I need to call all the individual ThingsBoard API's to populate the dashboard on my own?
If you are managing the Thingsboard then you can develop custom iframes with unnecessary headers/footers removed. Then you will get clean iframes to display inside your Ionic app.
In case the Thingsboard is not managed by you then you have to develop adapters to scrape the html data from the Webpage. These adapters will provide your Ionic app only the data you require.
You can use PHP Simple HTML DOM Parser for implementing these data scraping adapters.
PHP Simple HTML DOM Parser