I like to use scrapy and splash to grabb some data but poorly splash seems not to render the whole --> page <--.
The page should look like this:
But it looks like this:
So some of the more important information is missing. I already tried to increase the waiting time but this had no positive effect.
Does anyone have an idea what I could do, to make this work?
Try to look at Splash FAQ where common problems with page rendering are discussed. Especially, I've often seen problem with private mode. Here is described how to turn it off. You can check if private mode is the issue in Splash web console, just edit the default LUA script and add
splash.private_mode_enabled = false
Related
I'm working on a project and using AOS animation but all div blocks that have an animation on them disappear whenever I refresh the page. I want to trigger a function whenever the page is refreshed so I can use the AOS.refresh(). Is there any way to do this in nuxt.js or maybe a way to go around this AOS bug?
UPDATE:
here's an example this is how the page looks before reloading:
then after reloading:
As you can see. the elements that still appear after the reloading are the ones without AOS animation.
Additionally, if you focus your eyes on the sidebar, notice I have a home page. this problem happens literally everywhere except the home page. When I go to the home page then I go back to other pages the problem basically disappears.
Not sure where is your bug coming from but so far, the homepage of the library achieves to have an animation on a page refresh. Maybe it is a lifecycle code issue at some point. Maybe post some for us to help you debug this one.
To my knowledge, there is no way of watching if your webpage is actually refreshed because there is nothing related to state before this. The only somehow useful thing that may be used sometime is this.$router.history._startLocation, which will give you the first path your webapp was loaded on.
Not sure if it may help you anyhow.
Also, you can maybe give a shot to this library in VueJS and wire it to Nuxt. Should behave pretty much the same.
PhantomJS is a very cool tool for taking screenshots. But since it doesn't tell you how your HTML renders like until you save the image, it's quite tough to adjust small details with it. Is there any way to make the process easier?
For example, to output the display of the page rendered by PhantomJS live to somewhere so that I can see how it actually renders.
Thanks,
NOTE: New to this forum (UX/User Experience), so please let me know if this would be better in a different category. I searched Stack Exchange for "pinterest" and this forum seemed to have the most results. Thanks!
Hi guys. I'm writing a jQuery gist to grab links of all the images pinned to a given board in Pinterest. However, I've been running into the problem of having to repeatedly keep scrolling because all the results are not displayed on the same page. With the trendy "infinite scroll" or "lazy load" feature, one has to keep scrolling to the bottom without actually knowing if they are anywhere close because it seems to depend on your zoom percentage in your browser window and your window size as well, as to how many items display on your screen. I've been searching this for hours to no avail.
Searches I've already done keep returning non-productive results
The results I get when searching for
"Pinterest how to disable lazy loading" and "Pinterest how to disable infinite scroll"
keep returning the opposite of what I am looking for -- incorrect results for my purposes are anything like:
"How to add infinite scroll to my website",
"20 Useful Pinterest Tools",
or anything to do with adding infinite scroll.
The Problem: Infinite Scroll/Lazy Loading makes it hard for me to use browser plugins like jquerify (Chrome) and FireQuery (Firefox)
The issue for me is that I want to be able to view all my pins on a given board at once. Then I can use jQuery to manipulate all images on the page. Currently, infinite scroll makes it hard to keep track of where I'm at. I've tried stuff already by it's late at night and hard to remember everything. The important find was that in page source, Pinterest is using a "lazy" function. Here is what I found:
P.lazy = {
onImageLoad: function(a) {
var b = LOADED_CLASS;
P.overlap.isOverlappingViewport(a) && (b += FADE_CLASS);
a.className += b
}
};
This is just starting to be a deeper rabbit hole. I've checked for plugins to "remove", "disable", or "bypass" lazy loading, but haven't found any ... only those for adding it in.
Thanks in advance for your kind assistance and Cheers.
Pinterest loads cards via Ajax. When you scroll to the bottom of a page, browser javascript fires an Ajax call to load the next page full of cards.
This means it's not really possible to "disable" the infinite scrolling feature.
A few possible approaches:
Depending on how you're instantiating the browser, you might try setting or spoofing the window dimensions to a very large height. Pinterest may detect that height and attempt to load a window's worth of images, which may be enough to cover the feed you're trying to scrape.
If #1 is not practical for you, you can use javascript/jquery to keep scrolling the browser down until it has finished loading all the images. There are several ways to do this, since you are injecting javascript into the browser session.
(a) You can do this the "dumb" way with a loop that sets a timeout (setTimeout), then scrolls to the bottom (scrollTo()), then keeps going until the window stops scrolling and that comprises a kludgey auto-detect for the bottom of the page load.
(b) a more sophisticated approach would be to implement a listener for pinterest's ajax load function, (see the code, but it's a GET request to URL https://www.pinterest.com/resource/UserHomefeedResource/get/). An ajaxComplete() jQuery handler may help you detect the completion of a page load request so you can scrape the new images loaded.
Hope that helps
I am trying to recreate some effects similar to the div loading effects on this site i.e. there is no visible content when you load the page but upon clicking on a navigation link, it dynamically loads the divs.
http://worldofmerix.com
It is for a film studio website and I would like it to be interactive like this site. Does anyone know how I could achieve these effects with Javascript and/or jQuery?
Thanks for all the help in advance!
Have you used jQuery before or looked at the docs? This is really quite simple using jQuery's built-in animation effects such as fadeIn. The site you've linked to doesn't dynamically load the content - it's all part of the same page and simply displayed and hidden as appropriate.
Here's a rough fiddle showing how it works. Of course, you'd need to work out the styles and quirks in animations.
I'm not even sure what the name of that is to be able to make a search... but I would like to make those kind of things. Facebook has that too with the messages, notifications and friends requests. Thanks
I'm not sure if you expect anyone to give you a complete tutorial with source code included? :) You should probably do some digging around yourself, since a concrete answer on this could mean to write a few pages :)
How can you dig around?
Thé tool for a job like that is Firebug (IMO).
With bigger tasks like these it makes sense to try to split it up in smaller pieces.
Let's say you go for a widget like the user profile popup on SO.
you need some HTML to display in a popup: right click on any html element on the popup and click the 'inspect element' menu item. This brings you to the HTML tab in firebug. This allows you to figure out how the HTML is structured
you need some CSS to style that popup: when you're browsing the html structure, you might already have noticed that on the right side of it is the CSS that is applied to the active element
you might want to use some animation effects: for that you could use jquery. Have a look here to find out more on which effects are available and how they can be triggered. Fading is used in the profile popup on SO.
then you might ask yourself the question where SO get's that html structure from, right? To find out more about which server calls are made you can use the 'NET' tab in Firebug. (When you hover over your user name (only the first time?), then you should notice there's a call made to something like: http://stackoverflow.com/users/profile-link-stats?_=someLongNumberHere
In firebug you can then inspect the request and response. You should notice that the response is some HTML structure. This HTML structure is then inserted into the DOM.
Sooooo you can kinda glue it all together now:
the user hovers over his user name
the hovering triggers a server call (see step 4): use jquery hover to attach a handler to the user link. (subsequent hovers don't trigger that server call, so there needs to be a check to see if that profile popup was already loaded or not)
when the server call successfully returns (see jquery get), the returned html is inserted into the DOM and a fadeIn effect is triggered.
it seems a mouseout is used to fadeOut the popup
I HOPE this is the answer you were looking for. It took me a while ;)
You probably need to check out stackapps