Kimono labs - extracting data from a page that scrolls - kimono

I am trying to extract data from a research platform, the first x number of responses show on the page but when I try and scroll down the page to teach Kimono to extract the other responses it cant seem them thus just provides the first page.
Pagination doesn't work as the research platform is presenting a single page which is scrollable.
Anything I can do to teach Kimono to find the additional responses.

They eliminated infinite scrolling capabilities mid 2015 so theres really no way.

Related

Razor Pages horizontal Table

Good morning,
I am developing my Web Application using .Net Core 3.1 and Razor Pages.
In my application I need to display a large amount of data on a grid and would therefor like to rotate a standard grid 90 degrees. I would like to obtain something like the "Horizontal Scroll" of this blog post but instead of just scrolling horizontally I would likle to move the headers to the side and, as mentioned above, rotate the entire grid fields.
I can't find any API that suites my needs though. Does anyone have suggestions on something I could try? I wouldn't want to have to build the entire grid myself (with all the paging, filtering and ordering logic). I am open for alternative solutions as well!!
Thanks everyone!
This seems more like an HTML question, and does not really relate to Razor Pages. Here are a couple of examples of horizontal scrolling HTML tables:
https://mdbootstrap.com/docs/jquery/tables/scroll/#datatable-horizontal-scroll
https://datatables.net/examples/basic_init/scroll_x.html
You said you need to display a large amount of data, I think the
design you currently want is not suitable for the display of a large
amount of data.
Because if the title is on the left, then only one data can be displayed on each page, and each page needs to load the title, which will lead to great redundancy.
If you just want to display beautiful because of too many columns, I suggest you use the responsive property in jQuery datatable, which can dynamically display the number of columns according to your current page width, and collapse or display the extra columns.
You can refer to this.

How do I increase website page width specifically on Mr. Parker Shopify theme

I am trying to make my website pages full width but I can't seem to figure out how to do it with my specific shopify theme, Mr. Parker. I think I have to make changes to the stylesheet.css.liquid, but I am struggling to figure it out. Is there a way to make my pages full width in the Mr. Parker theme?
I am a newbie and trying to do this on my own with very little background in this area. Any help is appreciated.
Amy
Hard way:
go learn css , html , js stuff to the end.
Easy way:
i don't know these stuff you said but i know pure CSS and HTML, easy way is to inspect element the page by chrome and check the biggest part of the page, like the <body> tag, then check the style tab and see what is the most top CSS file it's affecting from. change the file and you are ok!
Attention: Front-end developing is a huge field, elements on the page may affect from JavaScript then it could be so much hard for you to change things.
btw its very hard to edit responsive websites for the newbies, cause the elements may affect from multiple lines and codes and by many actions.

Programatically disable Lazy Loading or infinite scroll with jQuery in Pinterest?

NOTE: New to this forum (UX/User Experience), so please let me know if this would be better in a different category. I searched Stack Exchange for "pinterest" and this forum seemed to have the most results. Thanks!
Hi guys. I'm writing a jQuery gist to grab links of all the images pinned to a given board in Pinterest. However, I've been running into the problem of having to repeatedly keep scrolling because all the results are not displayed on the same page. With the trendy "infinite scroll" or "lazy load" feature, one has to keep scrolling to the bottom without actually knowing if they are anywhere close because it seems to depend on your zoom percentage in your browser window and your window size as well, as to how many items display on your screen. I've been searching this for hours to no avail.
Searches I've already done keep returning non-productive results
The results I get when searching for
"Pinterest how to disable lazy loading" and "Pinterest how to disable infinite scroll"
keep returning the opposite of what I am looking for -- incorrect results for my purposes are anything like:
"How to add infinite scroll to my website",
"20 Useful Pinterest Tools",
or anything to do with adding infinite scroll.
The Problem: Infinite Scroll/Lazy Loading makes it hard for me to use browser plugins like jquerify (Chrome) and FireQuery (Firefox)
The issue for me is that I want to be able to view all my pins on a given board at once. Then I can use jQuery to manipulate all images on the page. Currently, infinite scroll makes it hard to keep track of where I'm at. I've tried stuff already by it's late at night and hard to remember everything. The important find was that in page source, Pinterest is using a "lazy" function. Here is what I found:
P.lazy = {
onImageLoad: function(a) {
var b = LOADED_CLASS;
P.overlap.isOverlappingViewport(a) && (b += FADE_CLASS);
a.className += b
}
};
This is just starting to be a deeper rabbit hole. I've checked for plugins to "remove", "disable", or "bypass" lazy loading, but haven't found any ... only those for adding it in.
Thanks in advance for your kind assistance and Cheers.
Pinterest loads cards via Ajax. When you scroll to the bottom of a page, browser javascript fires an Ajax call to load the next page full of cards.
This means it's not really possible to "disable" the infinite scrolling feature.
A few possible approaches:
Depending on how you're instantiating the browser, you might try setting or spoofing the window dimensions to a very large height. Pinterest may detect that height and attempt to load a window's worth of images, which may be enough to cover the feed you're trying to scrape.
If #1 is not practical for you, you can use javascript/jquery to keep scrolling the browser down until it has finished loading all the images. There are several ways to do this, since you are injecting javascript into the browser session.
(a) You can do this the "dumb" way with a loop that sets a timeout (setTimeout), then scrolls to the bottom (scrollTo()), then keeps going until the window stops scrolling and that comprises a kludgey auto-detect for the bottom of the page load.
(b) a more sophisticated approach would be to implement a listener for pinterest's ajax load function, (see the code, but it's a GET request to URL https://www.pinterest.com/resource/UserHomefeedResource/get/). An ajaxComplete() jQuery handler may help you detect the completion of a page load request so you can scrape the new images loaded.
Hope that helps

Modifying photosphere on website thing

What i am trying to do is to use a photosphere on my website so that it shows up on full screen as a website cover page. The problem is the the code to embed a photosphere in a webpage given here by google
https://developers.google.com/photo-sphere/web/
lets only the photosphere size to be hardcoded as
displaysize="600,400"
what ever the values but its still hardcoded. What i want is that it gets adjusted to the screen of the user and gets displayed in the whole browser window. Any one got an idea how to pull it off? I didn't find any stuff about 'photosphere on web' other than the google link i gave above.
Indeed the API is currently designed to take static values. I think it's a good point that users might want to set the dimensions to 100% and let it resize dynamically.
I put it on the TODO list and will try to get to it shortly.
In the meantime, one work around is the following: After the viewer loads you will find an iframe on the page which contains it. You can change it's dimensions dynamically to your liking and the viewer should adapt.
The API provided by Google wraps the whole photosphere in layers of iFrames.
You can use the API to request a certain photosphere but only use the response to parse it for the values you need. Then you create your own request and the result can be shown fullscreen.
An example link is this
I created this link dynamically from the JSON response from the elements
media$group media$content 0 url
Hope it helps.
Can't you take the raw image and just use webgl to project it on the inside of a sphere?

Responsive design and content duplication issues

i am working on responsive design site. I have a large navigation contained in one UL but want to turn it to two UL's on smaller screens.
From a technical point of view using media query this is not pausing me a problem but means that the same links are twice in the source.
Also, on some of the pages we want to add a condensed content for the smaller screens, again, I could have the two variations of the content into two DIV's, with alsways one hidden depending on the device.
The question I have is about the search engines, I am guessing this would be seen as content duplication and could lead to penalties. What would be the best option then?
Repeat navigation 2 times will not create real problem: even if Google do not ignore one of the navigations, will just mean that these links have a bit more of importante in your page.
But about duplicate content, this is bad for one aditional reason that is not just a possible SEO problem: you will serve two times same content, and will waste bandwidth. If this will be a real problem, maybe not if all time at least one have on CSS display:none, but is a better idea if you try to think a way that with only CSS you can serve the same content for diferent display widths.