I have web page with strong ajax pagination (button for only next page).
For go to page etc number 5, script should press button Next 5 times.
But after script click - data for current page will lost.
It's possible return html content from Lua script to scrapy, and after this continue script run?
Now i use bad way. I merge html code for each page inside Lua script, and after last page i return it. But i think it's not good.
Related
I want to read 'title' of list of projects which are in pagination & 335 records almost.
What i am trying to do is :
1) First I get response of the browser by this command in windows cmd:
scrapy shell https://www.slingshotinsights.com/projects
2)It shows the HTML rendered in cmd, and very next i write
reponse.css('a.grey-link').extract()
and press enter, it gives me [] 'empty array'.
The question is, How to get data from scrapy script for the URL's which appears after login? Because https://www.slingshotinsights.com/projects is the link, which comes when user successfully login the page.
And may be scrapy is unable to find that
reponse.css('a.grey-link').extract()
css selector becuase it can't be loaded in logout view.
Is it possible to use Selenium so that my code and browser will be integrated - I want to get updated HTML page every time I made any change on the web page in the browser?
In other words I would like to run my app which would automatically start a browser and every time I do any change on the web page selenium automatically get changed HTML in java/python code. For example select a dropdown item might be a good example.
Thanks!
I have code in a desribe block but some of my set up code is outside of the block. That's the code that sets up elements.
When my code executes it runs fine but then I see the browser go to another page before parts of my test have completed. With a different page on the browser then it looks for elements and throws an exception when they are not there.
So how can I handle this problem of the browser going to another page before tests are completed?
As per my understanding of the question you want to be on the same page and don't want to move to the new page which loads when you click on to a link/button. If this is the case then to be on the same page you cane use.
Use this command -String oldwindow=window.getWindowHandle();
window.switchTo().window(oldwindow);
Use this command just below the button code which throws a new window. I am a java programer thats why the code written is in java.
getWindowHandle Methods stores the url of current window in oldwindow string and by using window.switchTo().window(oldwindow) it won't move to new window because you are passing oldwindow as argument for the command.
For more go through the Webdriver documentation by clicking this link
I have a button that executes an Ajax request and then it successfully redirects to another page.
How do I assert that the redirected page was successfully reached?
I have a clickAndWait on the button. But after that..?
you can use verifyTextPresent command to verify a unique lable or text in the redirected page.by that way you can fix you have successfully reached the redirected page.
try like this
command : verifyTextPresent
Target : some unique text in the redirected page.
i think your problem can be fixed by this.
IDE has many assert commands. You can use any of the one and you can achieve the test scope(here the page is navigated or not).
Example:
command : assertTitle
Target : check the title of the page.
In the above he used verifyTextPresent it will check weather the text is present or not in the page and it will continue the next step. If you use assert commands it will proceed the next step when the assert step is passed. Otherwise it fails.
One thing you keep in mind, selenium won't wait for Ajax kind of loading, it will wait for the Page loading. So, you have to put Wait commands explicitly to finish the Ajax loading.
You can get moreassertions when you convert the selenese code into the preferred language and testing framework. You can see that option in the Option tab in the IDE.
For more info on AssertCommands in SIDE
Is it possible to make a program that open a page (as if a bookmark file were opened by IE), and based on its content generate a feedback, that should be fedback in a textbox on said page by pressing a button on said page?
I need this program to execute on a set time schedule to feed some data to a web server based on time dependent web page data.
Yes, that is possible. It is generally called screen scraping. You basically retrieve the web page in question via a HTTP request, parse/analyze the page you got, then send back the data that should go into the textbox (again a HTTP request).
There are libraries to do that. Here is an article describing an example in Perl:
http://www.perl.com/pub/a/2003/01/22/mechanize.html