Is it possible to use Selenium so that my code and browser will be integrated - I want to get updated HTML page every time I made any change on the web page in the browser?
In other words I would like to run my app which would automatically start a browser and every time I do any change on the web page selenium automatically get changed HTML in java/python code. For example select a dropdown item might be a good example.
Thanks!
Related
I want to scrape a webpage, but it periodically changes the content, by getting additional information periodically using WebSocket.
So I have to scrape a webpage every time when it changes its content. Or, if it's possible, I can record the webpage content.
Is there any opensource or program that I can achieve this? Can Selenium do this?
I am crawling a website. I must change the date of the given job when suddenly I realized that the element is missing. When I screen capture it, the element is really missing. Is there any way to render that element? The website runs with Angularjs because I noticed the ng in the HTML code. Here are the pictures, the first one is the desktop capture and the second one is from the phantomjs.
Normal date in web browser
No date, straight to the next label
I found how to solve this, just wait for the angularjs to load on phantomjs. It takes an estimate of 5 seconds before it loads. The best to do here is setTimeout function.
Summary: When I try to go to the third page in a web site I'm trying to
screen scrape using Selenium and Chrome I can't find any elements on the third page.
I see Html for 2nd page in Driver.PageSource.
Steps:
Navigate.GoToUrl("LoginPage.aspx")
Find username and password elements.
SendKeys to Username and Password.
Find and click on Submit\Login button.
Web Site displays main menu page with two Link style menu items.
Find desired menu item using FindElement(By.LinkTest("New Person System")).
Click on link menu item. This should get me to the "Person Search" page (the 3rd page).
Try to wait using WebDriverWait for element on "Person Search" page. This fails to find element on new page after 5-10 seconds.
Instead of using WebDriverWait I then simply wait 5 or 10 seconds for page to load using Thread.sleep(5000). (I realize WebDriverWait is the better design option.
Try to find link with text "Person".
Selenium fails to find link tag.
I see desired "Person Search" page displayed in Chrome.
I see last page Html in ChromeDriver.PageSource.
Chrome geckodriver.exe. Build 7/21/2017.
NUnit 3.7.1
Selenium 3.4
VB
I used IE for another project with similar environment. Didn't have a problem getting to any page (once I coded Selenium correctly).
The web site I'm trying to screen scrape only supports recent IE version. I'm testing a legacy app for another project that requires IE 8. So using IE is out of the question.
Maybe I should try Firefox...
Does Selenium/Protractor look for element in current loaded page, or look in the entire application when using the same Css Selector for same elements?
Eg:
Save Button on Customers Form class="save"
Save Button on Vendors Form class="save"
Protractor works by interacting with the browser (via selenium). Selenium uses browser drivers to interact with your page, and the browser only contains the code that it asked for (returned from the server, based on the type of request that was made).
So yes, it only looks for elements in the currently loaded page. It has no access to your entire application code.
Selenium looks for the element only on the loaded page. Not sure about Protractor though.
I am writing a program to automate link validations in a site. Our site is having more than 400 links per page and we need to open each link and see it is returning a valid page i.e 200, there are other requirements as well to check if the page is a 404 redirection page etc. It means to validate 400 inks it will take about 30 minutes or so.
My design is to integrate this with the Front-End (Selenium) automation in a way that each time the browser loads a new page or browser refreshes it will trigger a new thread by passing the page source for validating all the href available.
We are not following a page object model otherwise I could trigger this in my each page.
Question here is that is there any way we can listen to a browser refresh or page load event using Selenium Web Driver?
Correct me if I don't understand your question, but page_refresh and page_load_event can be two very different goals for you, if you are dealing with AJAX. You can try this article about the AJAX part
and this one for selenium custom events synchronization.
This solution here is the most actual I could find.
Actually Selenium is JS driver so this answers can be helpful if you want to try it too:
check-if-page-reloaded-or-refresh-in-js
is-page-reloaded-or-refreshed-using-jquery-or-javascript
post_detect_refresh_with_javascript