In this page :
https://www.bedbathandbeyond.com/store/product/o-o-by-olivia-oliver-turkish-modal-bath-towel-collection/5469128?categoryId=13434
I can see a button with "Add to Cart" text , I can also see it in dev tools.
But when the same page source is retrieved by ChromeHeadless using selenium, and my script searches for it, this text is not present.
I tried with selecting show page source in the browser, the source too did not have the "Add To Cart text"
Further I used a curl to GET page, "Add To Cart" wasn't in the returned page source either.
What am I doing wrong?
is the page hiding the button?
How can I check for its presence, for product availability check?
The elements you are looking for are inside the shadow DOM. You need to access the shadow root first. Hard to see exactly what is going on in the DOM without some trial and error, but something like this:
WebElement shadowHost = driver.findElement(By.cssSelector("#wmHostPdp"));
SearchContext shadowRoot = shadowHost.getShadowRoot();
WebElement addToCart = shadowRoot.findElement(By.cssSelector(".shipItBtnCont button"));
More info on Shadow DOM & Selenium — https://titusfortner.com/2021/11/22/shadow-dom-selenium.html
Related
How can I get page source of current page?
I make driver.get(link) and I am on main page. Then I use selenium to get other page (by tag and xpath) and when I get good page I'd like to obtain its page source.
I tried driver.page_source() but I obtain page source of main page not this current.
driver = webdriver.Chrome(ccc)
driver.get('https://aaa.com')
check1 = driver.find_element_by_xpath('/html/body/div[1]/div/div[2]/button')
check1.click()
time.sleep(1)
check2=driver.find_element_by_xpath('/html/body/div[1]/div[2]/div[2]/div[1]/div/a')
check2.click()
And after check2.click() I am on page with new link (this link only works by click not directly). How can I get page source for this new link?
I need it to change selenium for Beautiful Soup
I have used Webdriver and displaying sources of page
I'm doing a test on c# with selenium for this site http://onliner.by.
At first I have to be authorized on this site.
I found(by xpath) button with name"Вход" in the right upper corner and click on it.
Then there's refreshed and page changed, but the link remained the same (http://onliner.by).
And I need to enter login and password on this page and sumbit it. But I cannot do it.
I founded Xpath paths of this elements and I used this code:
//this doesn't work
driver.FindElement(By.XPath("//*[#id='auth-container__forms']/div/div[2]/form/div[1]/div[1]/input")).SendKeys("user");
driver.FindElement(By.XPath("//*[#id='auth-container__forms']/div/div[2]/form/div[1]/div[2]/input")).SendKeys("password");
driver.FindElement(By.XPath("//*[#id='auth-container__forms']/div/div[2]/form/div[3]/div/button")).Click();
How can I do it? I tried to use SwitchTo().Frame but it didn't help too.
I will be very grateful for help.
You have to wait element availablity like this
new WebDriverWait(driver, TimeSpan.FromSeconds(timeOut)).Until(ExpectedConditions.ElementExists((Locator Value)));
Summary: When I try to go to the third page in a web site I'm trying to
screen scrape using Selenium and Chrome I can't find any elements on the third page.
I see Html for 2nd page in Driver.PageSource.
Steps:
Navigate.GoToUrl("LoginPage.aspx")
Find username and password elements.
SendKeys to Username and Password.
Find and click on Submit\Login button.
Web Site displays main menu page with two Link style menu items.
Find desired menu item using FindElement(By.LinkTest("New Person System")).
Click on link menu item. This should get me to the "Person Search" page (the 3rd page).
Try to wait using WebDriverWait for element on "Person Search" page. This fails to find element on new page after 5-10 seconds.
Instead of using WebDriverWait I then simply wait 5 or 10 seconds for page to load using Thread.sleep(5000). (I realize WebDriverWait is the better design option.
Try to find link with text "Person".
Selenium fails to find link tag.
I see desired "Person Search" page displayed in Chrome.
I see last page Html in ChromeDriver.PageSource.
Chrome geckodriver.exe. Build 7/21/2017.
NUnit 3.7.1
Selenium 3.4
VB
I used IE for another project with similar environment. Didn't have a problem getting to any page (once I coded Selenium correctly).
The web site I'm trying to screen scrape only supports recent IE version. I'm testing a legacy app for another project that requires IE 8. So using IE is out of the question.
Maybe I should try Firefox...
I am trying to click the main menu items in seleniumhq.org but after clicking on the first link i am getting a StaleElementReferenceException:Element not found in the cache=perhaps the page was changed since it was looked up
Please provide solution to solve the above problem
Below was my code
WebDriver d=new FirefoxDriver();
d.get("http://docs.seleniumhq.org/");
d.manage().timeouts().implicitlyWait(100,TimeUnit.SECONDS);
List<WebElement> l=d.findElements(By.cssSelector("ul>li"));
for(WebElement e:l) {
e.click();
}
Thanks in advance
If you click on a link and you are taken to the different page or even if you stay in same page the DOM is refreshed. Those elements are not attached to DOM anymore. You need to write some code to come back to previous page if your click takes you to a different page or even if you stays in same page you have find the link on the fly to click instead of "e.click()"
Please help, I'm new in Selenium. I try to automate eCommerce website and I have problem to handle popup window. Here is the scenario:
Go to http://www.lampsplus.com
Click on Sale link in the header section.
Click on the 1st item/product displayed on the page. (This will show the product page).
On the product page, click on the red Add To Cart button. (This will add a product to cart and display a popup).
On the popup, click the dark-grey Continue Shopping button. (This will close the popup.)
I stuck on step 5 (Error message: Unable to locate element "Continue shopping button")
Here is my code before step 5:
WebDriver d1 = new FirefoxDriver();
d1.manage().window().maximize();
d1.get("http://www.lampsplus.com");
d1.findElement(By.name("hdr_sale")).click();
d1.findElement(By.xpath(".//*[#id='sortResultContainer60238']/a[2]/span")).click();
d1.findElement(By.id("pdAddToCart")).click(); //This is step 4
//Here is suppose to be some code which handles the popup - my problem
d1.findElement(By.id("aContinueShopping")).click(); //This is step 5
I'm aware about .getWindowHandle(); method. I tried several variations of it and none of them worked.
Can anyone give me an idea how to handle it. Many thanks! I use Java.
Note: I don't work for LampsPlus and not try to promote their products, this website was chosen for training purposes only.
The element aContinueShopping is contained within an iframe.
You'll have to switch to the iframe using:
WebElement frameID = d1.findElement(By.Css("#add-to-cart>iframe"));
d1.SwitchTo().Frame(frameID);
d1.findElement(By.id("aContinueShopping")).click();
There's no 'name' or 'id' on the iframe, so you'll have to use a WebElement or a numeric to find it.
Once you're done with that iframe, you'll switch back to 'top' by using:
d1.SwitchTo().DefaultContent();