PayPal Sandbox not able to find password field - selenium

Question

I'm currently using HtmlUnitDriver, and while I am able to set the username, I keep getting an error that Selenium could not find the password field. I am using JavascriptExecutor to set these values inside the PayPal sandbox form, but I'm still unable to get past the password step.
HtmlUnitDriver driver = new HtmlUnitDriver(BrowserVersion.CHROME)
JavascriptExecutor executor = (JavascriptExecutor)driver
driver.setJavascriptEnabled(true)
driver.get(url)
log.debug "setting username"
Thread.sleep(5000)
if(driver.findElement(By.xpath("//*[#id='email']")).displayed){
executor.executeScript("document.getElementById('email').value = 'email';")
log.debug "Username was set"
} else {
log.debug "We never set the username!"
}
if(driver.findElement(By.xpath("//*[#id='password']")).displayed){
executor.executeScript("document.getElementById('password').value='password';")
} else {
log.debug "We never set the password."
}
I understand that I am setting sleeps in there, and that's bad practice for Selenium testing, but I'm pretty much at my wits end here. The url in that case is the link to express checkout, which is something like this: https://www.sandbox.paypal.com/cgi-bin/webscr?cmd=_express-checkout&token=#################
Any help would be greatly appreciated!

Answer 1

As I mentioned earlier, there are multiple issues here:
When I load the initial page, I see this UI:
As you see, Password button is really invisible. So unless we click the Next button, there's no chance that this script will work.
So the following has to be added in order to achieve any kind of progress:
driver.findElement(By.id("btnNext")).submit()
However unfortunately I cannot get this button to be clicked correctly with HtmlUnitDriver. It seems button is clicked, but nothing happens, so Password field remains hidden. However as soon as I switch to ChromeDriver, this is not an issue anymore, and the same code works. So I guess you've hit one of the HtmlUnitDriver limitations, and need to use Chrome or Gecko driver.
Finally a few adjustments to the code will make it more reliable and faster. Here's the final version that works for "real" browsers (Chrome or Gecko):
WebDriver driver = new ChromeDriver()
WebDriverWait wait = new WebDriverWait(driver, 10)
driver.get(url)
// The following line waits for email field to appear. It's more economical and reliable than Thread.sleep
WebElement email = wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("email")))
log.debug "setting username"
email.sendKeys("email#gmail.com")
log.debug "Username was set " + email.getAttribute("value")
driver.findElement(By.id("btnNext")).submit()
// Here again using the same method to verify when password becomes visible
WebElement password = wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("password")))
log.debug "setting password"
password.sendKeys("password")
log.debug "Password was set " + password.getAttribute("value")
(note: I wrote this code in Java, so I hope I translated all correctly, but if I didn't feel free to fix it)
The result of this script looks like this in Chrome:
With HtmlUnitDriver, script will show an error:
org.openqa.selenium.TimeoutException: Expected condition failed: waiting for visibility of element located by By.id: email (tried for 10 second(s) with 500 MILLISECONDS interval)