I am trying to Crawl this site and get the links of each Job posting. I am using the Selenium and PhantomJS to render the webpage, but I am not able to find the links in the rendered content. This is the script I am using
public static void main(String[] args) {
// TODO Auto-generated method stub
String url="https://www.paycomonline.net/v4/ats/web.php/jobs?clientkey=D25120971391831BA4315C705AA7ABF1&jpt=";
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setJavascriptEnabled(true);
capabilities.setPlatform(Platform.LINUX);
RemoteWebDriver driver = getRemoteDriver("http://localhost:9515", capabilities);
driver.manage().window().maximize();
driver.get(url);
new WebDriverWait(driver, 360)
.until(RemoteWebDriver -> ((JavascriptExecutor) RemoteWebDriver)
.executeScript("return document.readyState").equals("complete"));
byte[] content = driver.getPageSource().getBytes();
String html = new String(content);
System.out.println(html);
}
I am running PhantomJS in my local machine at port 9515.Can someone help?
Related
I try to upload a pic on this site but exception comes up. Here is my code.
public static void main(String[] args) throws InterruptedException {
System.setProperty("webdriver.chrome.driver", "I:\\Selenium\\chromedriver.exe");
WebDriver driver= new ChromeDriver();
JavascriptExecutor js = (JavascriptExecutor) driver;
driver.get("https://jpg2png.com");
driver.manage().window().maximize();
js.executeScript("window.scroll(0,100)");
WebElement Pic=driver.findElement(By.id("pick-files"));
String js1 = "arguments[0].style.height='auto'; arguments[0].style.visibility='visible';";
Pic.sendKeys("D:\\University Work\\6th Semester\\IAP\\outline.jpg");
Thread.sleep(10000);
driver.quit();
}
}
The browser open website also open but uploading doesn't happen. type="file" is also not there in html code.
The Exception is:
Exception in thread "main" org.openqa.selenium.ElementNotInteractableException: element not interactable
try to click on upload button. As of now you enter a path to a file you want to upload but not clicking upload button
I am able to write scripts to give my email address into email element. But once click next through scripts Google uses ajax to replace that email element to password element dynamically. This is where I am stuck and won't be able to give password in that element and no login.
URL: https://accounts.google.com/signin/v2/identifier?flowName=GlifWebSignIn&flowEntry=ServiceLogin
Please write the selenium test scripts to achieve this.
Here is the code block to access the url https://accounts.google.com/signin login with your valid credentials and print the Page Title on your console:
String url = "https://accounts.google.com/signin";
driver.get(url);
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
WebElement email_phone = driver.findElement(By.xpath("//input[#id='identifierId']"));
email_phone.sendKeys("your_email");
driver.findElement(By.id("identifierNext")).click();
WebElement password = driver.findElement(By.xpath("//input[#name='password']"));
WebDriverWait wait = new WebDriverWait(driver, 10);
wait.until(ExpectedConditions.elementToBeClickable(password));
password.sendKeys("your_password");
driver.findElement(By.id("passwordNext")).click();
System.out.println(driver.getTitle());
driver.quit();
Console Output:
Google Accounts
Update(5-Jan-2020)
Optimizing the above code block and adding a couple of arguments you can use:
public class browserAppDemo
{
public static void main(String[] args) throws Exception
{
System.setProperty("webdriver.chrome.driver", "C:\\Utility\\BrowserDrivers\\chromedriver.exe");
ChromeOptions options = new ChromeOptions();
options.addArguments("start-maximized");
options.setExperimentalOption("useAutomationExtension", false);
options.setExperimentalOption("excludeSwitches", Collections.singletonList("enable-automation"));
WebDriver driver = new ChromeDriver(options);
driver.get("https://accounts.google.com/signin")
new WebDriverWait(driver, 10).until(ExpectedConditions.elementToBeClickable(By.xpath("//input[#id='identifierId']"))).sendKeys("emailID");
driver.findElement(By.id("identifierNext")).click();
new WebDriverWait(driver, 10).until(ExpectedConditions.elementToBeClickable(By.xpath("//input[#name='password']"))).sendKeys("password");
driver.findElement(By.id("passwordNext")).click();
}
}
I need to implement Headless Browser for HTTPS (validate certificate). For this I need to write extra line of code.
I have written for browser HTTP and it is working fine.
public class Headless {
public static void main(String[] args)
{
File src=new File("C:\\phantomjs-2.1.1-windows\\bin\\phantomjs.exe");
System.setProperty("phantomjs.binary.path",src.getAbsolutePath());
WebDriver driver=new PhantomJSDriver();
driver.get("https://www.google.co.in/");
System.out.println(driver.getTitle());
}
}
I google it and get some info:-
phantomjs --ignore-ssl-errors=yes;
DesiredCapabilities dcap = new DesiredCapabilities();
String[] phantomArgs = new String[] {
"--ssl-protocol=any",
"--ignore-ssl-errors=true"
};
dcap.setCapability(PhantomJSDriverService.PHANTOMJS_CLI_ARGS, phantomArgs);
PhantomJSDriver driver = new PhantomJSDriver(dcap);
You can read more about the command-line options on this link: Command Line Interface | PhantomJS
I am a new user of Selenium. I want to use it to start up the Chrome browser but I have a problem.
public static void processor(String url, String name) {
System.setProperty("webdriver.chrome.driver", "C:/Documents and Settings/jingxiong/Local Settings/Application Data/Google/Chrome/Application/chromedriver.exe");
WebDriver driver = new ChromeDriver();
driver.get(url);
WebElement element = driver.findElement(By.name(name));
element.sendKeys("google");
element.submit();
System.out.println("Page title is: " + driver.getTitle());
driver.quit();
}
When I run this example the Chrome browser starts ok but without configured plugins, my settings or bookmarks. What should I do to cause it load these?
Thank you.
You should first read chromedriver document in selenium wiki. Its available here - http://code.google.com/p/selenium/wiki/ChromeDriver
As mentioned in the wiki:-
Similarly, to load an extension when Chrome starts:
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
capabilities.setCapability("chrome.switches", Arrays.asList("--load-extension=/path/to/extension/directory"));
WebDriver driver = new ChromeDriver(capabilities);
I'm very new to selenium so I'm having trouble spotting the problem with my code. I'm using a webDriver backed selenium object, it starts the driver but never opens the URL and the driver just closes after a few moments. The last time this happened to me it was just because I had left "http" out of the URL. So what's causing it this time?
public void testImages() throws Exception {
Selenium selenium = new WebDriverBackedSelenium(driver, "http://www.testsite.com/login");
System.out.println(selenium.getXpathCount("//img"));
}
The setup looks like:
public void setUp() throws Exception {
System.setProperty("webdriver.chrome.driver", "C:\\Users\\User1\\Desktop\\chromedriver_win_16.0.902.0\\chromedriver.exe");
driver = new ChromeDriver();
Thread.sleep(2000);
}
The teardown method just consists of driver.close().
I'm using selenium 2.14 and the testNG Eclipse plug-in.
You might need to do the following
selenium.open("www.testsite.com/login");
Check out this example from the selenium site:
// You may use any WebDriver implementation. Firefox is used here as an example
WebDriver driver = new FirefoxDriver();
// A "base url", used by selenium to resolve relative URLs
String baseUrl = "http://www.google.com";
// Create the Selenium implementation
Selenium selenium = new WebDriverBackedSelenium(driver, baseUrl);
// Perform actions with selenium
selenium.open("http://www.google.com");
selenium.type("name=q", "cheese");
selenium.click("name=btnG");
// Get the underlying WebDriver implementation back. This will refer to the
// same WebDriver instance as the "driver" variable above.
WebDriver driverInstance = ((WebDriverBackedSelenium) selenium).getUnderlyingWebDriver();
//Finally, close the browser. Call stop on the WebDriverBackedSelenium instance
//instead of calling driver.quit(). Otherwise, the JVM will continue running after
//the browser has been closed.
selenium.stop();
link to selenium
You would need to add driver.get(url) like below.
public void setUp() throws Exception {
System.setProperty("webdriver.chrome.driver", "C:\\Users\\User1\\Desktop\\chromedriver_win_16.0.902.0\\chromedriver.exe");
driver = new ChromeDriver();
driver.get("http://www.testsite.com/login");
}