Screenscraping a Datepicker with Scrapy + Selenium - selenium

Question

So i need to scrap a page like this for example or this and i am using Scrapy + Seleninum to interact with a datepicker calendar.
Basically i need to check for the availabilities of the rooms for each day in a month for example, so my idea was trying to click the not disabled links/dates on the datepicker and check if an error message appears or not for me to know if its available or not.
I have no idea if this is the proper way or even how to do this with this tools. I have been playing around with them but still looks like i am far from a final solution.
Does anyone know how i can tackle this or even provide the code for me to achieve this?

Answer 1

Hi please find the answer below
WebDriver driver = new FirefoxDriver();
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
driver.manage().window().maximize();
driver.get("https://www.airbnb.pt/rooms/516301?check_in=2016-04-14&guests=1&check_out=2016-04-17");
// selecting firstdate picker -- check in
driver.findElement(By.xpath("//*[#class='col-sm-6']/input")).click();
// note calendar is not completely visible hence to make it visible
// scroll a little bit down
((JavascriptExecutor) driver).executeScript("window.scrollBy(0,200)");
// take all calendar dates inside the list
// here i have update my code to skip stale element exception
List<WebElement> alldates = driver.findElements(By.xpath("//*[#class='ui-datepicker-calendar']/tbody/tr/td/a[contains(#class, 'ui-state-default')]"));
System.out.println("Size is : "+ alldates.size());
// suppose you want to select 27 form the all dates
// keeping it as a parameter
String dateToBeSelected = "19"; // check in
for(int i=0;i<alldates.size();i++){
alldates = driver.findElements(By.xpath("//*[#class='ui-state-default']"));
System.out.println("dates is : " + alldates.get(i).getText());
// selects a check-in date
if( alldates.get(i).getText().equals(dateToBeSelected)){
alldates.get(i).click();
break;
}
}
// on selection of Checkin date check out calender automatically pop ups
System.out.println("--------------------------");
String datetobeselected = "27";
for(int i=0;i<alldates.size();i++){
alldates = driver.findElements(By.xpath("//*[#class='ui-state-default']"));
System.out.println("dates is : " + alldates.get(i).getText());
// selects a check-in date
if( alldates.get(i).getText().equals(datetobeselected)){
alldates.get(i).click();
break;
}
}
// actually after calendar selection whatever text is shown in red color
// i am not sure what is it (sorry only English)
// identify the text with red color is present or not
boolean errorMsg = driver.findElement(By.xpath("//*[#class='panel-body panel-light']/div[2]/p")).isDisplayed();
if(errorMsg){
System.out.println("error msg is displayed");
}else{
System.out.println("error msg is not - displayed");
}
Hope this helps you also note this above example takes check-in and checkout date as parameter if you want code to take non disabled date automatically please feel free to ask i will update my answer.