I'm trying to parse the HISTORICAL DATA that exists in the URL attached at the end of this question. The initial page view lists only around 45 rows of data, and I'm trying to get all of the data by changing the date range and then iterating over each page.
First, I change the date to start from 2000/01/01 and then click "Apply", then "Search". But I'm facing an issue that is redirecting me to the "OVERVIEW" page.
driver.get(url)
#entering the start date 'From'
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="themeBody"]/div[4]/div[3]/div/div[1]/input'))).send_keys('2000/01/01')
#clicking the 'Apply' button
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="themeBody"]/div[4]/div[3]/div/button[1]'))).click()
#clicking the 'Search' button
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="reloadHistoricalData"]'))).click()
Also I get this error:
Message:
Stacktrace:
#0 0x55d611b76553 <unknown>
#1 0x55d611861183 <unknown>
#2 0x55d611897619 <unknown>
The URL:
https://www.saudiexchange.sa/wps/portal/tadawul/market-participants/issuers/issuers-directory/company-details/!ut/p/z1/pZLLToNAFEC_pQvW93Z41h0OCAglmVJqmY2ZVvpIeKVSDf16p2hi2ihqnN0k58zknlzgsAReiZf9VrT7uhKFvGfceIz9iPpokdBz5yrahjOlszggiAgPPUAItcYTDSOMzLEEPAzYVFORqcD_5HtBbKLNbH9xt5CoRf7no_Y7H785Nv7kZ9I3PwFM04kEVMqYoxP0dEiAA3_OxWG9Y8f80EFGVOv8L798Gn3iSNMNQ4dK81b_AIbSXgJftBsEznF6YGD6e-Dbol69b8KubZsbBRVsxZN4PRaKrLOuy0ZUXdKVq1pC_WxJXl13uQpnGdCUaZouT_NN4878U7Sh9mj0BqnL79k!/p0/IZ7_NHLCH082KGET30A6DMCRNI2086=CZ6_NHLCH082KGET30A6DMCRNI2000=LA0=/#chart_tab2
I think you can simply prepare result date string and then clean input and paste it there.
as i see, the format is "Year/month/day - Year/month/day" So you can get current date using datetime( or any other date) to get your "start" date then add or subtract delta and get your "second" date.
in code it will looks like:
import datetime
from datetime import timedelta
first_date = datetime.datetime.now()
second_date = first_date + timedelta(days=5)
#in fact both can be any datetime objects
and you need to format it to result string using strftime function:
about: https://www.programiz.com/python-programming/datetime/strftime
#strftime convert datetime object to string using format
result_date = f"{first_date.strftime('%Y/%m/%d')} - {second_date.strftime('%Y/%m/%d')}"
in result you will get sth like this
and then you can clear your input
using .clear() or "Ctrl + A -> delete" method
input_date_element = driver.find_element_by() #here your selector
#by .clear() //not works for everytime :(
input_date_element.clear()
#by Ctrl + A -> delete // I prefer that because it is more "realistic"
from selenium.webdriver.common.keys import Keys
input_date_element.send_keys(Keys.CONTROL + 'a')
input_date_element.send_keys(Keys.DELETE)
In the end send you result date string:
input_date_element.send_keys(result_date)
remember to click on Search button :)
Related
I need to pass data in time format in "time" type element in "10:00 AM" format.
I am using following code:
public static void setShift()
{
txttime.sendkeys("1030AM");
}
this is not working. what is a correct way to enter such data?
Use Following Code :
It will work it for textbox/text area control
SimpleDateFormat formatter = new SimpleDateFormat("hh:mm a");
Date date = new Date();
txttime.sendkeys(date);
For the HTML input type datetime-local handling it from selenium is not ideal. It is not the most used date time picker, and it is not supported in firefox or safari.
For chrome, the date time format shows the format set in the browser's machine.
If you haven't changed anything, I'm guessing you are getting the format shown in the guru99 tutorial.
If that is the case, then you have missed that they also have provided the solution there.
After entering the date part you need to press tab to input the time part. Which is missing from your given code.
Try this:
First, input the date
WebElement dateBox = driver.findElement(By.xpath("//form//input[#name='bdaytime']"));
dateBox.sendKeys("09252013");
Second, press tab
dateBox.sendKeys(Keys.TAB);
Last, input the time
dateBox.sendKeys("0245PM");
Note:
If your machine has different DateTime formatting then this might not work. You have to check which part of the date time senKeys can actually input then split up that part and use Keys.TAB to press tab. Then input the next part until completion.
I am sending date of birth to an input element using selenium, data is getting entered in Chrome and Edge but in firefox that input element has an inbuilt datepicker and I am not able to inspect that default datepicker (right click doesn't work on datePicker and couldn't find any code for it). The input element is not taking values and I can't get the html element so I'm unable to fill the field.
Can anyone help me with this?
For Firefox, the date format is yyyy-mm-dd , regardless placeholder. I read Flaburgan's comment in https://github.com/mozilla/geckodriver/issues/1070 and searched documents, tested okay.
WebElement dateOfBirth = driver.findElement(By.name("dob"));
dateOfBirth.sendKeys("2012-12-24");
You handle this issue with the following trick:
Change the input type from date to text using JavaScriptExecutor by Selenium
After changing the element type to text the change the date field by sending text with the same format needed to change value, use sendKeys("dd/mm/yyyy").
OR try to use Selenium Actions move to element and click.
I've to select a user specified date and the snippet for picker is below.
sendkeys() functionality isn't working so I tried the below code.
JavascriptExecutor check = (JavascriptExecutor)driver;
check.executeScript("document.getElementById('hotel-checkin').setAttribute('value','10 Jan 2018')");
On entering the value, the date picker stays alive wherein the script fails to click the search button which is actually overlapped by date picker.
Any leads would be of great help. Thanks in advance!
Ok, you asked for leads so here're some thoughts and wild guesses (as long as you didn't give us neither url nor html).
sendkeys() doesn't work because the input field most probably has an event handler attached that opens datepicker widget. So, your first try to set value through JS is OK. I use slightly different way (sorry, python code here): driver.execute_script("arguments[0].value = arguments[1];", webelement, value)
But this mightn't work because you're entering a value to the visible field and an actual field (supposed to be filled by the datepicker widget) used by the page (a form or JS code) is hidden and holds no value. Then, try to find that field and enter a value to it by JS method.
In my current project I couldn't find that hidden field. I've decided to implement full user-like interaction with the datepicker widget. The scenario: click on the date field (datepicker widget opens), click on year or month selector buttons, then click on a day wanted. This way widget sets relevant data in a relevant fields (whatever and wherever they are) and closes.
Why I said fields (plural)? My recent thought was: May be setting the visible date field is right but there's some other field that must be set (like, say, date_was_set = "true").
Hope, this was helpful.
Edit. As for the 3rd paragraph. Here is my somewhat edited example (python, again).
The datepicker looks like this
As you can see it has buttons to adjust month and year combined.
from selenium.webdriver.support import expected_conditions as EC
class SetValCalendarStrategy(object):
def __init__(self, driver, calendar_param):
self.driver = driver
# this object holds parameters to find the calendar itself and its components
# pairs:
# sel_type - selector type ('xpath', 'id', 'name' etc.)
# sel_value - selector value to find ("//tr/td" etc)
self.param = calendar_param
def __call__(self, field, value, timeout):
"""
:param <webelement> field - input field
:param <datetime> value - value (date) to set
"""
# initiate datepicker with click on date input field
field.click()
# wait for a widget to show
cal = WebDriverWait(self.driver, timeout, 0.3).until(
EC.visibility_of_element_located(
(self.param.sel_type, self.param.sel_value)))
# decrease month/year button
prev_button = cal.find_element(
self.param.prev_month.sel_type,
self.param.prev_month.sel_value)
# increase button
next_button = cal.find_element(
self.param.next_month.sel_type,
self.param.next_month.sel_value)
today = datetime.now()
# calculate difference in months between today and the target date
month_diff = value.month + (value.year - today.year) * 12 - today.month
# select month/date
if month_diff < 0:
button = prev_button
else:
button = next_button
for i in range(abs(month_diff)):
button.click()
# template looks like this. It selects only days from target month (bold font)
# "//div[contains(#class, 'datePickerDay') and not(contains(#class, 'datePickerDayIsFiller')) and text()='{}']"
# insert day (21) into template. then it becomes
# "//div[contains(#class, 'datePickerDay') and not(contains(#class, 'datePickerDayIsFiller')) and text()='21']"
day_picker_sel_value =
self.param.day_picker.sel_template.format(value.day)
day = cal.find_element(
self.param.day_picker.sel_type,
day_picker_sel_value)
day.click()
I couldn't click the depart, arrvie dropdown menu form
https://m.flyscoot.com/search
I tried to use Xpath to click them, But I got no luck:
#driver.find_element(:xpath, "//~~~~~" ).click
And is there any way to let me fill the date value manually.
It's impractical to click the date by mouse, if I want to click the whole year.
The more practical solution is generating dates of whole year then fill it in, but I could't get it with selenium firefox driver.
No luck by xpath with contains method:
#driver.find_element(:xpath, "//*[contains(text(),'Departure Airport')]")
NoSuchElementError: Unable to locate element:
[4] pry(#)> arrival = #driver.find_element(:xpath, "//[contains(text(),'Departure Airport')]")
Selenium::WebDriver::Error::NoSuchElementError: Unable to locate element: {"method":"xpath","selector":"//[contains(text(),'Departure
Airport')]"}
from [remote server] file:///var/folders/6d/th4jpm90n1cx8h2l3kr49fr0000gn/T/webdriver-profile20150612-45453-15z20qu/extensions/fxdriver#googlecode.com/components/driver-component.js:10271:in
`FirefoxDriver.prototype.findElementInternal'
webdriver
➜ workspace gem list selenium-webdriver
*** LOCAL GEMS ***
selenium-webdriver (2.45.0)
To get departure you could use(other ways too but I dont know what your source code so grabbing at strings here):
arrival = find_element_by_xpath("//*[contains(text(),'Departure Airport')")
departure find_element_by_xpath("//*[contains(text(),'Arrival Airport')")
As for setting the date, I would need to see your page source code. I would imagine that this could be accomplished using either straight JS, or Jquery with the driver.execute_script('script here') command.
Use below code to select airport:
driver.get("https://m.flyscoot.com/search");
// click on departure airport
driver.findElement(By.xpath("//div[#id='departureAirport-button']"))
.click();
// Select any of option from the list of airports,Here i m selecting "sydney"
driver.findElement(
By.xpath("html/body/div[6]/div/div[2]/div/div[3]/div/div/div/div/div[2]/div[1]/div/div[1]/div[5]/div"))
.click();
//Click on done
driver.findElement(By.xpath("//div[text()='Done']")).click();
And this is the code to select Date:
// Click on Departure Date
driver.findElement(By.xpath("//input[#id='date-depart-display']"))
.click();
// select date by clicking on particular date,Here i m selecting today's
// date.
driver.findElement(
By.xpath(".//*[#id='ui-datepicker-div']/div[1]/table/tbody/tr[2]/td[6]/a"))
.click();
First of all, wait for the departure dropdown to be visible before interacting with it.
Also, use the Select abstraction and selectByVisibleText() (Java):
wait = WebDriverWait(driver, 10);
WebElement departure = wait.until(ExpectedConditions.visibilityOfElementLocated(By.ID("departureAirport")));
Select departureSelect = new Select(select);
departureSelect.selectByVisibleText("Hong Kong (HKG)");
WebElement arrival = driver.findElement(By.ID("arrivalAirport"));
Select arrivalSelect = Select(arrival);
arrivalSelect.selectByVisibleText("Melbourne (MEL)");
The same logic in Python:
from selenium import webdriver
from selenium.webdriver.support.select import Select
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Firefox()
driver.get("https://m.flyscoot.com/search")
wait = WebDriverWait(driver, 10)
departure = wait.until(EC.visibility_of_element_located((By.ID, "departureAirport")))
departure = Select(departure)
departure.select_by_visible_text("Hong Kong (HKG)")
arrival = Select(driver.find_element_by_id("arrivalAirport"))
arrival.select_by_visible_text("Melbourne (MEL)")
I tried this code, it partially works (at least the date is getting selected), But the selection on departure airport is not, I am still wondering why, Please see if this gives u any start. Mean while i shall get back if get some solution.
Driver.navigate().to("https://m.flyscoot.com/search");
Driver.findElement(By.id("departureAirport")).click();
Select Element = new Select(Driver.findElement(By.id("departureAirport")));
Element.selectByVisibleText("Singapore (SIN)");
Driver.findElement(By.id("date-depart-display")).click();
Thread.sleep(10);
Driver.findElement(By.xpath("//*[#id='ui-datepicker-div']/div[1]/table/tbody/tr[3]/td[4]/a")).click();
I have been trying to select the following value on the yahoo registration page(https://edit.yahoo.com/registration): Country dropdown for selecting country codes.
Code used:
new Select(driver.findElement(By.id("month"))).selectByVisibleText("July");
driver.findElement(By.id("selected-country-code-2")).click();
new Select(driver.findElement(By.id("country-code-rec"))).selectByVisibleText("Venezuela (+58)");
I keep getting the following error
org.openqa.selenium.ElementNotVisibleException: Element is not currently visible and so may not be interacted with
Command duration or timeout: 16 milliseconds
I can see few flaws in your code.
For selecting the month,day,year here is the code, that worked, tested on Firefox driver.
#org.junit.Test
public void testGoogle() throws Exception
{
driver.get("https://edit.yahoo.com/registration");
Thread.sleep(3000);
WebElement month = driver.findElement(By.id("month"));
WebElement day = driver.findElement(By.id("day"));
WebElement year = driver.findElement(By.id("year"));
Select monthDropDown = new Select(month);
Select dayDropDown = new Select(day);
Select yearDropDown = new Select(year);
monthDropDown.selectByVisibleText("July");
dayDropDown.selectByVisibleText("6");
yearDropDown.selectByVisibleText("2012");
}
Note that the country code, that your trying to select is not an select html tag, instead an div tag. So you cant use Select class here, from Selenium.
You have to come with our own algorithm to handle this.