I am trying to automate a page using selenium webdriver. The page contains a table which has the following xpath.
Find below a sample xpath validation which works fine.
I am iterating it through div as rows are considered as div under which there will be a table which has all the rows mentioned.
table = common.getObjectByXpath("html/body/div[4]/div[4]/div[1]/div/div/div[3]/div/div[2]/div/div/div/div");
rows = table.findElements(By.tagName("div"));
for(int i=1;i<=rows.size();i++){
if(driver.findElement(By.xpath("html/body/div[4]/div[4]/div[1]/div/div/div[3]/div/div[2]/div/div/div/div/div["+i+"]/table/tbody/tr/td[5]/span").getText().equals("endnode 11.1"))){
System.out.println(" print Something");
}
}
It works fine with xpath. But I want to do it with css selector. I am attaching the sample format of the table.
In the below figure, each div is considered as a row and under which you will see a table which has entries for columns.
The xpath you are using is a very fragile one since it starts right from the html element and heavily depends on the HTML structure of the page. Needless to say - it is huge and is not quite readable and easily understandable.
Instead rely on the element attributes - classes or ids. For example:
div#dojox_grid__View_11 div.dojoxGridContent div.dojoxGridRow
Related
I'm trying to scrape data from store: https://www.tibia.com/charactertrade/?subtopic=currentcharactertrades&page=details&auctionid=12140&source=overview
There is no problem with getting data from 1st and 2nd table, but when I goes down, xpath returns only empty lists.
even tried to save response in file:
scrapy fetch --nolog "https://www.tibia.com/charactertrade/?subtopic=currentcharactertrades&page=details&auctionid=3475&source=overview" > response.html
for table with skills everything works good
sword = response.xpath('//div [#class="AuctionHeader"]/a/text()').get()
but when it comes to getting for example gold value, I get only empty list:
gold = response.xpath('/html/body/div[3]/div[1]/div[2]/div/div[2]/div/div[1]/div[2]/div[5]/div/div/div[3]/div[2]/div[2]/table/tbody/tr/td/div/table/tbody/tr[2]/td/div[2]/div/table/tbody/tr[3]/td/div/text()').get()
In chrome/firefox both selectors works smooth, but in scrapy only 1st one
I know there might be some problems with data updated by javascript, but it doesn't look like this case
Doesn't look like it's a javascript problem. Think you're not getting your XPATH selectors correct. It's best to be as specific as possible and not to use multiple nodes down. Here we can select the attribute TableContent to get the tables you want. There you can select each individual table that you require if needed.
Code Example
table = response.xpath('//table[#class="TableContent"]')[3]
gold_title = table.xpath('tr/td/span/text()')[2].get()
gold_value = table.xpath('tr/td/div/text()')[2].get()
output
'Gold: '
'31,030'
Explanation
Using the class attribute TableContent, you can select which table you want. Here I've selected the table with the gold values. I've then selected each row and the specific element which has the gold value. The values are hidden behind span and div elements. get() returns a string, getall() returns a list.
If you're visually looking at a webpage and there is something clickable and unique on the page, you'll just click it. Without thinking about the page structure.
I'm interested to see what the most concise xpath is that could be constructed to simulate this in a versatile manner.
For example, target the "I'm feeling Lucky" button on the Google homepage:
//*[contains(#*, 'Lucky')]
The above works. But would fail in the element contained Lucky as inner text, or if the wrong case was specified. As such, our xpath needs to cater for any sensitivity and also look for the given string matching inner-text as well.
How could the above xpath be expressed in the most concise yet encompassing structure?
There is nothing thats very generic and executing such xpaths could be costly also at times.
"//*[contains(#*, 'Lucky')] | //*[contains(text(), 'Lucky')]"
Above is one xpath you can combine to get some results. You start specifying which nodes you don't to examine or ones which you want to examine
"//*[contains(#*, 'Lucky')] | //*[contains(text(), 'Lucky')][not(self::script|self::td)]"
And you can keep improving it
It's not possible to create a versatile XPath to accurately/reliability locate an element by text.
Why?
Because the text evaluated by an XPath is not necessary rendered in the page.
Because there's a hight chance to end-up with multiple matches since each ancestor also contains the expected text.
But mainly because there's too many rules/specific cases to consider.
But if I had to create one, then I'd start with this one:
"(html/body//*[not(self::script or self::style)][contains(concat(#value, normalize-space()), 'MyText')])[last()]"
Get all the descendants of the <body>
html/body//*
except <script> and <style>
[not(self::script or self::style)]
where the value attribute or normalize html contains 'MyText'
[contains(concat(#value, normalize-space()), 'MyText')]
then returns the last and deepest match
[last()]
http://i.stack.imgur.com/L4WUv.jpg
Link to Grid
I'm trying to detect the different drop downs on this page (depicted by the filters by the text boxes). The problem i'm having is that it seems that the filters all have the same ids. I can get the webdriver to find the initial filter button but not target the options in the drop down.
Note the filters I'm talking about are the ones from the funnel buttons. For example contains, isEqual, between etc *
This is wrong but an example
it('Should filter grid to -contain Civic', function() {
browser.element(by.id('ctl00_ContentPlaceHolder1_RadGrid1_ctl00_ctl02_ctl03_FilterTextBox_Model')).sendKeys("civic");
browser.element(by.id('ctl00$ContentPlaceHolder1$RadGrid1$ctl00$ctl02$ctl03$FilterTextBox_Model')).click();
browser.element(by.xpath("//*[contains(text(), 'Contains')]")).click();
})
NOTE The answer that was being looked for is at the bottom of this answer after the word "EDIT". The rest of this answer is retained because it is still useful.
It's a challenge to test webpages that dynamically generate ids and other attributes. Sometimes you just have to figure out how to navigate the stable attributes with an xpath. Here's an xpath that finds all four dropdowns:
//tr[#class='rgFilterRow']//input
To differentiate between each one, you can do this:
(//tr[#class='rgFilterRow']//input)[1] // Brand Name
(//tr[#class='rgFilterRow']//input)[2] // Classification
(//tr[#class='rgFilterRow']//input)[3] // Transmission
(//tr[#class='rgFilterRow']//input)[4] // Fuel
Using numbers to specify elements in an xpath isn't really desirable (it will behave incorrectly if the order of columns in the table changes), but it's probably the best you can do in this case because of all the dynamic ids and general lack of reliable identifying attributes.
EDIT
I misunderstood what you were trying to get because I didn't look at the image that you linked to. Once you've opened up that menu, you should be able to use an xpath to get whichever option you want by the text. For example, if you want the "Contains" option:
//a[#class='rmLink']//span[text()='Contains']
This page is highly dynamic. You had better brush up on your XPath, as nothing else will be able to help you. You can use this: http://www.zvon.org/xxl/XPathTutorial/General/examples.html .
Here is a simple example of how to access the Brand Name "pulldown". This is written in Groovy, which looks a lot like Java. If you know Java you should be able to get the idea from this:
WebElement brandName = driver.findElement(By.id("ctl00_ContentPlaceHolder1_RadGrid1_ctl00_ctl02_ctl03_BrandNameCombo_Arrow"))
brandName.click() // to open the "pulldown"
List<WebElement> brandItems = driver.findElements(By.xpath("//ul[#class='rcbList']/li"))
brandItems.each {
if(it.text == 'BMW')
it.click()
}
Unfortunately, the above id is not very reliable. A much better strategy would be something like:
WebElement classification = driver.findElement(By.xpath("//table[#summary='combobox']//a[contains(#id, 'ClassificationCombo_Arrow')]"))
Selecting its items is done similarly.
classification.click() // to open the "pulldown"
List<WebElement> classificationItems = driver.findElements(By.xpath("//ul[#class='rcbList']/li"))
classificationItems.each {
if(it.text == 'Sedan')
it.click()
}
If you are not up to the task, you should be able to get help from your development colleagues on how to locate all the elements in this page.
I am creating a framework for the data validation using selenium. The issue I am struggling with is I want to locate the element "td"(html tag) within element "tr"(html tag) . This is the code I have written.
Iterator<WebElement> i = rows.iterator();
While(i.hasnext()){
List<WebElement> columns = row.findElements(By.tagName("td"));
for(WebElement s:columns)
{
System.out.println("columnDetails : "+s.getText().toString());
}
if(columns.isEmpty())
{
ElementNotFoundException e = new ElementNotFoundException("No data in table");
throw e;
}
Iterator<WebElement> j = columns.iterator();// does some other work
ClusterData c = new ClusterData(); // does some other work
ClusterDataInitializer.initUI(c, j, lheaders); // does some other work
CUIData.put(c.getCN(), c); // does some other work
}
Now the issue with this is:
I am trying to fetch the data from the rows(see table data) in arraylist and use that arraylist further. Currently whats happening is the data for column header is fetched at start of which I have no use.I only want the rows's data. I am not able to determine the proper way to collect the data of table rows only.
if xPath of the table will help you understand it properly then here are the details :
Table header xPath of cluster name column:
/html/body/table/tbody/tr[2]/td[2]/div[2]/div/div/div[2]/div/div/div[2]/div/div/div[2]/div/div/div/div/div/div[2]/div/div/div[2]/div/div/div[2]/div[2]/div/table/tbody/tr/td[2]/div/div[2]
Table row (Table Data) xPath of test cluster 01:
/html/body/table/tbody/tr[2]/td[2]/div[2]/div/div/div[2]/div/div/div[2]/div/div/div[2]/div/div/div/div/div/div[2]/div/div/div[2]/div/div/div[3]/div[2]/div/table/tbody/tr/td[2]/div/div/a
Please let me know if you need anything else.
I am using the following code to extract row data from table.
List<WebElement> rows = getElement(driver,sBy,"table_div_id").findElements(By.tagName("tr"));
where sBy = By.id and table_div_id = id of div in which table is present. This extracts all the rows into arraylist and then i am using code to extract the row data into another arraylist. It is where I am stuck.
Each row from the table is in its own "table" tag so following things are not working :-
List<WebElement> rows = driver.findElements(By.xpath("//div[#id = 'table_div_id']//tr"));
List<WebElement> columns = row.findElements(By.xpath("./td"));
or the approach I used for the previous release of product i.e.
List<WebElement> columns = row.findElements(By.tagName("td"));
So, I used following approach which enabled me to capture all of the visible rows from the table.
List<WebElement> columns = row.findElements(By.xpath(".//table[#class='gridxRowTable']/tbody/tr"));
But after that I faced another issue that is since this table was implemented using dojo, the scrolling was impossible and Selenium was only able to capture the visible rows , so to overcome this I zoomed out in the browser using selenium. This is how i achieved my goal of getting the data.I believe others might have provided me answer if i would have shared some more details. Still , sorry about that and hope my answer helps you all.
instead of
List<WebElement> columns = row.findElements(By.tagName("td"));
try using
List<WebElement> columns = row.findElements(By.xpath("./td"));
Check if this helps. This should give you the td elements. If I have not understood your issue, let me know.
You can use this way-
driver.findElement(By.Xpath("//table[#id=\"table1\"]/tbody/tr[2]/td[1]"));
Regards,
Anuja
Do you have selenium IDE installed? Perform storeText operation on the row you want to retrieve, then xpath will get populated in IDE. There will be multiple xpaths; the most reliable is xpath:position, use that to capture your rows.
And use firebug for better visibilty of your AUT.
Firebug and Selenium IDE are the most basic component of Selenium Framework development.
You can manipulate xpath as you want.
I'm trying to select an element given by:
/html/body[#id='someid']/form[#id='formid']/div[#id='someid2']/div[#id='']/div[#id='']/div[#id='']/table/tbody[#id='tableid']/tr[7]/td[2]
Now the html of that row I'm trying to select looks like this:
<tr>
<td class="someClass">some text</td>
<td class="someClass2">my required text for verifying</td>
</tr>
I need to check whether my required text for verifying exists in the page.
I used selenium.isTextPresent("my required text for verifying"); and it doesnt work
So now I tried with selenium.isElementPresent("//td[contains(text(),'my required text for verifying')]")
This works sometimes but occassionally gives random failures.
Tried with selenium.isElementPresent(//*[contains(text(),'my required text for verifying')]) too..
How do I verify this text on the page using selenium?
The problem is not with the page taking time to load. I took screenshots before the failure occurs and found that the page was fully loaded so that shouldnt be the problem.
Could someone please suggest any way to select this element or any way to validate this text on the screen?
Try locating it by CSS:
assertText(selenium.getText("css=.someClass2"), "my required text for verifying");
The above should give a better failure message than isElementPresent, but you can still use that with CSS locators:
assertTrue(selenium.isElementPresent("css=.someClass2"));
If there is an issue with the load times you could try waiting for the element to be present:
selenium.waitForCondition("var value = selenium.isElementPresent('css=.someClass2'); value == true", "60000");
Some other XPath locators that might work for you, if you prefer not to use CSS locators:
//td[contains(#class, 'someClass2')
xpath=id('tableid')/tr[7]/td[2]
xpath=id('tableid')/descendant::td[contains(#class, 'someClass2')][7]
I've never heard of selenium; but your initial XPath is unnecessarily fragile and verbose.
If an element has an id, it's unique; using such a long XPath just to select a particular element is unnecessary; just select the last element with the id. Further, I see that you're occasionally selecting xyz[#id=''] - if you're trying to select elements without id attributes, you can do `xyz[not(#id)] instead.
Assuming your initial XPath is basically correct, it would suffice to do something like this:
//tbody[#id='tableid']/tr[7]/td[2]
However, using a specific row and column number like that is asking for trouble if ever anyhow changes details of the html. Also, it's atypical to have id's on tbody elements, perhaps the table element has the id?
Finally, you may be running into space-normalization issues. In xml, multiple consecutive spaces are often considered equivalent to a single space, and you're not accounting for that. In particular, if the xhtml is pretty-printed and contains a line-break in the middle of your sought-after text, it won't work.
//td[contains(normalize-space(text()),'my required text for verifying')]
Finally, text() explicitly selects
child text nodes - so the above xpath won't select elements where the text isn't the immediate child of td (e.g. <td><b>my required text for verifying</b></td>) won't match. Perhaps you mean to look up the concatenated text vale of all descendents:
//td[contains(normalize-space(string(.)),'my required text for verifying')]
Finally, type conversion can be implicit in XPath, so string(.) can be replaced by . in the above, leading to the version:
//td[contains(normalize-space(.),'my required text for verifying')]
This may be slow on large documents since it needs to normalize the spaces and perform a string search for each td element. If you run into perf problems, try to be more specific about which td elements need to be inspected, or, if you don't care where the text occurs, try to reduce the number of "calls" to normalize-space by normalizing the entire doc in one go (e.g. via /*[contains(normalize-space(.),'my required text for verifying')]).