Why I can't find any selector with Xpath using two classes - scrapy

I got a html page like this
<li class="class1 class2"></li>
but I try to find this selector by using
response.xpath('//li[#class="class1 class2"]')
I always got None.
I also try to run this code:
response.xpath('//li[#class="class1"]')
response.xpath('//li[#class="class1 class2"]')
Both return None
Who can tell me what's wrong?
Maybe I resolved but I don't understand. I found a phenomenon that I use DevTools on Chrome, that shows like this <li class="class1 class2"><\li>. But I use scrapy to crawl and return <li class="class1 class2 class3"><\li>
I encountered some phenomenon that the web page source code is different from the source code I crawled.

Main issue is your <li><\li> is not valid HTML like <li></li> and xpath should look like:
.//li[contains(#class, 'class1') and contains(#class, 'class2')]
May use a tool like xpather to check if a path works or not.

Related

Using Selenium with Java on Salesforce Lightning / Javascript problems

We had Selenium tests running on Sales Force non Lightning. Now I am converting to Lightning. There is a chain of menus to go through. I have successfully navigated to a certain menu, but now I have run into a problem. This is not the standard question about new element names. Here is the question. This is a small piece of what the page looks like (see below).
When using inspect with Chrome I can find the elements. This is what it looks like (a small cross section corresponding to above). The SPAN tag has the label ("Original Agreement") and the DIV below it will contain the box for the input value.
<div class="slds-form-element__control" data-aura-rendered-by="228:1224;a">
<div data-aura-rendered-by="1108:0" class="uiInput forceSearchInputLookupDesktop uiInput--default" data-aura-class="uiInput forceSearchInputLookupDesktop uiInput--default">
<label class="label inputLabel uiLabel-left form-element__label uiLabel" for="157:1224;a" data-aura-rendered-by="1103:0" data-aura-class="uiLabel">
<span class="" data-aura-rendered-by="1104:0">Original Agreement</span>
<!--render facet: 1106:0--><!--render facet: 1107:0--></label>
<div data-aura-rendered-by="161:1224;a"><div class="contentWrapper slds-box--border" data-aura-rendered-by="162:1224;a">
However, Selenium can not find the elements (though inspect does). When I did a "View Page Source instead of an
inspect, almost the entire thing is in JavaScript like this:
function rewriteAndInjectCss(linkEl, source, varLookup) {
var css = rewriteCssVars(source, varLookup);
injectStyles(linkEl, css);
}
but a lot more. Almost all functions, with maybe only a couple elements.
In the past when I have seen something similar there is often an iframe to switch to to get the elements. But there is no iframe to switch to. So I am stuck how to get these. Can anyone shed some light?
OK. Found it. I needed to do a driver.switchTo().defaultContent()

How to get fully qualified url with selenium on a link without any href attribute?

I would like to retrieve url from a link on an html page.
unfortunately, html code does not contain any href attribute (I suppose it is managed by some javascript code)
Here is html code :
<p class="ng-scope">
<a class="documentLink ng-binding" data-document-id="21928499">Electronic document</a>
</p>
I tried to do it with getattribute() function :
By linkPodPopover = new ByXpath("//div[#class='popover-content']//a[contains(.,'Electronic document')]");
find(linkPodPopover).getAttribute("href");
but it returns an empty String...
I also tried with this code but also without success :
driver.getCurrentUrl()
click(linkPodPopover)
Do you see another way ?
I did not find any answer on the internet.
And I tried to explore every javascript attribute of the DOM element of my link without finding URL.
Finally, I came across this problem by using browserstack functionnalities : http://browserstack.com/automate/java#enhancements-uploads-downloads
It allows to click on the download link, then the browser download it. then using Javascript, I can check if file is well downloaded, and if size and md5 are correct. –

How to write dynamic xpath for img src

i am trying to automate couple of Selenium-TestNg scenarios from the website - http://ecommerce.saipratap.net/checkpersonaldetail.php
I was trying to click on the "continue "button, but it seems to be an image.
Below is the snippet of the code. How to write the xpath for the same?
tried using the xpath - //a[contains(#href,'checkoutshiping.php')], but it didnt work
<a href="checkoutshiping.php"> ==$0
<img src="images/continue.gif" border="0"
style="cursor:hand;"> ==$0
</a>
Are you sure the user is logged in? I see other set of tags when check the website.
Have you tried below xpath.
//*[.='Checkout']
Use - //a[#href = "checkoutshiping.ph"], its gonna find your button. I checked it on your website.
Here is example from your website

selenium + capybara: find selector anywhere within element

Assume we have a <div class='whatever'> and somewhere deep inside there is an element <div class='inside-whatever'>
What i need is a way to access that particular inside-whatever-div using Capybara's and/or Selenium's methods.
Problem is, there is another <div class='inside-whatever'> on the page not inside <div class='whatever'>, so
within(:xpath,'//div[#class="whatever"]') do
find(:xpath,'//div[#class="inside-whatever"])
end
returns an error basically saying that there are multiple inside-whatever divs on the page.
What works is to build the xpath from whatever like
'//div[#class="whatever"]/div/div[3]/div/div[5]'
but that is pure madness.
So, is there any better way to look for selector anywhere inside any given element without having to specify a direct path?
You can merge your xpaths like this:
//div[#class="whatever"]//div[#class="inside-whatever"]
The real issue here is that you've fallen into the XPath // trap
find(:xpath,'//div[#class="inside-whatever"])
searches globally rather than from the context node. Instead you should get used to starting your XPaths with .// which will search from the current context node
within(:xpath,'.//div[#class="whatever"]') do
find(:xpath,'.//div[#class="inside-whatever"])
end
and do what you expect. This is mentioned in the Capybara README - https://github.com/teamcapybara/capybara#beware-the-xpath--trap
Note: CSS selectors don't have this issue and for most elements people are selecting read cleaner, which is why Capybara defaults to the :css selector
within('div.whatever') do
find('div.inside-whatever")
end

Selenium object identification

I am using Selenium webdriver to test my application & i am facing difficulties in identifiying button on the same. the code snippet is like :
<input type="submit" onclick="return sign(this);" value="Login">
and its xpath is :
html/body/table/tbody/tr[2]/td/center/form/center/table/tbody/tr[3]/td/center/input[1]
Which object property to use and how?
You should not use that XPath.
I would hazard a guess that you used some sort of tool, whether it's Firebug or IDE, to generate that XPath. Stop that now!
XPath is fine to use, and can be used here, just not relying on the tools to generate it for you! That XPath is destined for failure!
You will need to provide more HTML, specifically around that button.
However, you should just be able to use something as simple as:
//input[#value='Login']
You can use the xpath, if that is really stable. I found that it is much easier to define id tags in the html elements and the use a By.id locator. Alternatively you can use css selectors, depending on the "uniqueness" of your button something like this could work:
By.cssSelector("input[value='Login']")