Can't figure out why the following selector in scrapy fails:
>>> response.css('a[href="./viewforum.php?f=18"]::text')[0].get()
IndexError: list index out of range
But the same thing works for the same page using Chrome:
> $$('a[href="./viewforum.php?f=18"]')[0].innerText
'Contribute and help the VideoLAN project'
To reproduce, use the following webpage:
scrapy shell "https://forum.videolan.org/viewtopic.php?f=18&t=29792"
In [1]: response.css('a[href*="./viewforum.php?f=18"]::text').get()
Out[1]: 'Contribute and help the VideoLAN project'
Edit:
a[href="./viewforum.php?f=18"]:
Selects every <a> element whose href attribute value is "./viewforum.php?f=18"
a[href*="./viewforum.php?f=18"]:
Selects every <a> element whose href attribute value contains the substring "./viewforum.php?f=18"
Since the href value is "./viewforum.php?f=18&sid=be611e......0d35", and we want to match the value without the 'sid' part we use the selector '*='.
Related
I need to extract an attribute "sub elemement" after initial selector.
I'm using this syntax as selector:
${first_video}= Get WebElement xpath:(//div[contains(#data-store, 'videoID')])[1]
This correctly give me this single element part:
<div class="alpha" data-store="{"videoID":"1234","playerFormat":"inline","playerOrigin":"video_home","external_log_id":null,"external_log_type":null,"playerSuborigin":"entry_point",
"useOzLive":false,"playOnClick":true,
"videoScrollUseLowThrottleRate":true,"playInFullScreen":false,"type":"video","src":"https:\/\/video.test.com\/v\/a\/123_n.mp","width":320,"height":180,
"videoURL":"https:\/\/www.test.com\/test\/videos\/12345\/","disableLogging":false}" data-sigil="Video">
</div>
My goal is to get the VideoURL attribute.
I've tried with
${first_video_link}= Call Method ${first_video} value_of_css_property data-store.videoURL
but it gaves me empty output.
Is there any specific syntax that I'm missing?
Many Thanks
Get the value of the data-store atrribute:
${val}= Get Element Attribute ${your locator}
It appears to be a proper json, so convert it to a python dictionary and then just access its key:
${as json}= Evaluate json.loads("""${val}""") json
Log To Console The videoURL is ${as json['videoURL']}
I am new to this GUI Scripting.... My requirement is I want to click on the drop down arrow, & inside there are some elemets with tree structure to select the particular node/alarm..
So Intially I need to crack the XPATH for the drop down arrow of "COUNTERS" enter image description here
enter image description here
Tired Xpath by using absolute Xpath as below :
WebDriverWait(self.driver, 70).until(
EC.element_to_be_clickable((By.XPATH,
"/html/body/div/pm-application/epic-layout/div/div/div/div[5]/div[4]/epic-layout-panel/div/div/div/pm-reporting/epic-layout/div/div/div/div[3]/div[4]/epic-layout-panel/div/pm-tree-wrapper/div/div/div[1]/div[2]/div[1]/div[2]/div/div[2]/div/div[1]/div/div/div[3]/span/i")))
Also with Relative paths as well :
WebDriverWait(self.driver, 70).until(
EC.element_to_be_clickable((By.XPATH, "//div[#id='1483f603-ed09-6586-16e8-fc074ea8f908']//span//i[#class='icon tree-node-arrow collapsed']")))
WebDriverWait(self.driver, 70).until(
EC.element_to_be_clickable((By.XPATH,
"//*[#class='virtual-tree']//*[#class='icon tree-node-arrow collapsed']")))
WebDriverWait(self.driver, 70).until(
EC.element_to_be_clickable((By.XPATH,
"//i[#class='icon tree-node-arrow expanded']")))
But still I am facing below issue , Please help me to resolve this issues
AttributeError: 'str' object has no attribute 'find_element'
Please read about CSS selectors and Xpath selectors. Both of them don't like white or empty spaces.
It is a good practice to give Selenium the element actual tag, and not just asterisk sign.
Here is an example of the find element using CSS selector:
"div[class*='tree-node']"
This selector will return all the div elements that their class attribute contains the phrase "tree-node".
Using xpath you can find element by their attributes as well, or by the text written inside them:
"//div[contains(text(),'333')]"
This selector will give all div elements that have the text 333 inside them
So very new here to Selenium but I'm having trouble selecting the element I want from this website. In this case, I got the x_path using Chrome's 'copy XPath tool.' Basically, I'm looking to extract the CID text (in this case 4004) from the website, but my code seems to be unable to do this. Any help would be appreciated!
I have also tried using the CSS selector method as well but it returns the same error.
chrome_options = Options()
chrome_options.add_argument("--headless")
chrome_options.binary_location = '/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary'
driver= webdriver.Chrome()
chem_name = "D008294"
url = "https://pubchem.ncbi.nlm.nih.gov/#query=" + chem_name
driver.get(url)
elements = driver.find_elements_by_xpath('//*[#id="collection-results-container"]/div/div/div[2]/ul/li/div/div/div/div[2]/div[2]/div[2]/span/a/span/span')
driver.close()
print(elements.text)
As of now, this is the error I receive: 'list' object has no attribute 'text'
Here is the xpath that you can use.
//span[.='Compound CID']//following-sibling::a/descendant::span[2]
Why your script did not worked: I 2 issues in your code.
elements = driver.find_elements_by_xpath('//*[#id="collection-results-container"]/div/div/div[2]/ul/li/div/div/div/div[2]/div[2]/div[2]/span/a/span/span')
driver.close() # <== don't close the browser until you are done with all your steps on the browser or elements
print(elements.text) # <== you can not get text from list (python will through error here
How to fix it:
CID = driver.find_element_by_xpath("//span[.='Compound CID']//following-sibling::a/descendant::span[2]").text # <== returning the text using find_element (not find_elements)
driver.close()
print(CID) # <== now you can print `CID` though browser closed as the value already stored in variable.
Function driver.find_elements_by_xpath return list of Element. You should loop to get text of each element,
Like this:
for ele in print(elements.text):
print(ele.text)
Or if you want to match first Element, use driver.find_element_by_xpath function instead.
Using xpath provided chrome is always does not work as expected. First you have to know how to write xpath and verify it chrome console.
see these links, which helps you to know about xpaths
https://www.guru99.com/xpath-selenium.html
https://www.w3schools.com/xml/xpath_syntax.asp
In this case, first find the span contains text Compound CID and move to parent span the down to child a/span/span. something like //span[contains(text(),'Compound CID']/parent::span/a/span/span.
And also you need to findelement which return single element and get text from it. If you use findelements then it will return list of elements, so you need to loop and get text from those elements.
xpath: //a[contains(#href, 'compound')]/span[#class='breakword']/span
you can use the "href" as your attribute reference since I noticed that it has unique value for each component.
Example:
href="https://pubchem.ncbi.nlm.nih.gov/substance/53790330"
href="https://pubchem.ncbi.nlm.nih.gov/compound/4004"
I am not able to find the element using "id" in selenium as the id is randomly changing in every session of execution so the same id i am not getting in next execution.
As there is no other unique property is there to identify the element.
code sample
You didn't specify a language so I'm going to give you Java. You can do this by using the CSS class or probably a better choice (because of likely uniqueness) is data-lynx-name.
By CSS class
driver.findElement(By.cssSelector("div.http-lynx-json-org-text-input"));
By attribute
driver.findElement(By.cssSelector("div[data-lynx-name='username']"));
You really should read the question that I duped this one to:
Find element by attribute
Also read more about CSS selectors,
http://www.w3.org/TR/selectors/#selectors
You can use XPath.
String xpath = //div[#data-lynx-name='usernameLabel'][text='User ID']/following-sibling::div[1]
The above XPath will will find the div tag containing text 'User ID' and finds the next div which is is the required textbox.
It seems that you can even use the attribute 'data-lynx-name' attribute of the textbox div tag directly.
String xpath = //div[#data-lynx-name='username']
Selenium
driver.findElement(By.xpath(xpath));
I am using selenium for automating test cases for web application in that I have to get tool tip text
I tried
String result =element.getAttribute("span");
but this is retuning null. how can I get the text ?
element.getAttribute("span");
span must not be an attribute of the given element. I guess you've misunderstood, the getAttribute() method.
For e.g.
Example
For the above anchor tag, in order to get the title attributes' value, you can use :
element.getAttribute("title");
where element refers to the above anchor element.