I'm trying to retrieve data from a webpage forum however they do not have proper classes within their tags which I'm having trouble trying to get from.
Image
I've managed to retrieve the other results but the last one; I'm not sure how to extract it. I've tried following a few solutions given by others; etc container.div.a but it accesses the first one only, I'm not sure how to get the other one; using a css selector?
this is the the url link
try:
for container in _url.find_all('td', {'class': 'row1', 'valign': 'middle'}):
topic = container.select_one(
'a[href^="/topic/]"').text.replace("\n", "")
description = container.select_one(
'div.desc').text.replace("\n", "")
#review_or_guide is what im trying to edit
review_or_guide = container.find('a', href=True, {'style': 'float:right'}).text
if topic and description is not None:
#appends data to list
else:
None
except Exception as e:
print('Error.extractDataFromRow1:', e)
return None
So I'm trying to achieve
Topic: Greetings
Description: Hi how are you
Status: Guide
You can try this
review_or_guide = container.select_one("a:nth-of-type(2)").text.strip()
This should select the second element within them.
Here you can loop over the table row selector #forum_topic_list > table > tbody > tr and inside the loop you can select td:nth-child(3) > div > div:nth-child(2) > a
in jQuery
Or In css #forum_topic_list > table > tbody > tr > td:nth-child(3) > div > div:nth-child(2) > a to select all data
Related
I'd like to modify part of the text in a textarea with Selenium. The textarea seems almost as if it were read-only.
In this very simple example using a sample algo, it would be great to be able to change the stock id on this line:
context.aapl = sid(24)
... to something like:
context.aapl = sid(39840)
... which is the Tesla stock id. The variable name will no longer make sense, doesn't matter, just a start.
This Selenium code for me is able to open the sample with no login required.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
t = webdriver.Firefox() # t stands for tab as in browser tab in my mind
t.implicitly_wait(10)
t.get('https://www.quantopian.com/algorithms/')
o = t.find_element_by_xpath("//body") # o stands for object
o.send_keys(Keys.ESCAPE) # clearing the popup
o = t.find_element_by_link_text("Hello World Algorithm")
o.click()
''' for the fun of it if you want to run the backtest
o = t.find_element_by_xpath('//body')
o.send_keys(Keys.CONTROL + 'b')
o.send_keys(Keys.ESCAPE)
'''
print t.find_element_by_id('code-area').text
Here's the output from that
1
# Called once at the start of the simulation.
2
def initialize(context):
3
# Reference to the AAPL security.
4
context.aapl = sid(24)
5
6
# Rebalance every day, one hour and a half after market open.
7
schedule_function(my_rebalance,
8
date_rules.every_day(),
9
time_rules.market_open(hours=1, minutes=30))
10
11
# This function was scheduled to run once per day at 11AM ET.
12
def my_rebalance(context, data):
13
14
# Take a 100% long position in AAPL. Readjusts each day to
15
# account for price fluctuations.
16
if data.can_trade(context.aapl):
17
order_target_percent(context.aapl, 1.00)
That id is 'code-area'. The content includes margin numbers which might be a problem.
Next nested area is 'code-area-internal', seems the same.
Followed by these two.
<div class='ide-container' id='ide-container'>
<textarea class='width_100pct' id='codebox'>
In trying to obtain the content of the algorithm with 'codebox', content doesn't appear to be present, just u'' ...
>>> p = t.find_element_by_id('codebox').text
>>> p
u''
Attempt to do CTRL-A on it results in this exception...
>>> o = t.find_element_by_id('codebox')
>>> o.send_keys(Keys.CONTROL + 'a')
ElementNotInteractableException: Message: Element is not reachable by keyboard
If the text can be completely cut, then replace can done in Python and paste, that would be fine.
I wouldn't expect Selenium to be able to find and replace text, just surprised it finds a visible area for user input to be off limits from interactivity.
That textarea does have its own Find, and hoping won't have to resort to trying to use it as a workaround.
(The environment is an online IDE for stock market algorithms called Quantopian)
This is the one other thing I tried, with no apparent effect:
>>> t.execute_script("arguments[0].value = arguments[1]", t.find_element_by_id("ide-container"), "_new_")
Appreciate any pointers.
Textarea has style="display: none" attribute which means that you cannot get its content with text property. In this case you can use:
p = t.find_element_by_id('codebox').get_attribute("textContent")
To set new value to code field you can use:
field = driver.find_element_by_css_selector('div[role="presentation"]')
driver.execute_script("arguments[0].textContent = 'New value';", field)
But note that initially each code line in code field displayed as separate div node with specific value and styles. So to make new value looks exactly as code (in the same formatting) you can prepare HTML sample e.g.
value = """<div style="position: relative;"><div class="CodeMirror-gutter-wrapper" style="left: -48px;"><div class="CodeMirror-linenumber CodeMirror-gutter-elt" style="left: 15px; width: 21px;">1</div></div><pre class=" CodeMirror-line " role="presentation"><span role="presentation" style="padding-right: 0.1px;"><span class="cm-comment"># Comment for new code.</span></span></pre></div>"""
and do
driver.execute_script("arguments[0].innerHTML = arguments[1];", field, value)
The content of the algorithm with codebox which you are trying to extract is having the style attribute set to display: none;. So to extract the text you can use the following lines of code :
p = t.find_element_by_xpath("//div[#class='ide-container']/textarea[#id='codebox']")
t.execute_script("arguments[0].removeAttribute('style')", p)
print(t.get_attribute("innerHTML"))
Because as an input field on a site I am testing was created by the site developer with a random class, typeText will not work since Testcafe does not recognize it as a valid input field.
As a result, I am trying to use pressKey to simulate typeText to enter a string into the input field.
Unfortunately, the result is not what I was hoping for. Using .pressKey('L P') should result in LP appearing in the search box but instead LPL appears.
How do I solve this issue? I have no idea why there is another L appearing where it should not.
For reference, using typeText causes this to show as an error:
Uncaught TypeError: this.ui.searchBox.val is not a function
For further reference the selector looks like this:
#modals > div.selectra-dropdown > div > div.search > input[type="zax"]
On querying what the hell a "zax" was, I was told by the site developer that it was a made up classname used as a workaround for some problem and I am stuck with it.
Likely it means that TestCafe can't properly simulate keyboard events for your page. Please create a bug report and provide a page and a test that can be used to reproduce the issue.
TBH, I really hope you find your answer, but due to that type=[zax], for me its impossible to help here;
but nevertheless, you could simplify your code and try something like
replace this here this.empty_lp_group_search = Selector('#modals > div.selectra-dropdown > div > div.search > input[type="zax"]'); with this.empty_lp_group_search = Selector('#modals input[type="zax"]');
maybe also this this.empty_lp_group_select = Selector('#link-resources > div.modal-host > div.modal-element.modal-container > div > div.modal-body > div.form-container > div > div.lp-chooser-container > div > div > div > div > div'); with this this.empty_lp_group_select = Selector('#link-resources div.nth(12)');; this last one is based on this link here https://devexpress.github.io/testcafe/documentation/test-api/selecting-page-elements/selectors/functional-style-selectors.html#nth
I have some list elements:
< button id={"productBook"} className="fito-btn fito-btn-long" style={this.props.styles.btnBrandRevers} onClick={this.props.onOfferSelect.bind(null, product)}>
< FormattedMessage id='app.Book' defaultMessage='Book' />
< /button>
When asserting the element with id productBook as:
.assert.visible('button[id=productBook]')
I'm getting an error:
Testing if element <button[id=productBook]> is visible. Element could not be located. - expected "true" but got: "null"
I don't understand why this doesn't work for this specific element while it works for other elements. Is there some different way that list elements need to be verified?
Please help.
Try using the shorthand # for id:
assert.visible('button#productBook')
or
assert.visible('#productBook')
As per the HTML you have shared the id attribute of the <button> is dynamically generated. So you may not be able to assert through id. Instead you can assert the visibility of the FormattedMessage through the following Locator Strategy :
xpath :
//button[#class='fito-btn fito-btn-long']/FormattedMessage[#id=\"app.Book\"][#defaultMessage='Book']
Try the following:
.assert.not.elementPresent('tr[...]')
I am trying to iterate a table to get the results. The structure of the table looks as mentioned in the picture
When I open the page, there is only one with role="presentation" and I am able to retrieve the data using the below css locator.
driver.findElement(By.cssSelector("div[id^=dojox_grid__View_] div.dojoxGridContent div.dojoxGridRow:nth-child(1) tbody tr:nth-child(1) td:nth-child(6)")).getText();
When I scroll the page manually another tag is created dynamically with role="presentation" and has many rows(div.dojoxGridRow). I want to iterate these rows too.
Selenium is able to go only to first level (first ) and get the details. I am not sure how to reach the second level
I'm not sure about which element you're trying to access.
But you can access to all div with 'presentation' role (return a list):
driver.findElements(By.cssSelector("div[role='presentation']"));
If you're trying to access to each row under div with role presentation:
driver.findElements(By.cssSelector(".dojoxGridRow"));
If you want to get rows child of div with role 'presentation':
List<WebElement> presentations = driver.findElements(By.cssSelector("div[role='presentation']"));
for (WebElement presentation : presentations) {
List<WebElement> rows = presentation.findElements(By.cssSelector(".dojoxGridRow"));
// DO SOMETHING...
}
Hope that helps.
I have 3 custom dropdowns which open when clicked on "down arrow" image appended at the end, Code for the image is something like :
<img id="x-auto-2017" class="x-form-trigger x-form-trigger-arrow "
src="data:image/gif;base64,R0lGODlhAQABAID/AMDAwAAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==">
Image id are dynamic and are in order of like 2017, 2018 etc. So I cannot use contains and identify them uniquely.
I want to get them clicked one by one and select value from the dropdown. Please help how to identify them uniquely.
Below code should work for you:
List<WebElement> elements = driver.findElements(By.xpath("//img[contains(#class,'x-form-trigger x-form-trigger-arrow']"));
System.out.println("Number of drop downs on the page: " + elements.size());
for (WebElement ele : elements) {
ele.click();
//Do what ever you want
}
Thanks for your response, I used the position qualifier [N] as suggested by #Marcus, though have to hit and trial a bit but writing manual xpath on Chrome console I got the following answer to my query.
driver.findElement(By.xpath("(//img[starts-with(#id,'x-auto-2')])[2]")).click();