BeautifulSoup returning 'None' when trying to find value of input - beautifulsoup

I have been attempting to use BS to find the value of an input field on a webpage. However, no matter what I try, it always returns as 'None' although the element certainly exists. Here is a sample of the HTML.
<td class="formArea"><table border="0" cellspacing="2" cellpadding="2">
<tr>
<td class="main">Street Address:</td>
<td class="main">
<input type="text" name="entry_street_address" id="entry_street_address" value="1234 Example Ln" maxlength="64" required> <span class="fieldRequired">* Required</span></td>
So I attempt to use BS4 to grab the value of 'entry_street_address':
r = session.get("sampleurl.com/wheremydataisstored")
time.sleep(3)
soup = bs4(r.content,'html5lib')
info = soup.find('input', {'id': 'entry_street_address'}).get('value')
print(info)
Unfortunately, this always returns:
AttributeError: 'NoneType' object has no attribute 'get'
This always happens. No matter if I do html5lib, lxml, html.parser, no matter how long I .sleep() to wait for the page to load, etc. I'm not really sure where it's going wrong!

Related

Web Automation - How do I input text into a rich text box on a website (textarea) defined by a class?

I have been trying this for a few days and am completely stuck. If anybody could help me out I would be very grateful; Im using VB.NET.
The HTML Code:
<tbody>
<tr>
<td class="chat-attachment-cell"></td>
<td class="chat-input-cell">
<textarea class="chat-message-input"></textarea>
</td>
<td class="chat-send-cell"><a href="#" class="chat-send-button"> alt="Send" width="66" height="66" border="0">
</a>
</td>
</tr>
</tbody>
The text box I need to input into is this bit
<textarea class="chat-message-input"></textarea>
Thankyou in advance for any help provided
You select the element then change the .innertext property with what you want.
There are multiple ways to do it, and I can't give you an example because I don't know what you are using nor the whole html, but for example it could look like this:
WebBrowser1.Document.GetElementById("someid").InnerText="Sometext"
For start, you can try looking how the collection of elements you get looks, then you should be able to figure out what to do next.
Dim test = WebBrowser1.Document.GetElementsByTagName("textarea")
For example on this page:
Dim test = WebBrowser1.Document.GetElementsByTagName("tbody")
test(1).InnerText = "Hi there"

Adding field to Qweb report

There is field account.move.line.journal_id and i want it to be displayed in report.
i'm trying by
<tr t-foreach="p.account_move_line" t-as="p">
<span t-esc="p.journal_id"/>
</tr>
or something like this.
<tr t-foreach="p.account_invoice.payment_move_line_ids" t-as="p">
<span t-esc="p.journal_id"/>
but getting error
AttributeError: 'NoneType' object has no attribute 'account_move_line'
Error to render compiling AST
AttributeError: 'NoneType' object has no attribute 'account_move_line'
Template: account.report_invoice_document
Path: /templates/t/t/div/div[4]/div[2]/table/tr[2]/td[2]/tr
Node: <tr t-foreach="p.account_move_line" t-as="p">
<span t-esc="p.journal_id"/>
</tr>
In t-foreach you must have the list you want to iterate. I think the error is that you are assigning the value to p and at the same time you are iterating by this variable.
Try changing the variable (this is taking into account that p is your account_invoice record, otherwise you can access directly without the t-foreach):
<tr t-foreach="p.account_move_line" t-as="j">
<span t-esc="j.journal_id"/>
</tr>
I hope I've helped ;)

Copy text from website using Selenium for python

I'm trying to copy text from a website with the following code and I want it to automatically click the "NEXT" button at the end of the table, without clicking it the code works just fine but when I add the last line to click it gives the error:
"Message: Element is no longer attached to the DOM Stacktrace: at
fxdriver.cache.getElementAt
(resource://fxdriver/modules/web-element-cache.js:9354)
at Utils.getElementAt (file:///c:/users/user/appdata/local/temp/tmpa7dvts/extensions/fxdriver#googlecode.com/components/command-processor.js:8978)
at WebElement.getElementText (file:///c:/users/user/appdata/local/temp/tmpa7dvts/extensions/fxdriver#googlecode.com/components/command-processor.js:11965)
at DelayedCommand.prototype.executeInternal_/h (file:///c:/users/user/appdata/local/temp/tmpa7dvts/extensions/fxdriver#googlecode.com/components/command-processor.js:12534)
at DelayedCommand.prototype.executeInternal_ (file:///c:/users/user/appdata/local/temp/tmpa7dvts/extensions/fxdriver#googlecode.com/components/command-processor.js:12539)
at DelayedCommand.prototype.execute/< (file:///c:/users/user/appdata/local/temp/tmpa7dvts/extensions/fxdriver#googlecode.com/components/command-processor.js:12481)"
The code I'm using is:
driver = webdriver.Firefox()
driver.get("https://it.website.com/Statistics")
wait = ui.WebDriverWait(driver, 10)
wait.until(page_is_loaded)
Next_first = 0
Next_first = driver.find_elements_by_id("next")[0]
data_vector [i]=driver.find_elements_by_class_name("tn")[i].text
Next_first.click()
While the website code is:
<tr>
<td class="tn"> text
</td>
<table class="table class" id="table id">
<thead>...code for table title...
</thead>
<tbody>
<tr>
<td class="tn">data
</td>
</tr>
</tbody>
<dd>
<a class="option clickable" id="next" data-page="1">NEXT</a>
</dd>
</tr>
Is there more than the html-source, like javascript?
Because the input could be detached by javascript, which means you have to wait for it to load properly ( driver.implicitly_wait(#seconds) )
here the link to the discription of "implicit-waits"
The second solution could be that you have to clear the cache/cookies of the webdriver before testing, which is described here

Selenium IDE: How to assert attribute exists

I'm using Selenium IDE.
I know how to test if an element's attribute has a certain value. But how do I test if the attribute exists in the first place?
Here is the line which successfully tests if the attribute has a certain value:
<tr>
<td>assertAttribute</td>
<td>id=_ctl0_MainPlaceHolder_dgMemberList_DXSelBtn0#disabled</td>
<td>disabled</td>
</tr>
Here is the line which unsuccessfully tests if the attribute exists in the first place:
<tr>
<td>assertElementPresent</td>
<td>id=_ctl0_MainPlaceHolder_dgMemberList_DXSelBtn0#disabled</td>
<td></td>
</tr>
How do I get it to work?
Thank,
-Ilya
You can use the "getAttribute" method, and if it returns "null" it means that there is no attribute at all...
From Selenium documentation:
WebElement.getAttribute( attributeName ) → Thenable<(string|null)>
Schedules a command to query for the value of the given attribute of the element. Will return the current value, even if it has been modified after the page has been loaded. More exactly, this method will return the value of the given attribute, unless that attribute is not present, in which case the value of the property with the same name is returned. If neither value is set, null is returned (for example, the "value" property of a textarea element). The "style" attribute is converted as best can be to a text representation with a trailing semi-colon.
For example:
HTML code:
<div id="a" >1</div>
<div id="a" x="xxx" >2</div>
<div id="a" x >3</div>
Test code in Nodejs:
var webdriver = require('selenium-webdriver');
var capabilitiesObj = {
'browserName' : 'firefox'
};
var driver = new webdriver.Builder().withCapabilities(capabilitiesObj).build();
driver.get('http://localhost/customer.js/temp/index60.html');
driver.findElements(webdriver.By.css('div[id=a]')).then((elements) => {
elements[0].getAttribute('x').then((att)=> {
console.log(att);
});
}).catch(console.log.bind(console));
driver.quit();
The result for each div will be different:
for div 1: null
for div 2: 'xxx'
for div 3: ''
I don't think the simple id locator handles the attribute. So switch to xpath:
This should work:
<tr>
<td>verifyElementPresent</td>
<td>xpath=//.[#id='id3' and #attribute1]</td>
<td></td>
</tr>
Note that if you change the specific id (id1, id2, id3) it will pass for the first 2 ids but fail for the 3rd:
<div id="id1" attribute1="xyz" style="background-color: transparent;">Some text</div>
<div id="id2" attribute1="" style="background-color: transparent;">Some text</div>
<div id="id3" style="background-color: transparent;">Some text</div>

Selenium webdriver-java: how to get javascript tooltip

I want to get tooltip information enabling on JavaScript. I have tried with below code but its shown null every time.
Java Code:
Actions action = new Actions(driver);
WebElement element = driver.findElement(By.xpath("//*[#id='content']/div/div/div[1]/div/div[2]/div[2]/div/div[3]/div[1]/table/tbody/tr[1]/td[2]/span/div/a"));
action.moveToElement(element).build().perform();
System.out.println(driver.findElement(By.xpath("//*[#id='content']/div/div/div[1]/div/div[2]/div[2]/div/div[3]/div[1]/table/tbody/tr[1]/td[2]/span/div/a")).getAttribute("data-original-title"));
HTML Code is:
<tr class="single" name="Tasks_59c777d9-8d16-694a-7307-52caad36d751">
<td>
</td>
<td data-type="custom_task_name">
<span class="list" sfuuid="3840" data-original-title="">
<div class="ellipsis_inline" data-original-title="">
Task Testing
<br/>
Toney Harber
</div>
</span>
</td>
I have tried with span tag but this also shows null.
I am providing you 2 options try both.
Option 1
Please try below. It will return you a list. Loop through it. If there is only one span then you can get the element and then get the attribute value.
driver.findElements(By.xpath("//span[#data-original-title]"))
Option 2
Instead of
driver.findElement(By.xpath("//*[#id='content']/div/div/div[1]/div/div[2]/div[2]/div/div[3]/div[1]/table/tbody/tr[1]/td[2]/span/div/a"))
use findElements
driver.findElements(By.xpath("//*[#id='content']/div/div/div[1]/div/div[2]/div[2]/div/div[3]/div[1]/table/tbody/tr[1]/td[2]/span/div/a"))
It will return a list of web elements. Loop thorough it you might get more than one element.