I am trying to extract the table data stored within a KML file. There are hundreds of points in this file and each point has a table embedded in its 'description'. I need the data for each point in a format that can be put into a csv or excel file.
Each table is formatted with the same headers designated by the th tag and then the data is designated with the td tag. I have included a copy of the code below.
I am attempting this with BeautifulSoup and have attempted a few different iterations, but am still quite the beginner here. I just get a blank return each time. I'd like the final result to end up with a string of all of the data, but with each element from a single table separated with a ',' and in a row and then the next table starting on a new line.
node = soup.find_all('description')
header = node.find_all('th')
info = header.find_all('td')
<Placemark id="ID_00004">
<Snippet maxLines="0"></Snippet>
<description><![CDATA[<html xmlns:fo="http://www.w3.org/1999/XSL/Format" xmlns:msxsl="urn:schemas-microsoft-com:xslt">
<head>
<META http-equiv="Content-Type" content="text/html">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body style="margin:0px 0px 0px 0px;overflow:auto;background:#FFFFFF;">
<table style="font-family:Arial,Verdana,Times;font-size:12px;text-align:left;width:100%;border-collapse:collapse;padding:3px 3px 3px 3px">
<tr style="text-align:center;font-weight:bold;background:#9CBCE2">
<td> </td>
</tr>
<tr>
<td>
<table style="font-family:Arial,Verdana,Times;font-size:12px;text-align:left;width:100%;border-spacing:0px; padding:3px 3px 3px 3px">
<tr>
<th>FID</th>
<td>4</td>
</tr>
<tr bgcolor="#D4E4F3">
<th>COLnum</th>
<td>14</td>
</tr>
<tr>
<th>Fieldnum</th>
<td></td>
</tr>
<tr bgcolor="#D4E4F3">
<th>Country</th>
<td>USA</td>
</tr>
<tr>
<th>State</th>
<td>Montana</td>
</tr>
<tr bgcolor="#D4E4F3">
<th>County</th>
<td>Dawson</td>
</tr>
<tr>
<th>Age</th>
<td>Upper Cretaceous</td>
</tr>
<tr>
<th>Stage</th>
<td></td>
</tr>
<tr bgcolor="#D4E4F3">
<th>AgeDate</th>
<td></td>
</tr>
<tr>
<th>Zone</th>
<td></td>
</tr>
<tr bgcolor="#D4E4F3">
<th>Map</th>
<td></td>
</tr>
<tr>
<tr>
<th>fauna</th>
<td></td>
</tr>
<tr bgcolor="#D4E4F3">
<th>Collector</th>
<td>C.A. White</td>
</tr>
<tr>
<th>Date</th>
<td>1882</td>
</tr>
<tr bgcolor="#D4E4F3">
<th>FaunaIndex</th>
<td></td>
</tr>
<tr>
<th>Reference</th>
<td>Monograph 32</td>
</tr>
<tr bgcolor="#D4E4F3">
<th>Comments</th>
<td></td>
</tr>
<tr>
<th>F22</th>
<td></td>
</tr>
<tr bgcolor="#D4E4F3">
<th>F23</th>
<td><Null></td>
</tr>
<tr>
<th>F24</th>
<td></td>
</tr>
<tr bgcolor="#D4E4F3">
<th>F25</th>
<td></td>
</tr>
<tr>
<th>F26</th>
<td></td>
</tr>
</table>
</td>
</tr>
</table>
</body>
</html>]]></description>
<styleUrl>#IconStyle00</styleUrl>
<Point>
<coordinates>-104.5651611501076,47.2247073758612,0</coordinates>
</Point>
</Placemark>
Related
I have a table 1 with a column NUMBER_OF_DIGITS and another table 2 with a column READING_VALUE
I want to get the values from table 2 which are nearing maximum value
Example: NUMBER_OF_DIGITS - 4 , maximum value is 9999 and I need to get the values nearest to 9999 from table 2
<!DOCTYPE html>
<html>
<head>
<style>
table, th, td {
border: 1px solid black;
}
</style>
</head>
<body>
<p>Table 1</p>
<table>
<tr>
<th>PK</th>
<th>NUMBER_OF_DIGITS</th>
</tr>
<tr>
<td>1</td>
<td>4</td>
</tr>
<tr>
<td>2</td>
<td>4</td>
</tr>
<tr>
<td>3</td>
<td>3</td>
</tr>
<tr>
<td>4</td>
<td>3</td>
</tr>
<tr>
<td>5</td>
<td>5</td>
</tr>
<tr>
<td>6</td>
<td>5</td>
</tr>
</table>
<p>Table 2</p>
<table>
<tr>
<th>PK</th>
<th>VALUE</th>
</tr>
<tr>
<td>1</td>
<td>1000</td>
</tr>
<tr>
<td>2</td>
<td>9990</td>
</tr>
<tr>
<td>3</td>
<td>900</td>
</tr>
<tr>
<td>4</td>
<td>45</td>
</tr>
<tr>
<td>5</td>
<td>99789</td>
</tr>
<tr>
<td>6</td>
<td>23456</td>
</tr>
</table>
</body>
</html>
Try this, the join you will have to figure out for yourself:
select to_number(substr('99999999999999999', 1, max_digits), '99999999999999999') - Table2.READING_VALUE as Val_Diff
from Table1
inner join Table2
on Somecommoncol = someothercommoncol
I am trying to get Selenium to be Data Driven with Microsoft TFS. I understand Selenium IDE but from what I've researched Selenium IDE cannot be used with a data driven format. What I'm trying to have the data driven section do is cycle through the repositories and add different TFS team and permissions to them. I can set up what I want in Selenium IDE but have no idea how to make it data driven with an excel sheet. Here is my HTML Source Code
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head profile="http://selenium-ide.openqa.org/profiles/test-case">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<link rel="selenium.base" href="http://MyTFS/" />
<title>DragAndDropPrologue</title>
</head>
<body>
<table cellpadding="1" cellspacing="1" border="1">
<thead>
<tr><td rowspan="1" colspan="3">DragAndDropPrologue</td></tr>
</thead><tbody>
<tr>
<td>open</td>
<td>/tfs/DefaultCollection/XO/_build</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>6000</td>
<td>6000</td>
</tr>
<tr>
<td>clickAndWait</td>
<td>//label[text()="Repo1;]/</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>1000</td>
<td>1000</td>
</tr>
<tr>
<td>click</td>
<td>link=Edit</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>1000</td>
<td>1000</td>
</tr>
<tr>
<td>click</td>
<td>xpath=//span[text()='Add build step...']</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>1000</td>
<td>1000</td>
</tr>
<tr>
<td>click</td>
<td>xpath=(//button[#value='Add'])[11]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>xpath=(//button[#value='Add'])[12]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>id=ok</td>
<td></td>
</tr>
<tr>
<td>dragAndDropToObject</td>
<td>//div[text()="Fetch the Quality Profile from SonarQube"]/</td>
<td>//div[text()="Ant"]/</td>
</tr>
<tr>
<td>pause</td>
<td>1000</td>
<td>1000</td>
</tr>
<tr>
<td>type</td>
<td>css=div.wrap > input[type="text"]</td>
<td>Sonar</td>
</tr>
<tr>
<td>type</td>
<td>id=projectKey</td>
<td>test</td>
</tr>
<tr>
<td>type</td>
<td>id=projectName</td>
<td>test</td>
</tr>
<tr>
<td>dragAndDropToObject</td>
<td>//div[text()="SonarQube for MSBuild - End Analysis"]/</td>
<td>//div[text()="Ant"]/</td>
</tr>
<tr>
<td>click</td>
<td>xpath=//span[text()='Save']</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>1000</td>
<td>1000</td>
</tr>
<tr>
<td>click</td>
<td>id=ok</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>1000</td>
<td>1000</td>
</tr>
<tr>
<td>clickAndWait</td>
<td>//label[text()="Repo2;]/</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>1000</td>
<td>1000</td>
</tr>
<tr>
<td>click</td>
<td>link=Edit</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>1000</td>
<td>1000</td>
</tr>
<tr>
<td>click</td>
<td>xpath=//span[text()='Add build step...']</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>1000</td>
<td>1000</td>
</tr>
<tr>
<td>click</td>
<td>xpath=(//button[#value='Add'])[11]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>xpath=(//button[#value='Add'])[12]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>id=ok</td>
<td></td>
</tr>
<tr>
<td>dragAndDropToObject</td>
<td>//div[text()="Fetch the Quality Profile from SonarQube"]/</td>
<td>//div[text()="Ant"]/</td>
</tr>
<tr>
<td>type</td>
<td>css=div.wrap > input[type="text"]</td>
<td>Sonar</td>
</tr>
<tr>
<td>type</td>
<td>id=projectKey</td>
<td>test</td>
</tr>
<tr>
<td>type</td>
<td>id=projectName</td>
<td>test</td>
</tr>
<tr>
<td>dragAndDropToObject</td>
<td>//div[text()="SonarQube for MSBuild - End Analysis"]/</td>
<td>//div[text()="Ant"]/</td>
</tr>
<tr>
<td>click</td>
<td>xpath=//span[text()='Save']</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>id=ok</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>1000</td>
<td>1000</td>
</tr>
</tbody></table>
</body>
</html>
I'm using the Selenium IDE to store the values of a table in a page in some variables and then type the text in those variables in a text box, one per line.
The problem is that when I loop through the variable ${card} the contents of the variable replace themselves while they should be one at a line.
I would also like to know if I'm using the right tool for doing this. Should I be using the Selenium Webdriver or something else?
Here is the code:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head profile="http://selenium-ide.openqa.org/profiles/test-case">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<link rel="selenium.base" href="https://es.magiccardmarket.eu/" />
<title>test</title>
</head>
<body>
<table cellpadding="1" cellspacing="1" border="1">
<thead>
<tr><td rowspan="1" colspan="3">test</td></tr>
</thead><tbody>
<tr>
<td>open</td>
<td>https://eu.magiccardmarket.eu/?mainPage=browseUserProducts&idCategory=1&idUser=13786</td>
<td></td>
</tr>
<tr>
<td>store</td>
<td>1</td>
<td>x</td>
</tr>
<tr>
<td>label</td>
<td>target1</td>
<td></td>
</tr>
<tr>
<td>storeTable</td>
<td>//div[#id='siteContents']/div[2]/div[2]/div/form/table.${x}.2</td>
<td>card${x}</td>
</tr>
<tr>
<td>storeEval</td>
<td>new Number(${x}) + 1</td>
<td>x</td>
</tr>
<tr>
<td>gotoIf</td>
<td>storedVars['x'] < 31</td>
<td>target1</td>
</tr>
<tr>
<td>open</td>
<td>http://www.quackit.com/html/codes/html_text_box_code.cfm</td>
<td></td>
</tr>
<tr>
<td>store</td>
<td>1</td>
<td>x</td>
</tr>
<tr>
<td>label</td>
<td>target2</td>
<td></td>
</tr>
<tr>
<td>storeEval</td>
<td>storedVars['card' + storedVars['x']]</td>
<td>card</td>
</tr>
<tr>
<td>type</td>
<td>name=comments</td>
<td>${card}<br /></td>
</tr>
<tr>
<td>storeEval</td>
<td>new Number(${x}) + 1</td>
<td>x</td>
</tr>
<tr>
<td>gotoIf</td>
<td>storedVars['x'] < 31</td>
<td>target2</td>
</tr>
</tbody></table>
</body>
</html>
Type sets the value of an input field. So each time you call type it's only setting it to the 3rd column.
Something like this:
<tr>
<td>storeValue</td>
<td>name=comments</td>
<td>com</td>
</tr>
<tr>
<td>type</td>
<td>name=comments</td>
<td>${com}\n${newvalue}</td>
</tr>
Or you can build up the var and use the type command once.
Frankly I would do this looping in javascript rather than use flow control, which I don't like.
I am using Selenium to record my actions on a webpage, however, when I play the test case, there is an click action throwing error: Element is not currently visible and so may not be interacted with. However, I am sure that button is visible, and here is html for it:
<button class="btn btn-success btnSubmit" style="font-size: 1.9em; width: 100%; height: 2em; margin-top: 20px;" type="submit"> Get Your Report </button>
Also, here is my test script for Selenium IDE:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head profile="http://selenium-ide.openqa.org/profiles/test-case">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<link rel="selenium.base" href="http://hra.case.edu/" />
<title>New Test</title>
</head>
<body>
<table cellpadding="1" cellspacing="1" border="1">
<thead>
<tr><td rowspan="1" colspan="3">New Test</td></tr>
</thead><tbody>
<tr>
<td>open</td>
<td>/</td>
<td></td>
</tr>
<tr>
<td>clickAndWait</td>
<td>link=Get Started</td>
<td></td>
</tr>
<tr>
<td>keyPress</td>
<td>name=searchFirstName</td>
<td>Tiancheng</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//div[#id='content']//tbody[#id='select_user']/tr[1]//a</td>
<td></td>
</tr>
<tr>
<td>clickAndWait</td>
<td>//div[#id='content']//tbody[#id='select_user']/tr[1]//a</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>id=answer_heightFoot</td>
<td>5</td>
</tr>
<tr>
<td>type</td>
<td>id=answer_heightInch</td>
<td>11</td>
</tr>
<tr>
<td>type</td>
<td>id=answer_weight</td>
<td>222</td>
</tr>
<tr>
<td>select</td>
<td>id=answer_bodySize</td>
<td>label=Medium</td>
</tr>
<tr>
<td>click</td>
<td>id=answer_diabetes_no</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>link=Other</td>
<td></td>
</tr>
<tr>
<td>clickAndWait</td>
<td>id=next</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[#type='submit']</td>
<td></td>
</tr>
<tr>
<td>verifyElementPresent</td>
<td>//button[#type='submit']</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[#type='submit']</td>
<td></td>
</tr>
</tbody></table>
</body>
</html>
Anyone has an idea of why?
Thanks in advance!
Some webelement may be present at DOM but be invisible at the same time. So, after you perform some action use waitForVisible method to wait for element to become visible and interactible
UPDATE: #Kees de Kooter, here's the working example
<tr>
<td>open</td>
<td>http://stackoverflow.com/</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>css=.js-help-button</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>css=.js-help-button</td>
<td></td>
</tr>
<tr>
<td>waitForVisible</td>
<td>css=.js-help-dialog</td>
<td></td>
</tr>
Configuration Manifesting
Windows 7.
FireFox 22.
RAM 2 GB.
The following error occurs when we try to click on login button:-
[error] Unexpected Exception: fileName -> chrome://selenium-ide/content/selenium-core/scripts/htmlutils.js, lineNumber -> 309, columnNumber -> 8
Source code:-
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head profile="http://selenium-ide.openqa.org/profiles/test-case">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<link rel="selenium.base" href="http://foxtel-cms.local/" />
<title>TC_4_RT</title>
</head>
<body>
<table cellpadding="1" cellspacing="1" border="1">
<thead>
<tr><td rowspan="1" colspan="3">TC_4_RT</td></tr>
</thead><tbody>
<tr>
<td>open</td>
<td>/</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>id=edit-name</td>
<td>qa_editor</td>
</tr>
<tr>
<td>type</td>
<td>id=edit-pass</td>
<td>qa_editor</td>
</tr>
<tr>
<td>clickAndWait</td>
<td>id=edit-submit</td>
<td></td>
</tr>
<tr>
<td>clickAndWait</td>
<td>link=Foxtel Content(active tab)</td>
<td></td>
</tr>
<tr>
<td>clickAndWait</td>
<td>link=TC_3_RT_148</td>
<td></td>
</tr>
<tr>
<td>verifyText</td>
<td>link=Summary(active tab)</td>
<td>Summary(active tab)</td>
</tr>
<tr>
<td>verifyText</td>
<td>link=Overview</td>
<td>Overview</td>
</tr>
<tr>
<td>verifyText</td>
<td>link=Rotten Tomatoes</td>
<td>Rotten Tomatoes</td>
</tr>
<tr>
<td>verifyText</td>
<td>link=Edit</td>
<td>Edit</td>
</tr>
<tr>
<td>verifyText</td>
<td>link=Edit Genre</td>
<td>Edit Genre</td>
</tr>
<tr>
<td>verifyText</td>
<td>link=Edit Other Metadata</td>
<td>Edit Other Metadata</td>
</tr>
<tr>
<td>verifyText</td>
<td>link=Edit Synopsis</td>
<td>Edit Synopsis</td>
</tr>
<tr>
<td>verifyText</td>
<td>link=Revisions</td>
<td>Revisions</td>
</tr>
<tr>
<td>clickAndWait</td>
<td>link=Summary(active tab)</td>
<td></td>
</tr>
<tr>
<td>clickAndWait</td>
<td>link=Overview</td>
<td></td>
</tr>
<tr>
<td>clickAndWait</td>
<td>link=Rotten Tomatoes</td>
<td></td>
</tr>
<tr>
<td>clickAndWait</td>
<td>link=Edit</td>
<td></td>
</tr>
<tr>
<td>clickAndWait</td>
<td>link=Edit Genre</td>
<td></td>
</tr>
<tr>
<td>clickAndWait</td>
<td>link=Edit Other Metadata</td>
<td></td>
</tr>
<tr>
<td>clickAndWait</td>
<td>link=Edit Synopsis</td>
<td></td>
</tr>
<tr>
<td>clickAndWait</td>
<td>link=Revisions</td>
<td></td>
</tr>
<tr>
<td>clickAndWait</td>
<td>link=Log out</td>
<td></td>
</tr>
</tbody></table>
</body>
</html>
My Question:- Is there any solution for this error?
there is a fix (that works fine for my test case), see also http://code.google.com/p/selenium/issues/detail?id=5841: http://release.seleniumhq.org/selenium-ide/2.1.0/selenium-ide-2.1.0.xpi
This error is probably due to FF version 22. I uninstalled my FF 22 and reinstalled firefox 21 and it is working fine for me.