Multiple SQL statements using Groovy - sql

Delete multiple entries from DB using Groovy in SoapUI
I am able to execute one SQL statement, but when I do a few it just hangs.
How can I delete multiple rows?
def sql = Sql.newInstance('jdbc:oracle:thin:#jack:1521:test1', 'test', 'test', 'oracle.jdbc.driver.OracleDriver')
log.info("SQL connetced")
sql.connection.autoCommit = false
try {
log.info("inside try")
log.info("before")
String Que =
"""delete from table name where user in (select user from user where ID= '123' and type= 262);
delete from table name where user in (select user from user where ID= '1012' and type= 28)
delete from table name where user in (select user from user where ID= '423' and type= 27)
"""
log.info (Que)
def output = sql.execute(Que);
log.info(sql)
log.info(output)
log.info("after")
sql.commit()
println("Successfully committed")
}catch(Exception ex) {
sql.rollback()
log.info("Transaction rollback"+ex)
}
sql.close()

Here is what you are looking for.
I feel it is more effective way if you want bulk number of records using the following way.
Create a map for the data i.e., id, type as key value pair that needs to be removed in your case.
Used closure to execute the query by iterating thru it.
Added comments appropriately.
//Closure to execute the query with parameters
def runQuery = { entry ->
def output = sql.execute("delete from table name where user in (select user from user where ID=:id and type=:type)", [id:entry.key, type:entry.value] )
log.info(output)
}
//Added below two statements
//Create the data that you want to remove in the form of map id, and type
def deleteData = ['123':26, '1012':28, '423':27]
def sql = Sql.newInstance('jdbc:oracle:thin:#jack:1521:test1', 'test', 'test', 'oracle.jdbc.driver.OracleDriver')
log.info("SQL connetced")
sql.connection.autoCommit = false
try {
log.info(sql)
log.info("inside try")
log.info("before")
//Added below two statements
//Call the above closure and pass key value pair in each iteration
deleteData.each { runQuery(it) }
log.info("after")
sql.commit()
println("Successfully committed")
}catch(Exception ex) {
sql.rollback()
log.info("Transaction rollback"+ex)
}
sql.close()
If you are just looking after execution of multiple queries only approach, then you may look at here and not sure if your database supports the same.

Related

ActiveJDBC , How can i query some columns i interest with in a single table

when i query a single table , i do not want all columns , i just want some column that i interest in.
For example, when i use where method to query a table, it will query all columns in a table like
public class SubjectSpecimenType extends Model {
}
SubjectSpecimenType.where("SUBJECT_ID = ? AND SITE_ID = ?", subjectId, siteId);
i don't know if there has a method named select that i can use to query some column like
SubjectSpecimenType.select("SUBJECT_NAME", "SITE_NAME").where("SUBJECT_ID = ? AND SITE_ID = ?", subjectId, siteId);
there are the source code in LazyList.java
/**
* Use to see what SQL will be sent to the database.
*
* #param showParameters true to see parameter values, false not to.
* #return SQL in a dialect for current connection which will be used if you start querying this
* list.
*/
public String toSql(boolean showParameters) {
String sql;
if(forPaginator){
sql = metaModel.getDialect().formSelect(null, null, fullQuery, orderBys, limit, offset);
}else{
sql = fullQuery != null ? fullQuery
: metaModel.getDialect().formSelect(metaModel.getTableName(), null, subQuery, orderBys, limit, offset);
}
if (showParameters) {
StringBuilder sb = new StringBuilder(sql).append(", with parameters: ");
join(sb, params, ", ");
sql = sb.toString();
}
return sql;
}
when call formSelect method, the second param columns always be null
is there a unfinish TODO ?
When operating on Models, ActiveJDBC always selects all columns, because if you load a model and it has partial attributes loaded, then you have a deficient model. The columns are specified in some edge cases, as in the RawPaginator: https://github.com/javalite/javalite/blob/e91ebdd1e4958bc0965d7ee99e6b7debc59a7b85/activejdbc/src/main/java/org/javalite/activejdbc/RawPaginator.java#L141
There is nothing to finish here, the behavior is intentional.

A remote access exception in DolphinDB,Can't find the object with name loadTable('dfs://zctestDB','trainInfoTable')

I want to remotely query the database in DolphinDB.The database is created on the server 38.124.2.173 with the following script in the server ,
tableSchema = table(100:0,`trainID`ts`tag01`tag02,`tag03,[INT,TIMESTAMP,FLOAT,FLOAT,FLOAT )
db1 = database("",VALUE,(today()-92)..(today()+60))
db2 = database("",RANGE,0..80*10+1)
db = database("dfs://zctestDB",COMPO,[db1,db2])
dfsTable = db.createPartitionedTable(tableSchema,"trainInfoTable",`ts`trainID)
My query code as below,
def testParallelQuery( connVector,trainIDs,startTime, endTime ){
cols=`trainID`ts`tag01
whereConditions=[<trainID in trainIDs>,expr(sqlCol(`ts),between,startTime:endTime)]
script=sql(sqlCol(cols),"loadTable('dfs://zctestDB','trainInfoTable')",whereConditions)
return ploop(remoteRun{,script}, connVector)
}
host="38.124.2.173"
port=30599
connVector = loop(xdb, take(host, 10), port, "admin", "123456")
testParallelQuery( connVector,1..5,2019.06.14T00:00:00.000, 2019.06.14T01:00:00.000 )
The following exception occurred after I ran it,
Error was raised when execution : Can't find the object with name loadTable('dfs://zctestDB','trainInfoTable')
How can I solve this problem?
Function sql helps one construct a sql statement dynamically. The parameter from accepts three types of data: (1) a table object, (2) an expression representing a table or a table join, (3) a variable associated with a table object.
In your particular case, please pass an expression to from as follows.
def runSQL(trainIDs, startTime, endTime){
cols = `trainID`ts`tag01
whereConditions = [<trainID in trainIDs>, expr(sqlCol(`ts), between, startTime:endTime)]
return sql(sqlCol(cols), loadTable('dfs://zctestDB','trainInfoTable'), whereConditions).eval()
}
def testParallelQuery( connVector,trainIDs,startTime, endTime ){
return ploop(remoteRun{,runSQL{trainIDs, startTime, endTime}}, connVector)
}
host="38.124.2.173"
port=30599
connVector = loop(xdb, take(host, 10), port, "admin", "123456")
testParallelQuery( connVector,1..5,2019.06.14T00:00:00.000, 2019.06.14T01:00:00.000 )

how to call a sql script or query in background for a Karate feature?

I have Karate feature with a long list of operations. I need to manually have some test data setup before running these tests which is essentially one sql query. Is there a way we can have this query run in "background" in Karate?
I'm trying to update all the values in status which aren't "ready_to_test" to "ready_to_test".
Say my query is
update my_table
set status = 'ready_to_test'
where status != 'ready_to_test';
EDIT:
I am trying to run the update query as follows
use JDBC to setup test data
* def config = {username: 'postgres', password: 'postgres', url: 'jdbc:postgresql://localhost:5432/postgres', driverClassName: 'org.postgresql.Driver'}
* def DbUtil = Java.type('com.utils.DbUtils')
* def db = new DbUtil(config)
* def correctStatus = 'ready_to_test'
* def testData = db.cleanRows('UPDATE MY_TABLE M SET M.STATUS = ' 'WHERE M.STATUS != ' + correctStatus)
Also tried
* def testData = db.cleanRows('UPDATE MY_TABLE SET STATUS = 'ready_to_test' WHERE STATUS != 'ready_to_test)
Please see if the callSingle API solves your need.
https://github.com/intuit/karate#karate-callsingle
var result = karate.callSingle('classpath:common.feature', { some: 'config' });
Also see hooks: https://github.com/intuit/karate#hooks
I also wanted to update particular row in database and I tried add the function below to DbUtils then use it feature file , It works.
public void insertRows(final String sql) {
System.out.println("Inserting data to database...");
jdbc.batchUpdate(new String[]{sql});
}
For more information you can watch this video

How do I loop thought each DB field to see if range is correct

I have this response in soapUI:
<pointsCriteria>
<calculatorLabel>Have you registered for inContact, signed up for marketing news from FNB/RMB Private Bank, updated your contact details and chosen to receive your statements</calculatorLabel>
<description>Be registered for inContact, allow us to communicate with you (i.e. update your marketing consent to 'Yes'), receive your statements via email and keep your contact information up to date</description>
<grades>
<points>0</points>
<value>No</value>
</grades>
<grades>
<points>1000</points>
<value>Yes</value>
</grades>
<label>Marketing consent given and Online Contact details updated in last 12 months</label>
<name>c21_mrktng_cnsnt_cntct_cmb_point</name>
</pointsCriteria>
There are many many many pointsCriteria and I use the below xquery to give me the DB value and Range of what that field is meant to be:
<return>
{
for $x in //pointsCriteria
return <DBRange>
<db>{data($x/name/text())}</db>
<points>{data($x//points/text())}</points>
</DBRange>
}
</return>
And i get the below response
<return><DBRange><db>c21_mrktng_cnsnt_cntct_cmb_point</db><points>0 1000</points></DBRange>
That last bit sits in a property transfer. I need SQL to bring back all rows where that DB field is not in that points range (field can only be 0 or 1000 in this case), my problem is I dont know how to loop through each DBRange/DBrange in this manner? please help
I'm not sure that I really understand your question, however I think that you want to make queries in your DB using specific table with a column name defined in your <db> field of your xml, and using as values the values defined in <points> field of the same xml.
So you can try using a groovy TestStep, first parse your Xml and get back your column name, and your points. To iterate over points if the values are separated with a blank space you can make a split(" ") to get a list and then use each() to iterate over the points on this list. Then using groovy.sql.Sql you can perform the queries in your DB.
Only one more thing, you need to put the JDBC drivers for your vendor DB in $SOAPUI_HOME/bin/ext and then restart SOAPUI in order that it can load the necessary driver classes.
So the follow code approach can achieve your goal:
import groovy.sql.Sql
import groovy.util.XmlSlurper
// soapui groovy testStep requires that first register your
// db vendor drivers, as example I use oracle drivers...
com.eviware.soapui.support.GroovyUtils.registerJdbcDriver( "oracle.jdbc.driver.OracleDriver")
// connection properties db (example for oracle data base)
def db = [
url : 'jdbc:oracle:thin:#db_host:d_bport/db_name',
username : 'yourUser',
password : '********',
driver : 'oracle.jdbc.driver.OracleDriver'
]
// create the db instance
def sql = Sql.newInstance("${db.url}", "${db.username}", "${db.password}","${db.driver}")
def result = '''<return>
<DBRange>
<db>c21_mrktng_cnsnt_cntct_cmb_point</db>
<points>0 1000</points>
</DBRange>
</return>'''
def resXml = new XmlSlurper().parseText(result)
// get the field
def field = resXml.DBRange.db.text()
// get the points
def points = resXml.DBRange.points.text()
// points are separated by blank space,
// so split to get an array with the points
def pointList = points.split(" ")
// for each point make your query
pointList.each {
def sqlResult = sql.rows "select * from your_table where ${field} = ?",[it]
log.info sqlResult
}
sql.close();
Hope this helps,
Thanks again for your help #albciff, I had to add this into a multidimensional array (I renamed field to column and result is a large return from the Xquery above)
def resXml = new XmlSlurper().parseText(result)
//get the columns and points ranges
def Column = resXml.DBRange.db*.text()
def Points = resXml.DBRange.points*.text()
//sorting it all out into a multidimensional array (index per index)
count = 0
bigList = Column.collect
{
[it, Points[count++]]
}
//iterating through the array
bigList.each
{//creating two smaller lists and making it readable for sql part later
def column = it[0]
def points = it[1]
//further splitting the points to test each
pointList = points.split(" ")
pointList.each
{//test each points range per column
def sqlResult = sql.rows "select * from my_table where ${column} <> ",[it]
log.info sqlResult
}
}
sql.close();
return;

DbUnit does not update postgresql sequences on insert

I am using DbUnit to run some test on a postgreSql database. In order to be able to run my test, I bring the database into a well known state by repopulating the database tables before each test, running a clean insert. Therefore I use the FlatXmlDataSet definition below (compare with the attached SQL schema).
However, if I run the testCreateAvatar() test case, I get an exception because of a status code mismatch, which is caused by a failed sql insert, because of an already existing primary key (id field). A look into my database shows me, that the insert of the test datasets does not update the corresponding *avatars_id_seq* and *users_id_seq* sequence tables, which are used to generate the id fields (mechanism of postgresql to generate auto-increment values).
That means, that the auto-increment value is not updated, if I define static IDs in the FlatXmlDataSet definitions. So my question is how I could change this behavior or set the auto-increment value on my own (using DbUnit).
Avatar creation test case
#Test
public void testCreateAvatar() throws Exception {
// Set up the request url.
final HttpPost request = new HttpPost(
"http://localhost:9095/rest/avatars");
// Setup the JSON blob, ...
JSONObject jsonAvatar = new JSONObject();
jsonAvatar.put("imageUrl", "images/dussel.jpg");
// ... add it to the post request ...
StringEntity input = new StringEntity(jsonAvatar.toString());
input.setContentType("application/json");
request.setEntity(input);
// ... and execute the request.
final HttpResponse response = HttpClientBuilder.create().build()
.execute(request);
// Verify the result.
assertThat(response.getStatusLine().getStatusCode(),
equalTo(HttpStatus.SC_CREATED));
// Fetch dussel duck from the database ...
Avatar dussel = getServiceObjDao().queryForFirst(
getServiceObjDao().queryBuilder().where()
.eq("image_url", "images/dussel.jpg")
.prepare());
// ... and verify that the object was created correctly.
assertThat(dussel, notNullValue());
assertThat("images/dussel.jpg", equalTo(dussel.getImageUrl()));
}
The DbUnit dataset
<?xml version='1.0' encoding='UTF-8'?>
<dataset>
<!-- Avatars -->
<avatars
id="1"
image_url="images/donald.jpg" />
<avatars
id="2"
image_url="images/daisy.jpg" />
<!-- Users -->
<users
id = "1"
name = "Donald Duck"
email = "donald.duck#entenhausen.de"
password = "quack" />
<users
id = "2"
name = "Daisy Duck"
email = "daisy.duck#entenhausen.de"
password = "flower" />
</dataset>
The users and avatars table schema
CREATE TABLE avatars (
id BIGSERIAL PRIMARY KEY,
cdate TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
mdate TIMESTAMP,
image_url VARCHAR(200),
UNIQUE (image_url)
);
CREATE TABLE users (
id BIGSERIAL PRIMARY KEY,
cdate TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
mdate TIMESTAMP,
name VARCHAR(160) NOT NULL,
email VARCHAR (355) UNIQUE NOT NULL,
password VARCHAR(30) NOT NULL,
avatar_id BIGINT,
UNIQUE (name),
CONSTRAINT user_avatar_id FOREIGN KEY (avatar_id)
REFERENCES avatars (id) MATCH SIMPLE
ON UPDATE NO ACTION ON DELETE NO ACTION
);
The function below finds all sequences in a database, extracts the name of the corresponding table from the sequence name and finally updates the current value of the sequences based on the maximum id value in the corresponding table. As there has been no better solution yet, this seems to be the way to go. Hope, this helps someone.
Simple solution based on harmic's suggestion
#Before
public void resetSequence() {
Connection conn = null;
try {
// Establish a database connection.
conn = DriverManager.getConnection(
this.props.getProperty("database.jdbc.connectionURL"),
this.props.getProperty("database.jdbc.username"),
this.props.getProperty("database.jdbc.password"));
// Select all sequence names ...
Statement seqStmt = conn.createStatement();
ResultSet rs = seqStmt.executeQuery("SELECT c.relname FROM pg_class c WHERE c.relkind = 'S';");
// ... and update the sequence to match max(id)+1.
while (rs.next()) {
String sequence = rs.getString("relname");
String table = sequence.substring(0, sequence.length()-7);
Statement updStmt = conn.createStatement();
updStmt.executeQuery("SELECT SETVAL('" + sequence + "', (SELECT MAX(id)+1 FROM '" + table + "'));");
}
} catch (SQLException e) {
e.printStackTrace();
} finally {
try {
conn.close();
} catch (SQLException e) {
}
}
}
You can set the value of a sequence using setval, for example
SELECT SETVAL('sequence_name', 1000);
Where sequence_name is the name of the sequence, visible in psql using /dt on the table, and 1000 is the value you want to set it to. You would probably want to set it to the Max value of Id in the table.
What I don't really know is how to get DbUnit to emit this SQL.