SQL Isolation level - sql

We are using session isolation serializable in our application. The intended behavior is that when a user is going insert a new row, it should-should check for the presence of the row with the same key and update the same if row found. But I have found multiple rows created for the same key in SQL server. Is this issue with isolation or the way we are handling the case?
Following is the code I am using,
private int getNextNumber(String objectName, Connection sqlConnection) throws SQLException {
// TODO Auto-generated method stub
int number = 0;
try{
sqlConnection.setTransactionIsolation(Connection.TRANSACTION_SERIALIZABLE);
System.out.println("##### Transaction isolation set : " + sqlConnection.getTransactionIsolation());
Statement stmt = sqlConnection.createStatement();
ResultSet rs = stmt.executeQuery("select * from [dbo].[db] where DocumentNumber = '" + objectName.toString() + "' FOR UPDATE");
while(rs.next()) {
printNumber = rs.getInt("PrintNumber");
}
System.out.println("#### Print number found from sql is : " + printNumber);
if(printNumber == 0) {
printNumber = printNumber + 1;
stmt.execute("INSERT INTO [dbo].[db] (number, DocumentNumber) VALUES (1 ,'" + objectName.toString() + "')");
} else {
number = number + 1;
stmt.execute("UPDATE [dbo].[db] SET Number =" + number + " WHERE DocumentNumber ='" + objectName.toString() + "'");
}
//sqlConnection.commit();
}catch(Exception e) {
sqlConnection.rollback();
e.printStackTrace();
} finally {
sqlConnection.commit();
}
return number;
}
Thanks,
Kishor Koli

It's an issue with the way your database is set up. You need a unique constraint to enforce uniqueness. You can check at insert time all you like but a unique constraint is the only way it's going to work 100% so it's just a waste of time selecting before inserting in the hope you'll prevent a duplicate. Insert, catch the exception/error or proceed.

Related

MERGE INTO multiple statements at once Oracle SQL

I have a merge into statement as this :
private static final String UPSERT_STATEMENT = "MERGE INTO " + TABLE_NAME + " tbl1 " +
"USING (SELECT ? as KEY,? as DATA,? as LAST_MODIFIED_DATE FROM dual) tbl2 " +
"ON (tbl1.KEY= tbl2.KEY) " +
"WHEN MATCHED THEN UPDATE SET DATA = tbl2.DATA, LAST_MODIFIED_DATE = tbl2.LAST_MODIFIED_DATE " +
"WHEN NOT MATCHED THEN " +
"INSERT (DETAILS,KEY, DATA, CREATION_DATE, LAST_MODIFIED_DATE) " +
"VALUES (SEQ.NEXTVAL,tbl2.KEY, tbl2.DATA, tbl2.LAST_MODIFIED_DATE,tbl2.LAST_MODIFIED_DATE)";
This is the execution method:
public void mergeInto(final JavaRDD<Tuple2<Long, String>> rows) {
if (rows != null && !rows.isEmpty()) {
rows.foreachPartition((Iterator<Tuple2<Long, String>> iterator) -> {
JdbcTemplate jdbcTemplate = jdbcTemplateFactory.getJdbcTemplate();
LobCreator lobCreator = new DefaultLobHandler().getLobCreator();
while (iterator.hasNext()) {
Tuple2<Long, String> row = iterator.next();
String details = row._2();
Long key = row._1();
java.sql.Date lastModifiedDate = Date.valueOf(LocalDate.now());
Boolean isSuccess = jdbcTemplate.execute(UPSERT_STATEMENT, (PreparedStatementCallback<Boolean>) ps -> {
ps.setLong(1, key);
lobCreator.setBlobAsBytes(ps, 2, details.getBytes());
ps.setObject(3, lastModifiedDate);
return ps.execute();
});
System.out.println(row + "_" + isSuccess);
}
});
}
}
I need to upsert multiple of this statement inside of PLSQL, bulks of 10K if possible.
what is the efficient way to save time : execute 10K statements at once, or how to execute 10K statements in the same transaction?
how should I change the method for support it?
Thanks,
Me
the most efficient way would be one that bulk-loads your data into the database. In comparison to one-by-one uploads (as in your example), I'd expect performance gains of at least 1 or 2 orders of magnitude ("bigger" data means less to be gained by bulk-inserting).
you could use a technique as described in this answer to bulk-insert your records into a temporary table first and then perform a single merge statement using the temporary table.

Inserting data through GUI into sql server

I'm able to execute sql statements by writing the sql codes (Insert etc) on Eclipse and it is being displayed into sql server correctly. Connection has been done. But what should I do when a user wants to add data through a GUI interface (text field) and the data need to get stored into the database automatically ??
my code in the ADD button, but i'm getting the Error: java.lang.NullPointerException ! Help please..
try {
String pid = ProductID.getText();
String sql = "insert into Products_tbl values (' " +pid + " ')";
// Running the sql query
rs = st.executeQuery(sql);
int count = 0;
while (rs.next()) {
count = count + 1;
}
if (count == 1) {
JOptionPane.showMessageDialog(null, "Welcome");
}
else if (count > 1) {
JOptionPane.showMessageDialog(null,"Duplicate User Access Denied");
}
else {
JOptionPane.showMessageDialog(null, " User Not Found ");
}
}
catch (Exception ex) {
System.out.println("Error: " + ex);
}
1- Using (' " +pid + " ')" is not safe because SQL injection may occur. Use SqlParameters instead. Please check:
https://www.w3schools.com/sql/sql_injection.asp
2- I am pretty sure something is wrong with the line: rs = st.executeQuery(sql);
Here, I bet the value of st is null. Make sure that your connection variable is defined and set correctly and you created the statement like below:
st = connection.createStatement();
You can also try executeupdate(query) instead of executequery(query) like:
int flag = st.executeUpdate(query);
Ref: https://docs.oracle.com/javase/7/docs/api/java/sql/Statement.html#executeUpdate%28java.lang.String%29
3- Please use printStackTrace() method while printing the error in the catch blog, the error message would be more understandable.
System.out.println("Error: " + ex.printStackTrace());

H2 - How to truncate all tables?

I assume there is a way of doing this from code, at least some good workaround.
Please consider that I do not want to delete all tables (I've seen this command), just to delete rows from them but to keep existing schema and all constraints.
Maybe I can somehow obtain the list of all tables from metadata and apply TRUNCATE command for each separately? But what about their relations and foreign keys?
Any Idea?
You may do it this way:
Disable referential integrity using SET REFERENTIAL_INTEGRITY FALSE
Get the list of all tables using SHOW TABLES
Delete the data from each table using TRUNCATE TABLE tableName
Enable referential integrity using SET REFERENTIAL_INTEGRITY TRUE
For now, I came up with this solution... But still need to test it more thoroughly.
private void truncateDatabase () throws SQLException {
String tempDir = System.getProperty("java.io.tmpdir");
File tempRestoreFile = new File(tempDir + File.separator + "tempRestore");
Connection connection = dataSource.getConnection();
Statement statement = connection.createStatement();
statement.execute("SCRIPT SIMPLE NODATA DROP TO '" + tempRestoreFile + "' CHARSET 'UTF-8'");
statement.execute("RUNSCRIPT FROM '" + tempRestoreFile.getAbsolutePath() + "' CHARSET 'UTF-8'");
}
Here is the working Java code to truncate all tables:
public void truncate() throws SQLException {
try (Connection connection = dataSource.getConnection();
PreparedStatement setChecks = connection.prepareStatement("SET FOREIGN_KEY_CHECKS = ?");
PreparedStatement getTables = connection.prepareStatement("SELECT table_name FROM information_schema.tables WHERE table_schema = SCHEMA()")) {
try (ResultSet tablesRes = getTables.executeQuery()) {
setChecks.setBoolean(1, false);
setChecks.executeUpdate();
while (tablesRes.next()) {
String table = tablesRes.getString(1);
try (PreparedStatement truncateTable = connection.prepareStatement("TRUNCATE TABLE " + table + " RESTART IDENTITY")) {
truncateTable.executeUpdate();
}
}
} finally {
setChecks.setBoolean(1, true);
setChecks.executeUpdate();
}
}
}
Here is an example of stored procedure truncate_all_tables that disables foreign keys, then truncates all tables in current schema and then enables foreign keys back:
DROP ALIAS IF EXISTS truncate_all_tables;
CREATE ALIAS truncate_all_tables AS $$
void truncateAllTables(Connection conn) throws SQLException {
conn.createStatement().executeUpdate("SET FOREIGN_KEY_CHECKS=0");
ResultSet rs = conn.createStatement().
executeQuery("SELECT table_name FROM information_schema.tables WHERE table_schema = SCHEMA()");
while (rs.next()) {
String tableName = rs.getString(1);
conn.createStatement().executeUpdate("TRUNCATE TABLE \"" + tableName + "\" RESTART IDENTITY");
}
conn.createStatement().executeUpdate("SET FOREIGN_KEY_CHECKS=1");
}
$$;
CALL truncate_all_tables();
Or you can define the function in your code:
public class H2Functions {
public static void truncateAllTables(Connection conn) throws SQLException {
conn.createStatement().executeUpdate("SET FOREIGN_KEY_CHECKS=0");
ResultSet rs = conn.createStatement().
executeQuery("SELECT table_name FROM information_schema.tables WHERE table_schema = SCHEMA()");
while (rs.next()) {
String tableName = rs.getString(1);
conn.createStatement().executeUpdate("TRUNCATE TABLE \"" + tableName + "\" RESTART IDENTITY");
}
conn.createStatement().executeUpdate("SET FOREIGN_KEY_CHECKS=1");
}
}
and then use as alias:
SET MODE REGULAR;
CREATE ALIAS IF NOT EXISTS truncate_all_tables FOR "com.yourcompany.H2Functions.truncateAllTables";
CALL truncate_all_tables();
SET MODE MySQL;
Here I added SET MODE statements as an example if you are using H2 in MySQL mode you have to switch back to H2 mode, then declare the function and then switch back to MySQL mode.
Unfortunately the truncate_all_tables doesn't resets auto_inc columns. See Spring test with H2 in memory database truncate all tables for details.

How to solve Data truncated?

I insert the data from CSV file into MySql database, especially into one table.
I use CSVRead, and the CSV file format is :
ts,val
2013-03-31T23:45:00-04:00 New_York,10
And the table is hisdata(ts, val).
Here is my code:
try{
try {
CSVReader reader = new CSVReader(new FileReader(csvFile));
List<String[]> csvList;
csvList = reader.readAll();
System.out.println("Start: size is " + csvList.size());
for(int i = 0; i<csvList.size(); i++){
String[] eachStr = csvList.get(i);
int j = 0;
//insert(ts, val) into hisdata of sql
String sql = "INSERT INTO hisdata" + "(ts, val)" + " VALUES"
+ "('" + eachStr[j] + "', '" + eachStr[j+1] + "')";
Statement st = (Statement) conn.createStatement();
count = st.executeUpdate(sql);
}
System.out.println("access table is inserted: " + count
+ " records");
reader.close();
} catch(IOException e){
System.out.println("Error: " + e.getMessage());
}
conn.close();
} catch (SQLException e) {
System.out.println("insert is failure " + e.getMessage());
}
I think probably, the import data is too large. When I did size(), size is 8835.
Basically, I set connector. Then read CSV file and insert data line by line. Finally, I closed reader and connection.
Here is the Console print out:
Sql Connection starts
Driver loaded
Database connected
Start: size is 8835
insert is failure Data truncated for column 'val' at row 1
Is the problem the data is too large.
Please give help to solve this problem.
Add System.out.println(sql); execute the result in mysql.
if data is too large, increase column length, or get column length then reduce your data with substring.

Column is not indexed even though it is. PreparedStatement inside

I'm really struggling with a bug that did not appear on my dev environment, only once deployed in test.
I'm using a prepared Statement to run around 30 000 query in a row. The query check for the similarity of a string with what's in our database, using the oracle fuzzy method.
The column checked is indexed, but, don't know why, it fails randomly after some iterations, saying that index does not exists.
I don't understand what's going on, as the index really exists. My method never rebuild or delete the index so there is no reason for this error to appear ...
public List<EntryToCheck> checkEntriesOnSuspiciousElement(List<EntryToCheck> entries, int type,int score, int numresults, int percentage) throws Exception {
Connection connection = null;
PreparedStatement statementFirstName = null;
PreparedStatement statementLastname = null;
int finalScore = checkScore(score);
int finalNumResults = checkNumResults(numresults);
int finalPercentage = checkPercentage(percentage);
try {
connection = dataSource.getConnection();
StringBuilder requestLastNameOnly = new StringBuilder("SELECT SE.ELEMENT_ID, SE.LASTNAME||' '||SE.FIRSTNAME AS ELEMENT, SCORE(1) AS SCORE ");
requestLastNameOnly.append("FROM BL_SUSPICIOUS_ELEMENT SE ");
requestLastNameOnly.append("WHERE CONTAINS(SE.LASTNAME, 'fuzzy({' || ? || '},' || ? || ',' || ? || ', weight)', 1)>? ");
requestLastNameOnly.append((type > 0 ? "AND SE.ELEMENT_TYPE_ID = ? " : " "));
requestLastNameOnly.append("ORDER BY SCORE DESC");
statementLastname = connection.prepareStatement(requestLastNameOnly.toString());
for (EntryToCheck entryToCheck : entries) {
ResultSet rs;
boolean withFirstName = (entryToCheck.getEntryFirstname() != null && !entryToCheck.getEntryFirstname().equals(""));
statementLastname.setString(1, entryToCheck.getEntryLastname().replaceAll("'","''"));
statementLastname.setInt(2, finalScore);
statementLastname.setInt(3, finalNumResults);
statementLastname.setInt(4, finalPercentage);
if(type > 0){
statementLastname.setInt(5, type);
}
System.out.println("Query LastName : " + entryToCheck.getEntryLastname().replaceAll("'","''") );
rs = statementLastname.executeQuery();
while (rs.next()) {
Alert alert = new Alert();
alert.setEntryToCheck(entryToCheck);
alert.setAlertStatus(new AlertStatus(new Integer(AlertStatusId.NEW)));
alert.setAlertDate(new Date());
alert.setBlSuspiciousElement(new BlSuspiciousElement(new Integer(rs.getInt("ELEMENT_ID"))));
alert.setMatching(rs.getString("ELEMENT") + " (" + rs.getInt("SCORE") + "%)");
entryToCheck.addAlert(alert);
}
}
}
catch (Exception e) {
e.printStackTrace();
throw e;
}
finally {
DAOUtils.closeConnection(connection, statementLastname);
}
return entries;
}
Really don't know what to look at ...
Thanks !
F
I never used Oracle text tables but my advice is:
Make sure that no one else is executing DDL statements on the table simultaneously.
Also, make sure that, index you have is context index.
Create an index for your column where you want to apply search
........................................
CREATE INDEX "MTU219"."SEARCHFILTER" ON "BL_SUSPICIOUS_ELEMENT " ("LASTNAME")
INDEXTYPE IS "CTXSYS"."CONTEXT" PARAMETERS ('storage CTXSYS.ST_MTED_NORMAL SYNC(ON COMMIT)');
..........................................