DataIntegrityViolationException from concurrent access - sql

I have a table that has a unique index constraint on two columns. My bit of code that does the check and insert into the database is as below:
def createNewEntry(myTableType: MyTableType) = database withSession { implicit session: Session =>
val autoInc = myTableElems returning myTableElems.map(_.elemId)
getElemForId(myTableType.elemId) match {
case Some(oldTableType) =>
if (oldTableType.isModified(myTableType))
myTableElems.insert(myTableType.copy(version = oldTableType.version + 1))
case None =>
autoInc.insert(myTableType)
}
getElemForId(myTableType.elemId)
}
Is there any possibility that the above code could result in a DataIntegrityViolation exception because of concurrent access tp the above method? I have a unique index defined on a column called name and another column called version. Every time I call this createNewEntry method, I check if one exists if yes, I copy everything, increment the version and insert the new version in the database.
Since I'm using Slick for database access, the above method works within a database transaction context and in auto commit mode.
My table with entries in the database would look like:
id name version
1 abc 1
2 xyz 1
3 abc 2
name and version has unique index constraint enforced!
Now the question is could there be a scenario where I could have a DataIntegrityViolation exception? Should I guard against it?

Related

Postgres upsert using on conflict - multiple fields in on conflict condition

I'm trying to implement the concept of liking / disliking an item in a postgres db - when the user likes / dislikes something, I want to insert an item into my DB to represent this.
This is my schema:
id | postID | userID | type
1 2 1 like
Now if the user has already liked the item, and now they decide to dislike it - I want to update the type field, from like to dislike.
Similarly, if they've disliked something and now decided to like it, I want to perform the opposite update.
Furthermore, a user can only like / dislike something once - so if the user has liked / disliked the post previously and now decides to like / dislike it again, nothing should happen.
This means I need to implement an upsert statement in postgres, which inserts a new row, if the user has not interacted with the post previously, and updates the type field, if a row with the specified postID + userID + type already exists.
I was looking at doing this using the on conflict syntax -
INSERT INTO table_name(postID,userID,type)
VALUES(2,1,'like')
ON CONFLICT (????) DO UPDATE
SET type = 'like'
but I'm not sure what to pass into the ON CONFLICT section, since the match needs to happen on multiple fields.
I considered setting a unique index on the (postID, userID) fields - something like this:
create unique index idx_1 on table (postID, userID)
The problem is I want to use this DB in the future to store comment information, and a user is allowed to comment on the same post multiple times.
An example would be:
id | postID | userID | type
1 2 1 comment
2 2 1 comment
3 2 1 like
If you want to limit the number of updates to a row, you can use a check constraint and secondary column counting the updates.
alter table t add column num_updates int default 1 check (num_updates <= 2);
Then, if you want to prevent duplicate rows on types other than comment, you can use a filtered unique index:
create unique index unq_table_name_postid_userid
on table_name(postid, userid)
where type <> 'comment';
Then I think you can express the logic using on conflict:
INSERT INTO table_name (postID, userID, type)
VALUES(2, 1, 'like')
ON CONFLICT (postID, userID) DO UPDATE
SET type = 'like',
num_updates = excluded.num_updates + 1;
This allows only one update. You might want more refined logic, such as updating the value only if type changes:
num_update = (excluded.num_updates = num_updates)::int + 1

Select .. for Update in cluster environment

I am working on some legacy product and they are using a table to generate unique keys (primary key) for each table. This table contains latest ID for all other tables. When they want to insert a row in any other table, following is the logic they use to generate unique ID for that new row
Table to generate keys look likes
ID | NEXT_ID | TABLE_NAME
public synchronized long generateKey(Connection con){
// select the latest ID value from the table against a row
// increment the value by 1
// update the table with this latest value
// return the latest value
}
In single node threaded environment everything goes fine. But there are changes to get race condition while executing the above logic in clustered environment. So in order to over come this issue we thought of having a java function that calls a PL/SQL function that do the above job. Code is as follows
public long generateKey(Connection con) {
// call PL/SQL function and return the value
}
Following is the skeleton of PL/SQL function
FUNCTION GET_NEXT_ID(tablename IN VARCHAR2)
RETURN NUMBER IS
PRAGMA AUTONOMOUS_TRANSACTION;
nextID NUMBER;
BEGIN
SELECT NEXT_ID INTO nextID FROM <Key_generator_table> WHERE TABLE_NAME=tablename FOR UPDATE;
UPDATE <Key_generator_table> SET NEXT_ID=NEXT_ID+1 WHERE TABLE_NAME=tablename;
commit;
RETURN (nextID);
END;
What I understand wrt SELECT FOR UPDATE is, it locks the row when we retrieve so that no other transaction can see while it is trying to update the record. So it hold good in non clustered environment. My questions is would this same hold good in clustered environment? Would there be any race conditions with this approach?
Unfortunately we could not change the unique ID generation approach due to product constraints.
This will serialize your transactions. Not a good design. How about having a sequence and getting the value from that in your PL/SQL function?

Forcing data entry in order

I have a sql server database with ms access frontend.
I want to force data to be entered in a particular table in order.
eg I can only enter id = 2 after id = 1 has been added and I can only add id = 3 after 2 and 1 have been added.
What is the best way to enforce this? Do I need a restriction added to the access frontend or do I need to add a trigger or similar to the sql table?
i agree with #Vamsi Pamula, You can do that. But, however if you want to do only what your are looking for then :
First of all, When user tries to enter an new record, track the New ID Given Suppose 10 for now. And, first Query in the database for the Max id before like :
Select isnull(Max(id),0) from YourTable
Suppose the above query returned 8. So, 10 should not be allowed. Now now check :
if (ReturnedValuefromAboveQry + 1 = NEWID) then
msgbox "Allowed."
else
msgbox "Not Allowed."
End If
If you want to do so, keep that id as primary key and set it property as identity (auto increment). You no need to insert that value. Insert the remaining columns and that id will be automatically saved as 1,2 and so on

How do you modify existing records using ScalaQuery?

By modify I mean counterparts of SQL UPDATE and DELETE.
In both cases I have an object-record and I would like to delete it in the database. The table has always primary key, and it is set in my object-record.
Please note that I don't have query or other source which "created" that object-record, all I have is it and the table. So in general it looks like this:
fetch the Record from Table
...
// forget how I get the Record
...
Record.person_name = "joe"
? update Record ?
How to do it?
I define records and tables as below:
case class Topic(var id : Long,
var sectionId : Int,
...
object TopicTable extends Table[Topic]("Topic") {
def id = column[Long]("top_Id", O.PrimaryKey)
def sectionId = column[Int]("sect_Id")
...
It seems there are no direct methods, so you have to create explicitly a recordset in order to modify (for comparison -- I know SQ is not ORM -- in EF you fetch records, modify them and at this point your data context "knows" they were modify, so all you have to do is submit changes).
So first you create RS as you like:
val rs = for (rec <- MyTable if rec.id===10) yield rec;
and the delete records:
rs.mutate(rec => rec.delete())
for update:
rs.update(new MyRecord(...))
or (gossip is, it is faster ;-) )
rs.mutate(rec => rec.row = new MyRecord(...))
Please note I am complete newbie with SQ so I might just misinformed you. I works for me though.
Now, the only missing part is adding some nice wrappers, so delete and update could be done directly per record.

Auto Increment after delete in MySQL

I have a MySQL table with a primary key field that has AUTO_INCREMENT on.
After reading other posts on here I've noticed people with the same problem and with varied answers. Some recommend not using this feature, others state it can't be 'fixed'.
I have:
table: course
fields: courseID, courseName
Example: number of records in the table: 18. If I delete records 16, 17 and 18 - I would expect the next record entered to have the courseID of 16, however it will be 19 because the last entered courseID was 18.
My SQL knowledge isn't amazing but is there anyway to refresh or update this count with a query (or a setting in the phpMyAdmin interface)?
This table will relate to others in a database.
Given all the advice, I have decided to ignore this 'problem'. I will simply delete and add records whilst letting the auto increment do it's job. I guess it doesn't really matter what the number is since it's only being used as a unique identifier and doesn't have a (as mentioned above) business meaning.
For those who I may have confused with my original post: I do not wish to use this field to know how many records I have. I just wanted the database to look neat and have a bit more consistency.
What you're trying to do sounds dangerous, as that's not the intended use of AUTO_INCREMENT.
If you really want to find the lowest unused key value, don't use AUTO_INCREMENT at all, and manage your keys manually. However, this is NOT a recommended practice.
Take a step back and ask "why you need to recycle key values?" Do unsigned INT (or BIGINT) not provide a large enough key space?
Are you really going to have more than 18,446,744,073,709,551,615 unique records over the course of your application's lifetime?
ALTER TABLE foo AUTO_INCREMENT=1
If you've deleted the most recent entries, that should set it to use the next lowest available one. As in, as long as there's no 19 already, deleting 16-18 will reset the autoincrement to use 16.
EDIT: I missed the bit about phpmyadmin. You can set it there, too. Go to the table screen, and click the operations tab. There's an AUTOINCREMENT field there that you can set to whatever you need manually.
Primary autoincrement keys in database are used to uniquely identify a given row and shouldn't be given any business meaning. So leave the primary key as is and add another column called for example courseOrder. Then when you delete a record from the database you may want to send an additional UPDATE statement in order to decrement the courseOrder column of all rows that have courseOrder greater than the one you are currently deleting.
As a side note you should never modify the value of a primary key in a relational database because there could be other tables that reference it as a foreign key and modifying it might violate referential constraints.
Try :
SET #num := 0;
UPDATE your_table SET id = #num := (#num+1);
ALTER TABLE `your_table` AUTO_INCREMENT = 1;
That'll reset the autoincremented value, and then count every row while a new value is created for it.
example : before
1 : first value here
2 : second value here
X : deleted value
4 : The rest of the table
5 : The rest of the rest..
so the table will display the array : 1,2,4,5
Example : AFTER (if you use this command you will obtain)
1 : first value here
2 : second value here
3 : The rest of the table
4 : the rest of the rest
No trace of the deleted value, and the rest of the incremented continues with this new count.
BUT
If somewhere on your code something use the autoincremented value... maybe this attribution will cause problem.
If you don't use this value in your code everything should be ok.
You shouldn't be relying on the AUTO_INCREMENT id to tell you how many records you have in the table. You should be using SELECT COUNT(*) FROM course. ID's are there to uniquely identifiy the course and can be used as references in other tables, so you shouldn't repeat ids and shouldn't be seeking to reset the auto increment field.
I came here looking for an answer to the Title question "MySQL - Auto Increment after delete" but I could only find an answer for that in the questions
How to delete certain row from mysql table?
How to reset AUTO_INCREMENT in MySQL?
By using something like:
DELETE FROM table;
ALTER TABLE table AUTO_INCREMENT = 1;
Note that Darin Dimitrov's answer explain really well AUTO_INCREMENT and it's usage. Take a look there before doing something you might regret.
PS: The question itself is more "Why you need to recycle key values?" and Dolph's answer cover that.
What you are trying to do is very dangerous. Think about this carefully. There is a very good reason for the default behaviour of auto increment.
Consider this:
A record is deleted in one table that has a relationship with another table. The corresponding record in the second table cannot be deleted for auditing reasons. This record becomes orphaned from the first table. If a new record is inserted into the first table, and a sequential primary key is used, this record is now linked to the orphan. Obviously, this is bad. By using an auto incremented PK, an id that has never been used before is always guaranteed. This means that orphans remain orphans, which is correct.
There is actually a way to fix that. First you delete the auto_incremented primary key column, and then you add it again, like this:
ALTER TABLE table_name DROP column_name;
ALTER TABLE table_name ADD column_name int not null auto_increment primary key first;
you can select the ids like so:
set #rank = 0;
select id, #rank:=#rank+1 from tbl order by id
the result is a list of ids, and their positions in the sequence.
you can also reset the ids like so:
set #rank = 0;
update tbl a join (select id, #rank:=#rank+1 as rank from tbl order by id) b
on a.id = b.id set a.id = b.rank;
you could also just print out the first unused id like so:
select min(id) as next_id from ((select a.id from (select 1 as id) a
left join tbl b on a.id = b.id where b.id is null) union
(select min(a.id) + 1 as id from tbl a left join tbl b on a.id+1 = b.id
where b.id is null)) c;
after each insert, you can reset the auto_increment:
alter table tbl auto_increment = 16
or explicitly set the id value when doing the insert:
insert into tbl values (16, 'something');
typically this isn't necessary, you have count(*) and the ability to create a ranking number in your result sets. a typical ranking might be:
set #rank = 0;
select a.name, a.amount, b.rank from cust a,
(select amount, #rank:=#rank+1 as rank from cust order by amount desc) b
where a.amount = b.amount
customers ranked by amount spent.
I can think of plenty of scenarios where you might need to do this, particularly during a migration or development process. For instance, I just now had to create a new table by cross-joining two existing tables (as part of a complex set-up process), and then I needed to add a primary key after the event. You can drop the existing primary key column, and then do this.
ALTER TABLE my_table ADD `ID` INT NOT NULL AUTO_INCREMENT FIRST, ADD PRIMARY KEY (`ID`);
For a live system, it is not a good idea, and especially if there are other tables with foreign keys pointing to it.
I got a very simple but tricky method.
While deleting a row, you can preserve the IDs into another temporary table. After that, when you will insert new data into the main table then you can search and pick IDs from the temporary table. So use a checking here. If the temporary table has no IDs then calculate maximum ID into the main table and set the new ID as: new_ID = old_max_ID+1.
NB: You can not use auto-increment feature here.
You may think about making a trigger after delete so you can update the value of autoincrement and the ID value of all rows that does not look like what you wanted to see.
So you can work with the same table and the auto increment will be fixed automaticaly whenever you delete a row the trigger will fix it.
You can use your mysql client software/script to specify where the primary key should start from after deleting the required records.
Its definitely not recommendable. If you have a large database with multiple tables, you may probably have saved a userid as id in table 2. if you rearrange table 1 then probably the intended userid will not end up being the intended table 2 id.
MYSQL Query
Auto Increment Solution. It works perfect when you have inserted many records during testing phase of software. Now you want to launch your application live to your client and You want to start auto increment from 1.
To avoid any unwanted problems, for safer side
First export .sql file.
Then follow the below steps:
Step 1)
First Create the copy of an existing table
MySQL Command to create Copy:
CREATE TABLE new_Table_Name SELECT * FROM existing_Table_Name;
The exact copy of a table is created with all rows except Constraints.
It doesn’t copy constraints like Auto Increment and Primary Key into new_Table_name
Step 2)
Delete All rows If Data is not inserted in testing phase and it is not useful.
If Data is important then directly go to Step 3.
DELETE from new_Table_Name;
Step 3) To Add Constraints, Goto Structure of a table
3A) Add primary key constraint from More option (If You Require).
3B) Add Auto Increment constraint from Change option. For this set Defined value as None.
3C) Delete existing_Table_Name and
3D) rename new_Table_Name to existing_Table_Name.
Now It will work perfectly. The new first record will take first value in Auto Increment column.
Here is a step to solve your problem.
On your .php file, just add this query given below:
<?php
$servername = "localhost";
$username = "root";
$password = "";
$dbname = "";
$conn = new mysqli($servername, $username, $password, $dbname);
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
//write the number or id you want to start with the next user in AUTO_INCREMENT
$sql = "ALTER TABLE `table_name` AUTO_INCREMENT = number";
$conn->query($sql);
?>
I hope your problem will be solved.
if($id == 1){ // deleting first row
mysqli_query($db,"UPDATE employees SET id=id-1 WHERE id>1");
}
else if($id>1 && $id<$num){ // deleting middle row
mysqli_query($db,"UPDATE employees SET id=id-1 WHERE id>$id");
}
else if($id == $num){ // deleting last row
mysqli_query($db,"ALTER TABLE employees AUTO_INCREMENT = $num");
}
else{
echo "ERROR";
}
mysqli_query($db,"ALTER TABLE employees AUTO_INCREMENT = $num");
here is a function that fix your problem
public static void fixID(Connection conn, String table) {
try {
Statement myStmt = conn.createStatement();
ResultSet myRs;
int i = 1, id = 1, n = 0;
boolean b;
String sql;
myRs = myStmt.executeQuery("select max(id) from " + table);
if (myRs.next()) {
n = myRs.getInt(1);
}
while (i <= n) {
b = false;
myRs = null;
while (!b) {
myRs = myStmt.executeQuery("select id from " + table + " where id=" + id);
if (!myRs.next()) {
id++;
} else {
b = true;
}
}
sql = "UPDATE " + table + " set id =" + i + " WHERE id=" + id;
myStmt.execute(sql);
i++;
id++;
}
} catch (SQLException e) {
e.printStackTrace();
}
}