Related
Hi I'm running into the following problem on SQlite3
I have a simple table
CREATE TABLE TestTable (id INT, cnt INT);
There are some rows already in the table.
I have some data I want to be inserted into the table: {(id0, cnt0), (id1, cnt1)...}
I want to insert data into the table, on id conflict, update TestTable.cnt = TestTable.cnt + value.cnt
(values.cnt is cnt0, cnt1 ... basically my data to be inserted)
*** But the problem is, there is no primary or unique constraint on id, and I am not allowed to change it!
What I currently have :
In my program I loop through all the values
UPDATE TestTABLE SET count = count + value.cnt WHERE id = value.id;
if (sqlite3_changes() == 0)
INSERT INTO MyTable (id, cnt) values (value.id, value.cnt);
But the problem is, with a very large dataset, doing 2 queries for each data entry takes too long. I'm trying to bundle multiple entries together into one call.
Please let me know if you have questions about my description, thank you for helping!
If you are able to create temporary tables, then do the following. Although I don't show it here, I suggest wrapping all this in a transaction. This technique will likely increase efficiency even if you are also able to add a temporary unique index. (In that case you could use an UPSERT with source data in the temporary table.)
CREATE TEMP TABLE data(id INT, cnt INT);
Now insert the new data into the temporary table, whether by using the host-language data libraries or crafting an insert statement similar to
INSERT INTO data (id, cnt)
VALUES (1, 100),
(2, 200),
(5, 400),
(7, 500);
Now update all existing rows using the single UPDATE statement. SQLite does not have a convenient syntax for joining tables and/or providing a source query for an UPDATE statement. However, one can use nested statement to provide similar convenience:
UPDATE TestTable AS tt
SET cnt = cnt + ifnull((SELECT cnt FROM data WHERE data.id == tt.id), 0)
WHERE tt.id IN (SELECT id FROM data);
Note that the two nested queries are independent of each other. In fact, one could eliminate the WHERE clause altogether and get the same results for this simple case. The WHERE clause is simply to make it more efficient, only attempting to update matching id's. The other subquery in the SET clause also specifies a match on id, but alone it would still allow updates of rows that don't have a match, defaulting to a null value and being converted to 0 (by isnull() function) for a no-op. By the way, without the isnull() function, the sum would result in null and would overwrite non-null values.
Finally, insert only rows with non-existing id values:
INSERT INTO TestTable (id, cnt)
SELECT data.id, data.cnt
FROM data LEFT JOIN TestTable
ON data.id == TestTable.id
WHERE TestTable.id IS NULL;
I am running a python script that inserts a large amount of data into a Postgres database, I use a single query to perform multiple row inserts:
INSERT INTO table (col1,col2) VALUES ('v1','v2'),('v3','v4') ... etc
I was wondering what would happen if it hits a duplicate key for the insert. Will it stop the entire query and throw an exception? Or will it merely ignore the insert of that specific row and move on?
The INSERT will just insert all rows and nothing special will happen, unless you have some kind of constraint disallowing duplicate / overlapping values (PRIMARY KEY, UNIQUE, CHECK or EXCLUDE constraint) - which you did not mention in your question. But that's what you are probably worried about.
Assuming a UNIQUE or PK constraint on (col1,col2), you are dealing with a textbook UPSERT situation. Many related questions and answers to find here.
Generally, if any constraint is violated, an exception is raised which (unless trapped in subtransaction like it's possible in a procedural server-side language like plpgsql) will roll back not only the statement, but the whole transaction.
Without concurrent writes
I.e.: No other transactions will try to write to the same table at the same time.
Exclude rows that are already in the table with WHERE NOT EXISTS ... or any other applicable technique:
Select rows which are not present in other table
And don't forget to remove duplicates within the inserted set as well, which would not be excluded by the semi-anti-join WHERE NOT EXISTS ...
One technique to deal with both at once would be EXCEPT:
INSERT INTO tbl (col1, col2)
VALUES
(text 'v1', text 'v2') -- explicit type cast may be needed in 1st row
, ('v3', 'v4')
, ('v3', 'v4') -- beware of dupes in source
EXCEPT SELECT col1, col2 FROM tbl;
EXCEPT without the key word ALL folds duplicate rows in the source. If you know there are no dupes, or you don't want to fold duplicates silently, use EXCEPT ALL (or one of the other techniques). See:
Using EXCEPT clause in PostgreSQL
Generally, if the target table is big, WHERE NOT EXISTS in combination with DISTINCT on the source will probably be faster:
INSERT INTO tbl (col1, col2)
SELECT *
FROM (
SELECT DISTINCT *
FROM (
VALUES
(text 'v1', text'v2')
, ('v3', 'v4')
, ('v3', 'v4') -- dupes in source
) t(c1, c2)
) t
WHERE NOT EXISTS (
SELECT FROM tbl
WHERE col1 = t.c1 AND col2 = t.c2
);
If there can be many dupes, it pays to fold them in the source first. Else use one subquery less.
Related:
Select rows which are not present in other table
With concurrent writes
Use the Postgres UPSERT implementation INSERT ... ON CONFLICT ... in Postgres 9.5 or later:
INSERT INTO tbl (col1,col2)
SELECT DISTINCT * -- still can't insert the same row more than once
FROM (
VALUES
(text 'v1', text 'v2')
, ('v3','v4')
, ('v3','v4') -- you still need to fold dupes in source!
) t(c1, c2)
ON CONFLICT DO NOTHING; -- ignores rows with *any* conflict!
Further reading:
How to use RETURNING with ON CONFLICT in PostgreSQL?
How do I insert a row which contains a foreign key?
Documentation:
The manual
The commit page
The Postgres Wiki page
Craig's reference answer for UPSERT problems:
How to UPSERT (MERGE, INSERT ... ON DUPLICATE UPDATE) in PostgreSQL?
Will it stop the entire query and throw an exception? Yes.
To avoid that, you can look on the following SO question here, which describes how to avoid Postgres from throwing an error for multiple inserts when some of the inserted keys already exist on the DB.
You should basically do this:
INSERT INTO DBtable
(id, field1)
SELECT 1, 'value'
WHERE
NOT EXISTS (
SELECT id FROM DBtable WHERE id = 1
);
We have a status table. When the status changes we currently delete the old record and insert a new.
We are wondering if it would be faster to do a select to check if it exists followed by an insert or update.
Although similar to the following question, it is not the same, since we are changing individual records and the other question was doing a total table refresh.
DELETE, INSERT vs UPDATE || INSERT
Since you're talking SQL Server 2008, have you considered MERGE? It's a single statement that allows you to do an update or insert:
create table T1 (
ID int not null,
Val1 varchar(10) not null
)
go
insert into T1 (ID,Val1)
select 1,'abc'
go
merge into T1
using (select 1 as ID,'def' as Val1) upd on T1.ID = upd.ID --<-- These identify the row you want to update/insert and the new value you want to set. They could be #parameters
when matched then update set Val1 = upd.Val1
when not matched then insert (ID,Val1) values (upd.ID,upd.Val1);
What about INSERT ... ON DUPLICATE KEY? First doing a select to check if a record exists and checking in your program the result of that creates a race condition. That might not be important in your case if there is only a single instance of the program however.
INSERT INTO users (username, email) VALUES ('Jo', 'jo#email.com')
ON DUPLICATE KEY UPDATE email = 'jo#email.com'
You can use ##ROWCOUNT and perform UPDATE. If it was 0 rows affected - then perform INSERT after, nothing otherwise.
Your suggestion would mean always two instructions for each status change. The usual way is to do an UPDATE and then check if the operation changed any rows (Most databases have a variable like ROWCOUNT which should be greater than 0 if something changed). If it didn't, do an INSERT.
Search for UPSERT for find patterns for your specific DBMS
Personally, I think the UPDATE method is the best. Instead of doing a SELECT first to check if a record already exists, you can first attempt an UPDATE but if no rows are affected (using ##ROWCOUNT) you can do an INSERT.
The reason for this is that sooner or later you might want to track status changes, and the best way to do this would be to keep an audit trail of all changes using a trigger on the status table.
http://en.wikipedia.org/wiki/Upsert
Insert Update stored proc on SQL Server
Is there some clever way to do this in SQLite that I have not thought of?
Basically I want to update three out of four columns if the record exists,
If it does not exists I want to INSERT the record with the default (NUL) value for the fourth column.
The ID is a primary key so there will only ever be one record to UPSERT.
(I am trying to avoid the overhead of SELECT in order to determine if I need to UPDATE or INSERT obviously)
Suggestions?
I cannot confirm that Syntax on the SQLite site for TABLE CREATE.
I have not built a demo to test it, but it doesn't seem to be supported.
If it was, I have three columns so it would actually look like:
CREATE TABLE table1(
id INTEGER PRIMARY KEY ON CONFLICT REPLACE,
Blob1 BLOB ON CONFLICT REPLACE,
Blob2 BLOB ON CONFLICT REPLACE,
Blob3 BLOB
);
but the first two blobs will not cause a conflict, only the ID would
So I assume Blob1 and Blob2 would not be replaced (as desired)
UPDATEs in SQLite when binding data are a complete transaction, meaning
Each sent row to be updated requires: Prepare/Bind/Step/Finalize statements
unlike the INSERT which allows the use of the reset function
The life of a statement object goes something like this:
Create the object using sqlite3_prepare_v2()
Bind values to host parameters using sqlite3_bind_ interfaces.
Run the SQL by calling sqlite3_step()
Reset the statement using sqlite3_reset() then go back to step 2 and repeat.
Destroy the statement object using sqlite3_finalize().
UPDATE I am guessing is slow compared to INSERT, but how does it compare to SELECT using the Primary key?
Perhaps I should use the select to read the 4th column (Blob3) and then use REPLACE to write a new record blending the original 4th Column with the new data for the first 3 columns?
Assuming three columns in the table: ID, NAME, ROLE
BAD: This will insert or replace all columns with new values for ID=1:
INSERT OR REPLACE INTO Employee (id, name, role)
VALUES (1, 'John Foo', 'CEO');
BAD: This will insert or replace 2 of the columns... the NAME column will be set to NULL or the default value:
INSERT OR REPLACE INTO Employee (id, role)
VALUES (1, 'code monkey');
GOOD: Use SQLite On conflict clause
UPSERT support in SQLite! UPSERT syntax was added to SQLite with version 3.24.0!
UPSERT is a special syntax addition to INSERT that causes the INSERT to behave as an UPDATE or a no-op if the INSERT would violate a uniqueness constraint. UPSERT is not standard SQL. UPSERT in SQLite follows the syntax established by PostgreSQL.
GOOD but tedious: This will update 2 of the columns.
When ID=1 exists, the NAME will be unaffected.
When ID=1 does not exist, the name will be the default (NULL).
INSERT OR REPLACE INTO Employee (id, role, name)
VALUES ( 1,
'code monkey',
(SELECT name FROM Employee WHERE id = 1)
);
This will update 2 of the columns.
When ID=1 exists, the ROLE will be unaffected.
When ID=1 does not exist, the role will be set to 'Benchwarmer' instead of the default value.
INSERT OR REPLACE INTO Employee (id, name, role)
VALUES ( 1,
'Susan Bar',
COALESCE((SELECT role FROM Employee WHERE id = 1), 'Benchwarmer')
);
INSERT OR REPLACE is NOT equivalent to "UPSERT".
Say I have the table Employee with the fields id, name, and role:
INSERT OR REPLACE INTO Employee ("id", "name", "role") VALUES (1, "John Foo", "CEO")
INSERT OR REPLACE INTO Employee ("id", "role") VALUES (1, "code monkey")
Boom, you've lost the name of the employee number 1. SQLite has replaced it with a default value.
The expected output of an UPSERT would be to change the role and to keep the name.
Eric B’s answer is OK if you want to preserve just one or maybe two columns from the existing row. If you want to preserve a lot of columns, it gets too cumbersome fast.
Here’s an approach that will scale well to any amount of columns on either side. To illustrate it I will assume the following schema:
CREATE TABLE page (
id INTEGER PRIMARY KEY,
name TEXT UNIQUE,
title TEXT,
content TEXT,
author INTEGER NOT NULL REFERENCES user (id),
ts TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
Note in particular that name is the natural key of the row – id is used only for foreign keys, so the point is for SQLite to pick the ID value itself when inserting a new row. But when updating an existing row based on its name, I want it to continue to have the old ID value (obviously!).
I achieve a true UPSERT with the following construct:
WITH new (name, title, author) AS ( VALUES('about', 'About this site', 42) )
INSERT OR REPLACE INTO page (id, name, title, content, author)
SELECT old.id, new.name, new.title, old.content, new.author
FROM new LEFT JOIN page AS old ON new.name = old.name;
The exact form of this query can vary a bit. The key is the use of INSERT SELECT with a left outer join, to join an existing row to the new values.
Here, if a row did not previously exist, old.id will be NULL and SQLite will then assign an ID automatically, but if there already was such a row, old.id will have an actual value and this will be reused. Which is exactly what I wanted.
In fact this is very flexible. Note how the ts column is completely missing on all sides – because it has a DEFAULT value, SQLite will just do the right thing in any case, so I don’t have to take care of it myself.
You can also include a column on both the new and old sides and then use e.g. COALESCE(new.content, old.content) in the outer SELECT to say “insert the new content if there was any, otherwise keep the old content” – e.g. if you are using a fixed query and are binding the new values with placeholders.
This answer has been updated and so the comments below no longer apply.
2018-05-18 STOP PRESS.
UPSERT support in SQLite! UPSERT syntax was added to SQLite with version 3.24.0 (pending) !
UPSERT is a special syntax addition to INSERT that causes the INSERT to behave as an UPDATE or a no-op if the INSERT would violate a uniqueness constraint. UPSERT is not standard SQL. UPSERT in SQLite follows the syntax established by PostgreSQL.
alternatively:
Another completely different way of doing this: in my application I set my in memory rowID to be long.MaxValue when I create the row in memory. (MaxValue will never be used as an ID you won't live long enough....) Then if rowID is not that value then it must already be in the database so needs an UPDATE if it is MaxValue then it needs an insert. This is only useful if you can track the rowIDs in your app.
If you are generally doing updates I would ..
Begin a transaction
Do the update
Check the rowcount
If it is 0 do the insert
Commit
If you are generally doing inserts I would
Begin a transaction
Try an insert
Check for primary key violation error
if we got an error do the update
Commit
This way you avoid the select and you are transactionally sound on Sqlite.
I realize this is an old thread but I've been working in sqlite3 as of late and came up with this method which better suited my needs of dynamically generating parameterized queries:
insert or ignore into <table>(<primaryKey>, <column1>, <column2>, ...) values(<primaryKeyValue>, <value1>, <value2>, ...);
update <table> set <column1>=<value1>, <column2>=<value2>, ... where changes()=0 and <primaryKey>=<primaryKeyValue>;
It's still 2 queries with a where clause on the update but seems to do the trick. I also have this vision in my head that sqlite can optimize away the update statement entirely if the call to changes() is greater than zero. Whether or not it actually does that is beyond my knowledge, but a man can dream can't he? ;)
For bonus points you can append this line which returns you the id of the row whether it be a newly inserted row or an existing row.
select case changes() WHEN 0 THEN last_insert_rowid() else <primaryKeyValue> end;
Beginning with version 3.24.0 UPSERT is supported by SQLite.
From the documentation:
UPSERT is a special syntax addition to INSERT that causes the INSERT to behave as an UPDATE or a no-op if the INSERT would violate a uniqueness constraint. UPSERT is not standard SQL. UPSERT in SQLite follows the syntax established by PostgreSQL. UPSERT syntax was added to SQLite with version 3.24.0 (pending).
An UPSERT is an ordinary INSERT statement that is followed by the special ON CONFLICT clause
Image source: https://www.sqlite.org/images/syntax/upsert-clause.gif
Example:
CREATE TABLE t1(id INT PRIMARY KEY, c TEXT);
INSERT INTO t1(id, c) VALUES (1,'a'), (2, 'b');
SELECT * FROM t1;
INSERT INTO t1(id, c) VALUES (1, 'c');
-- UNIQUE constraint failed: t1.id
INSERT INTO t1(id, c) VALUES (1, 'c')
ON CONFLICT DO NOTHING;
SELECT * FROM t1;
INSERT INTO t1(id, c)
VALUES (1, 'c')
ON CONFLICT(id) DO UPDATE SET c = excluded.c;
SELECT * FROM t1;
db<>fiddle demo
Here is a solution that really is an UPSERT (UPDATE or INSERT) instead of an INSERT OR REPLACE (which works differently in many situations).
It works like this:
1. Try to update if a record with the same Id exists.
2. If the update did not change any rows (NOT EXISTS(SELECT changes() AS change FROM Contact WHERE change <> 0)), then insert the record.
So either an existing record was updated or an insert will be performed.
The important detail is to use the changes() SQL function to check if the update statement hit any existing records and only perform the insert statement if it did not hit any record.
One thing to mention is that the changes() function does not return changes performed by lower-level triggers (see http://sqlite.org/lang_corefunc.html#changes), so be sure to take that into account.
Here is the SQL...
Test update:
--Create sample table and records (and drop the table if it already exists)
DROP TABLE IF EXISTS Contact;
CREATE TABLE [Contact] (
[Id] INTEGER PRIMARY KEY,
[Name] TEXT
);
INSERT INTO Contact (Id, Name) VALUES (1, 'Mike');
INSERT INTO Contact (Id, Name) VALUES (2, 'John');
-- Try to update an existing record
UPDATE Contact
SET Name = 'Bob'
WHERE Id = 2;
-- If no record was changed by the update (meaning no record with the same Id existed), insert the record
INSERT INTO Contact (Id, Name)
SELECT 2, 'Bob'
WHERE NOT EXISTS(SELECT changes() AS change FROM Contact WHERE change <> 0);
--See the result
SELECT * FROM Contact;
Test insert:
--Create sample table and records (and drop the table if it already exists)
DROP TABLE IF EXISTS Contact;
CREATE TABLE [Contact] (
[Id] INTEGER PRIMARY KEY,
[Name] TEXT
);
INSERT INTO Contact (Id, Name) VALUES (1, 'Mike');
INSERT INTO Contact (Id, Name) VALUES (2, 'John');
-- Try to update an existing record
UPDATE Contact
SET Name = 'Bob'
WHERE Id = 3;
-- If no record was changed by the update (meaning no record with the same Id existed), insert the record
INSERT INTO Contact (Id, Name)
SELECT 3, 'Bob'
WHERE NOT EXISTS(SELECT changes() AS change FROM Contact WHERE change <> 0);
--See the result
SELECT * FROM Contact;
Updates from Bernhardt:
You can indeed do an upsert in SQLite, it just looks a little different than you are used to. It would look something like:
INSERT INTO table_name (id, column1, column2)
VALUES ("youruuid", "value12", "value2")
ON CONFLICT(id) DO UPDATE
SET column1 = "value1", column2 = "value2"
The best approach I know is to do an update, followed by an insert.
The "overhead of a select" is necessary, but it is not a terrible burden since you are searching on the primary key, which is fast.
You should be able to modify the below statements with your table & field names to do what you want.
--first, update any matches
UPDATE DESTINATION_TABLE DT
SET
MY_FIELD1 = (
SELECT MY_FIELD1
FROM SOURCE_TABLE ST
WHERE ST.PRIMARY_KEY = DT.PRIMARY_KEY
)
,MY_FIELD2 = (
SELECT MY_FIELD2
FROM SOURCE_TABLE ST
WHERE ST.PRIMARY_KEY = DT.PRIMARY_KEY
)
WHERE EXISTS(
SELECT ST2.PRIMARY_KEY
FROM
SOURCE_TABLE ST2
,DESTINATION_TABLE DT2
WHERE ST2.PRIMARY_KEY = DT2.PRIMARY_KEY
);
--second, insert any non-matches
INSERT INTO DESTINATION_TABLE(
MY_FIELD1
,MY_FIELD2
)
SELECT
ST.MY_FIELD1
,NULL AS MY_FIELD2 --insert NULL into this field
FROM
SOURCE_TABLE ST
WHERE NOT EXISTS(
SELECT DT2.PRIMARY_KEY
FROM DESTINATION_TABLE DT2
WHERE DT2.PRIMARY_KEY = ST.PRIMARY_KEY
);
Expanding on Aristotle’s answer you can SELECT from a dummy 'singleton' table (a table of your own creation with a single row). This avoids some duplication.
I've also kept the example portable across MySQL and SQLite and used a 'date_added' column as an example of how you could set a column only the first time.
REPLACE INTO page (
id,
name,
title,
content,
author,
date_added)
SELECT
old.id,
"about",
"About this site",
old.content,
42,
IFNULL(old.date_added,"21/05/2013")
FROM singleton
LEFT JOIN page AS old ON old.name = "about";
If someone wants to read my solution for SQLite in Cordova, I got this generic js method thanks to #david answer above.
function addOrUpdateRecords(tableName, values, callback) {
get_columnNames(tableName, function (data) {
var columnNames = data;
myDb.transaction(function (transaction) {
var query_update = "";
var query_insert = "";
var update_string = "UPDATE " + tableName + " SET ";
var insert_string = "INSERT INTO " + tableName + " SELECT ";
myDb.transaction(function (transaction) {
// Data from the array [[data1, ... datan],[()],[()]...]:
$.each(values, function (index1, value1) {
var sel_str = "";
var upd_str = "";
var remoteid = "";
$.each(value1, function (index2, value2) {
if (index2 == 0) remoteid = value2;
upd_str = upd_str + columnNames[index2] + "='" + value2 + "', ";
sel_str = sel_str + "'" + value2 + "', ";
});
sel_str = sel_str.substr(0, sel_str.length - 2);
sel_str = sel_str + " WHERE NOT EXISTS(SELECT changes() AS change FROM "+tableName+" WHERE change <> 0);";
upd_str = upd_str.substr(0, upd_str.length - 2);
upd_str = upd_str + " WHERE remoteid = '" + remoteid + "';";
query_update = update_string + upd_str;
query_insert = insert_string + sel_str;
// Start transaction:
transaction.executeSql(query_update);
transaction.executeSql(query_insert);
});
}, function (error) {
callback("Error: " + error);
}, function () {
callback("Success");
});
});
});
}
So, first pick up the column names with this function:
function get_columnNames(tableName, callback) {
myDb.transaction(function (transaction) {
var query_exec = "SELECT name, sql FROM sqlite_master WHERE type='table' AND name ='" + tableName + "'";
transaction.executeSql(query_exec, [], function (tx, results) {
var columnParts = results.rows.item(0).sql.replace(/^[^\(]+\(([^\)]+)\)/g, '$1').split(','); ///// RegEx
var columnNames = [];
for (i in columnParts) {
if (typeof columnParts[i] === 'string')
columnNames.push(columnParts[i].split(" ")[0]);
};
callback(columnNames);
});
});
}
Then build the transactions programmatically.
"Values" is an array you should build before and it represents the rows you want to insert or update into the table.
"remoteid" is the id I used as a reference, since I'm syncing with my remote server.
For the use of the SQLite Cordova plugin, please refer to the official link
I think this may be what you are looking for: ON CONFLICT clause.
If you define your table like this:
CREATE TABLE table1(
id INTEGER PRIMARY KEY ON CONFLICT REPLACE,
field1 TEXT
);
Now, if you do an INSERT with an id that already exists, SQLite automagically does UPDATE instead of INSERT.
Hth...
This method remixes a few of the other methods from answer in for this question and incorporates the use of CTE (Common Table Expressions). I will introduce the query then explain why I did what I did.
I would like to change the last name for employee 300 to DAVIS if there is an employee 300. Otherwise, I will add a new employee.
Table Name: employees
Columns: id, first_name, last_name
The query is:
INSERT OR REPLACE INTO employees (employee_id, first_name, last_name)
WITH registered_employees AS ( --CTE for checking if the row exists or not
SELECT --this is needed to ensure that the null row comes second
*
FROM (
SELECT --an existing row
*
FROM
employees
WHERE
employee_id = '300'
UNION
SELECT --a dummy row if the original cannot be found
NULL AS employee_id,
NULL AS first_name,
NULL AS last_name
)
ORDER BY
employee_id IS NULL --we want nulls to be last
LIMIT 1 --we only want one row from this statement
)
SELECT --this is where you provide defaults for what you would like to insert
registered_employees.employee_id, --if this is null the SQLite default will be used
COALESCE(registered_employees.first_name, 'SALLY'),
'DAVIS'
FROM
registered_employees
;
Basically, I used the CTE to reduce the number of times the select statement has to be used to determine default values. Since this is a CTE, we just select the columns we want from the table and the INSERT statement uses this.
Now you can decide what defaults you want to use by replacing the nulls, in the COALESCE function with what the values should be.
Following Aristotle Pagaltzis and the idea of COALESCE from Eric B’s answer, here it is an upsert option to update only few columns or insert full row if it does not exist.
In this case, imagine that title and content should be updated, keeping the other old values when existing and inserting supplied ones when name not found:
NOTE id is forced to be NULL when INSERT as it is supposed to be autoincrement. If it is just a generated primary key then COALESCE can also be used (see Aristotle Pagaltzis comment).
WITH new (id, name, title, content, author)
AS ( VALUES(100, 'about', 'About this site', 'Whatever new content here', 42) )
INSERT OR REPLACE INTO page (id, name, title, content, author)
SELECT
old.id, COALESCE(old.name, new.name),
new.title, new.content,
COALESCE(old.author, new.author)
FROM new LEFT JOIN page AS old ON new.name = old.name;
So the general rule would be, if you want to keep old values, use COALESCE, when you want to update values, use new.fieldname
If you don't mind doing this in two operations.
Steps:
1) Add new items with "INSERT OR IGNORE"
2) Update existing items with "UPDATE"
The input to both steps is the same collection of new or update-able items. Works fine with existing items that need no changes. They will be updated, but with the same data and therefore net result is no changes.
Sure, slower, etc. Inefficient. Yep.
Easy to write the sql and maintain and understand it? Definitely.
It's a trade-off to consider.
Works great for small upserts. Works great for those that don't mind sacrificing efficiency for code maintainability.
Complete example of upserting using WHERE to select the newer dated record.
-- https://www.db-fiddle.com/f/7jyj4n76MZHLLk2yszB6XD/22
DROP TABLE IF EXISTS db;
CREATE TABLE db
(
id PRIMARY KEY,
updated_at,
other
);
-- initial INSERT
INSERT INTO db (id,updated_at,other) VALUES(1,1,1);
SELECT * FROM db;
-- INSERT without WHERE
INSERT INTO db (id,updated_at,other) VALUES(1,2,2)
ON CONFLICT(id) DO UPDATE SET updated_at=excluded.updated_at;
SELECT * FROM db;
-- WHERE is FALSE
INSERT INTO db (id,updated_at,other) VALUES(1,2,3)
ON CONFLICT(id) DO UPDATE SET updated_at=excluded.updated_at, other=excluded.other
WHERE excluded.updated_at > updated_at;
SELECT * FROM db;
-- ok to SET a PRIMARY KEY. WHERE is TRUE
INSERT INTO db (id,updated_at,other) VALUES(1,3,4)
ON CONFLICT(id) DO UPDATE SET id=excluded.id, updated_at=excluded.updated_at, other=excluded.other
WHERE excluded.updated_at > updated_at;
SELECT * FROM db;
Having just read this thread and been disappointed that it wasn't easy to just to this "UPSERT"ing, I investigated further...
You can actually do this directly and easily in SQLITE.
Instead of using: INSERT INTO
Use: INSERT OR REPLACE INTO
This does exactly what you want it to do!
SELECT COUNT(*) FROM table1 WHERE id = 1;
if COUNT(*) = 0
INSERT INTO table1(col1, col2, cole) VALUES(var1,var2,var3);
else if COUNT(*) > 0
UPDATE table1 SET col1 = var4, col2 = var5, col3 = var6 WHERE id = 1;
What's wrong with this query:
INSERT INTO Users( weight, desiredWeight ) VALUES ( 160, 145 ) WHERE id = 1;
It works without the WHERE clause. I've seemed to have forgot my SQL.
MySQL INSERT Syntax does not support the WHERE clause so your query as it stands will fail. Assuming your id column is unique or primary key:
If you're trying to insert a new row with ID 1 you should be using:
INSERT INTO Users(id, weight, desiredWeight) VALUES(1, 160, 145);
If you're trying to change the weight/desiredWeight values for an existing row with ID 1 you should be using:
UPDATE Users SET weight = 160, desiredWeight = 145 WHERE id = 1;
If you want you can also use INSERT .. ON DUPLICATE KEY syntax like so:
INSERT INTO Users (id, weight, desiredWeight) VALUES(1, 160, 145) ON DUPLICATE KEY UPDATE weight=160, desiredWeight=145
OR even like so:
INSERT INTO Users SET id=1, weight=160, desiredWeight=145 ON DUPLICATE KEY UPDATE weight=160, desiredWeight=145
It's also important to note that if your id column is an autoincrement column then you might as well omit it from your INSERT all together and let mysql increment it as normal.
You can't combine a WHERE clause with a VALUES clause. You have two options as far as I am aware-
INSERT specifying values
INSERT INTO Users(weight, desiredWeight)
VALUES (160,145)
INSERT using a SELECT statement
INSERT INTO Users(weight, desiredWeight)
SELECT weight, desiredWeight
FROM AnotherTable
WHERE id = 1
You use the WHERE clause for UPDATE queries. When you INSERT, you are assuming that the row doesn't exist.
The OP's statement would then become;
UPDATE Users SET weight = 160, desiredWeight = 45 where id = 1;
In MySQL, if you want to INSERT or UPDATE, you can use the REPLACE query with a WHERE clause. If the WHERE doesn't exist, it INSERTS, otherwise it UPDATES.
EDIT
I think that Bill Karwin's point is important enough to pull up out of the comments and make it very obvious. Thanks Bill, it has been too long since I have worked with MySQL, I remembered that I had issues with REPLACE, but I forgot what they were. I should have looked it up.
That's not how MySQL's REPLACE works. It does a DELETE (which may be a no-op if the row does not exist), followed by an INSERT. Think of the consequences vis. triggers and foreign key dependencies. Instead, use INSERT...ON DUPLICATE KEY UPDATE.
I do not believe the insert has a WHERE clause.
Insert query doesn't support where keyword*
Conditions apply because you can use where condition for sub-select statements.
You can perform complicated inserts using sub-selects.
For example:
INSERT INTO suppliers
(supplier_id, supplier_name)
SELECT account_no, name
FROM customers
WHERE city = 'Newark';
By placing a "select" in the insert statement, you can perform multiples inserts quickly.
With this type of insert, you may wish to check for the number of rows being inserted. You can determine the number of rows that will be inserted by running the following SQL statement before performing the insert.
SELECT count(*)
FROM customers
WHERE city = 'Newark';
You can make sure that you do not insert duplicate information by using the EXISTS condition.
For example, if you had a table named clients with a primary key of client_id, you could use the following statement:
INSERT INTO clients
(client_id, client_name, client_type)
SELECT supplier_id, supplier_name, 'advertising'
FROM suppliers
WHERE not exists (select * from clients
where clients.client_id = suppliers.supplier_id);
This statement inserts multiple records with a subselect.
If you wanted to insert a single record, you could use the following statement:
INSERT INTO clients
(client_id, client_name, client_type)
SELECT 10345, 'IBM', 'advertising'
FROM dual
WHERE not exists (select * from clients
where clients.client_id = 10345);
The use of the dual table allows you to enter your values in a select statement, even though the values are not currently stored in a table.
See also How to insert with where clause
The right answer to this question will be sth like this:
a). IF want select before insert :
INSERT INTO Users( weight, desiredWeight )
select val1 , val2 from tableXShoulatNotBeUsers
WHERE somecondition;
b). IF record already exists use update instead of insert:
INSERT INTO Users( weight, desiredWeight ) VALUES ( 160, 145 ) WHERE id = 1;
Should be
Update Users set weight=160, desiredWeight=145 WHERE id = 1;
c). If you want to update or insert at the same time
Replace Users set weight=160, desiredWeight=145 WHERE id = 1;
Note):- you should provide values to all fields else missed field in query
will be set to null
d). If you want to CLONE a record from SAME table, just remember you cann't select
from table to which you are inserting therefore
create temporary table xtable ( weight int(11), desiredWeight int(11) ;
insert into xtable (weight, desiredWeight)
select weight, desiredWeight from Users where [condition]
insert into Users (weight, desiredWeight)
select weight , desiredWeight from xtable;
I think this pretty covers most of the scenarios
You simply cannot use WHERE when doing an INSERT statement:
INSERT INTO Users( weight, desiredWeight ) VALUES ( 160, 145 ) WHERE id = 1;
should be:
INSERT INTO Users( weight, desiredWeight ) VALUES ( 160, 145 );
The WHERE part only works in SELECT statements:
SELECT from Users WHERE id = 1;
or in UPDATE statements:
UPDATE Users set (weight = 160, desiredWeight = 145) WHERE id = 1;
A way to use INSERT and WHERE is
INSERT INTO MYTABLE SELECT 953,'Hello',43 WHERE 0 in (SELECT count(*) FROM MYTABLE WHERE myID=953);
In this case ist like an exist-test. There is no exception if you run it two or more times...
I think that the correct form to insert a value on a specify row is:
UPDATE table SET column = value WHERE columnid = 1
it works, and is similar if you write on Microsoft SQL Server
INSERT INTO table(column) VALUES (130) WHERE id = 1;
on mysql you have to Update the table.
Insert into = Adding rows to a table
Upate = update specific rows.
What would the where clause describe in your insert?
It doesn't have anything to match, the row doesn't exist (yet)...
You can do conditional INSERT based on user input.
This query will do insert only if input vars '$userWeight' and '$userDesiredWeight' are not blank
INSERT INTO Users(weight, desiredWeight )
select '$userWeight', '$userDesiredWeight'
FROM (select 1 a ) dummy
WHERE '$userWeight' != '' AND '$userDesiredWeight'!='';
If you are looking to insert some values into a new column of an altered table in each rows by mentioning its primary key, then just-->
UPDATE <table_name> SET <column_name> = '<value> WHERE <primary_key> = <primary_value>
It depends on the situation INSERT can actually have a where clause.
For example if you are matching values from a form.
Consider INSERT INTO Users(name,email,weight, desiredWeight) VALUES (fred,bb#yy.com,160,145) WHERE name != fred AND email != bb#yy.com
Makes sense doesn't it?
The simplest way is to use IF to violate your a key constraint. This only works for INSERT IGNORE but will allow you to use constraint in a INSERT.
INSERT INTO Test (id, name) VALUES (IF(1!=0,NULL,1),'Test');
After WHERE clause you put a condition, and it is used for either fetching data or for updating a row. When you are inserting data, it is assumed that the row does not exist.
So, the question is, is there any row whose id is 1? if so, use MySQL UPDATE, else use MySQL INSERT.
If you are specifying a particular record no for inserting data its better to use UPDATE statement instead of INSERT statement.
This type of query you have written in the question is like a dummy query.
Your Query is :-
INSERT INTO Users( weight, desiredWeight ) VALUES ( 160, 145 ) WHERE id = 1;
Here , you are specifying the id=1 , so better you use UPDATE statement to update the existing record.It is not recommended to use WHERE clause in case of INSERT.You should use UPDATE .
Now Using Update Query :-
UPDATE Users SET weight=160,desiredWeight=145 WHERE id=1;
Does WHERE-clause can be actually used with INSERT-INTO-VALUES in any
case?
The answer is definitively no.
Adding a WHERE clause after INSERT INTO ... VALUES ... is just invalid SQL, and will not parse.
The error returned by MySQL is:
mysql> INSERT INTO Users( weight, desiredWeight ) VALUES ( 160, 145 ) WHERE id = 1;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'WHERE id = 1' at line 1
The most important part of the error message is
... syntax to use near 'WHERE id = 1' ...
which shows the specific part the parser did not expect to find here: the WHERE clause.
its totall wrong. INSERT QUERY does not have a WHERE clause, Only UPDATE QUERY has it. If you want to add data Where id = 1 then your Query will be
UPDATE Users SET weight=160, desiredWeight= 145 WHERE id = 1;
No. As far as I am aware you cannot add the WHERE clause into this query. Maybe I've forgotten my SQL too, because I am not really sure why you need it anyway.
You Should not use where condition in Insert statement. If you want to do, use insert in a update statement and then update a existing record.
Actually can i know why you need a where clause in Insert statement??
Maybe based on the reason I might suggest you a better option.
I think your best option is use REPLACE instead INSERT
REPLACE INTO Users(id, weight, desiredWeight) VALUES(1, 160, 145);
DO READ THIS AS WELL
It doesn't make sense... even literally
INSERT means add a new row and when you say WHERE you define which row are you talking about in the SQL.
So adding a new row is not possible with a condition on an existing row.
You have to choose from the following:
A. Use UPDATE instead of INSERT
B. Use INSERT and remove WHERE clause ( I am just saying it...) or if you are real bound to use INSERT and WHERE in a single statement it can be done only via INSERT..SELECT clause...
INSERT INTO Users( weight, desiredWeight )
SELECT FROM Users WHERE id = 1;
But this serves an entirely different purpose and if you have defined id as Primary Key this insert will be failure, otherwise a new row will be inserted with id = 1.
I am aware that this is a old post but I hope that this will still help somebody, with what I hope is a simple example:
background:
I had a many to many case: the same user is listed multiple times with multiple values and I wanted to Create a new record, hence UPDATE wouldn't make sense in my case and I needed to address a particular user just like I would do using a WHERE clause.
INSERT into MyTable(aUser,aCar)
value(User123,Mini)
By using this construct you actually target a specific user (user123,who has other records) so you don't really need a where clause, I reckon.
the output could be:
aUser aCar
user123 mini
user123 HisOtherCarThatWasThereBefore
correct syntax for mysql insert into statement using post method is:
$sql="insert into ttable(username,password) values('$_POST[username]','$_POST[password]')";
i dont think that we can use where clause in insert statement
INSERT INTO Users(weight, desiredWeight )
SELECT '$userWeight', '$userDesiredWeight'
FROM (select 1 a ) dummy
WHERE '$userWeight' != '' AND '$userDesiredWeight'!='';
You can't use INSERT and WHERE together. You can use UPDATE clause for add value to particular column in particular field like below code;
UPDATE Users
SET weight='160',desiredWeight ='145'
WHERE id =1
You can do that with the below code:
INSERT INTO table2 (column1, column2, column3, ...)
SELECT column1, column2, column3, ...
FROM table1
WHERE condition
I think you should do it like this, if you want to validate table not to use email twice
Code :
INSERT INTO tablename(fullname,email)
SELECT * FROM (SELECT 'fullnameValue' AS fullname_field,'emailValue' AS email_field) entry WHERE entry.email_field NOT IN (SELECT email FROM tablename);
All the above answers give you the plain MySQL statements.
If you are using the where condition in PHPMyAdmin to update the existing row of a table, below is the suggestion to use.
UPDATE `Table_Name` SET `row1`='[value-1]',`row2`='[value-2]',`row3`='[value-3]' WHERE 1
You can include any value between '' of each row. Not necessarily to put [] to indicate the value. Do not change anything after where. Simply click go after giving the value to update.
Example:
UPDATE `Mytable_name` SET `abc`='xyz',`def`='uvw'
WHERE 1