This question reflects my issue. How to do this in SQLite?
I've tried UPDATE with self-joins, isolating the self join in sub-query, triggers, and something similar to this. Here is an example:
UPDATE stage
SET title =
(
SELECT
prior.title
FROM
stage prior,
stage now
WHERE
prior.rownum+1 = now.rownum
)
WHERE
title is null
Every table in SQLite has got a pseudo-column called rowid (which can be accessible under several different names: rowid, oid, _rowid_, unless those names are assigned to other, real, columns). The rowid column is essentially a unique row identifier and you can use it as a sort criterion and/or in conditions.
The following query demonstrates how your problem can be solved with the help of rowid:
UPDATE stage
SET title = (
SELECT title
FROM stage AS prev
WHERE title IS NOT NULL AND prev.rowid < stage.rowid
ORDER BY prev.rowid DESC
LIMIT 1
)
WHERE title IS NULL
Here's a demo on SQL Fiddle.
You can read more about rowid in this SQLite manual.
I presented a solution for the problem you referred (I successfully tested on SQL2008, SQLite3 and Oracle11g). I copied that solution below:
CREATE TABLE test(mysequence INT, mynumber INT);
INSERT INTO test VALUES(1, 3);
INSERT INTO test VALUES(2, NULL);
INSERT INTO test VALUES(3, 5);
INSERT INTO test VALUES(4, NULL);
INSERT INTO test VALUES(5, NULL);
INSERT INTO test VALUES(6, 2);
SELECT t1.mysequence, t1.mynumber AS ORIGINAL
, (
SELECT t2.mynumber
FROM test t2
WHERE t2.mysequence = (
SELECT MAX(t3.mysequence)
FROM test t3
WHERE t3.mysequence <= t1.mysequence
AND mynumber IS NOT NULL
)
) AS CALCULATED
FROM test t1;
-- below here it was only tested in SQLite3, but I believe it should
-- work on other DBMS since it uses standard/non-proprietary SQL
UPDATE test
SET mynumber = (
SELECT t2.mynumber
FROM test t2
WHERE t2.mysequence = (
SELECT MAX(t3.mysequence)
FROM test t3
WHERE t3.mysequence <= test.mysequence
AND mynumber IS NOT NULL
)
);
Related
I have a table that looks like:
ID|CREATED |VALUE
1 |1649122158|200
1 |1649122158|200
1 |1649122158|200
That I'd like to look like:
ID|CREATED |VALUE
1 |1649122158|200
And I run the following query:
DELETE FROM MY_TABLE T USING (SELECT ID,CREATED,ROW_NUMBER() OVER (PARTITION BY ID ORDER BY CREATED DESC) AS RANK_IN_KEY FROM MY_TABLE T) X WHERE X.RANK_IN_KEY <> 1 AND T.ID = X.ID AND T.CREATED = X.CREATED
But it removes everything from MY_TABLE and not just other rows with the same value. This is more than just selecting distinct records, I'd like to enforce a unique constraint to get the latest value of ID and keep just one record for it, even if there were duplicates.
So
ID|CREATED |VALUE
1 |1649122158|200
1 |1649122159|300
2 |1649122158|200
2 |1649122158|200
3 |1649122170|500
3 |1649122160|200
Would become (using the same final unique constraint statement):
ID|CREATED |VALUE
1 |1649122159|300
2 |1649122158|200
3 |1649122170|500
How can I improve my logic to properly handle these unique constraint modifications?
Check out this post: https://community.snowflake.com/s/question/0D50Z00008EJgemSAD/how-to-delete-duplicate-records-
If all columns make up a unique records, the recommended solution is the insert all the records into a new table with SELECT DISTINCT * and do a swap. You could also do a INSERT OVERWRITE INTO the same table.
Something like INSERT OVERWRITE INTO tableA SELECT DISTINCT * FROM tableA;
The following setup should leave rows with id of 1 and 3. And not delete all rows as you say.
Schema
create table t (
id int,
created int ,
value int
);
insert into t values(1, 1649122158, 200);
insert into t values(1 ,1649122159, 300);
insert into t values(2 ,1649122158, 200);
insert into t values(2 ,1649122158, 200);
insert into t values(3 ,1649122170, 500);
insert into t values(3 ,1649122160, 200);
Delete statement
with x as (
SELECT
id, created,
row_number() over(partition by id) as r
FROM t
)
delete from t
using x
where x.id = t.id and x.r <> 1 and x.created = t.created
;
Output
select * from t;
1 1649122158 200
3 1649122170 500
The logic is such, that the table in the using clause is joined with the operated on table. Following the join logic, it just matches by some key. In your case, you have key as {id,created}. This key is duplicated for rows with id of 2. So the whole group is deleted.
I'm no savvy in database schemas. But as a thought, you may add a row with a rank to existing table. And after that you can proceed with deletion. This way you do not need to create other table and insert values to that. Be warned that data may become fragmented(physically, on disks). So you will need to run some kind of tune up later.
Update
You may find this almost one-liner interesting:
SO answer
I will duplicate code here, as it is so small and well written.
WITH
u AS (SELECT DISTINCT * FROM your_table),
x AS (DELETE FROM your_table)
INSERT INTO your_table SELECT * FROM u;
I have two tables:
CREATE TABLE public.test
(
id integer NOT NULL GENERATED ALWAYS AS IDENTITY
)
and
CREATE TABLE public.test2
(
id integer,
test_id integer
)
Table test2 has two rows (1, null) and (2, null). Table test has nothing. Now I want to fill test_id by creating new rows in test. I nead a new entity each time so that I will have (1, 1), (2, 2), etc. I try to prepare update query with an insert statement but I don't understand how to do it. This is what I try:
update t2 set t2.test_id = t.id
from test2 t2 full join (INSERT INTO test(id) VALUES (default) RETURNING id) t on t2.test_id = t.id
but I get the following:
ERROR: syntax error at or near "INTO"
LINE 2: from test2 t2 full join (INSERT INTO test(id) VALUES (defaul...
^
SQL state: 42601
Character: 65
Can I create the query I want somehow?
Having just one column in the target table makes things a little tricky. It might be simpler to generate and assign the new ids first, using next_val, and then insert the values in test (we need option overriding system value to insert into a generated always colum,)
with t2 as (
update test2
set test_id = nextval('test_id_seq')
where test_id is null
returning test_id
)
insert into test(id) overriding system value
select test_id from t2
Demo on DB Fiddlde
I'm writing a function in node.js to query a PostgreSQL table.
If the row exists, I want to return the id column from the row.
If it doesn't exist, I want to insert it and return the id (insert into ... returning id).
I've been trying variations of case and if else statements and can't seem to get it to work.
A solution in a single SQL statement. Requires PostgreSQL 8.4 or later though.
Consider the following demo:
Test setup:
CREATE TEMP TABLE tbl (
id serial PRIMARY KEY
,txt text UNIQUE -- obviously there is unique column (or set of columns)
);
INSERT INTO tbl(txt) VALUES ('one'), ('two');
INSERT / SELECT command:
WITH v AS (SELECT 'three'::text AS txt)
,s AS (SELECT id FROM tbl JOIN v USING (txt))
,i AS (
INSERT INTO tbl (txt)
SELECT txt
FROM v
WHERE NOT EXISTS (SELECT * FROM s)
RETURNING id
)
SELECT id, 'i'::text AS src FROM i
UNION ALL
SELECT id, 's' FROM s;
The first CTE v is not strictly necessary, but achieves that you have to enter your values only once.
The second CTE s selects the id from tbl if the "row" exists.
The third CTE i inserts the "row" into tbl if (and only if) it does not exist, returning id.
The final SELECT returns the id. I added a column src indicating the "source" - whether the "row" pre-existed and id comes from a SELECT, or the "row" was new and so is the id.
This version should be as fast as possible as it does not need an additional SELECT from tbl and uses the CTEs instead.
To make this safe against possible race conditions in a multi-user environment:
Also for updated techniques using the new UPSERT in Postgres 9.5 or later:
Is SELECT or INSERT in a function prone to race conditions?
I would suggest doing the checking on the database side and just returning the id to nodejs.
Example:
CREATE OR REPLACE FUNCTION foo(p_param1 tableFoo.attr1%TYPE, p_param2 tableFoo.attr1%TYPE) RETURNS tableFoo.id%TYPE AS $$
DECLARE
v_id tableFoo.pk%TYPE;
BEGIN
SELECT id
INTO v_id
FROM tableFoo
WHERE attr1 = p_param1
AND attr2 = p_param2;
IF v_id IS NULL THEN
INSERT INTO tableFoo(id, attr1, attr2) VALUES (DEFAULT, p_param1, p_param2)
RETURNING id INTO v_id;
END IF;
RETURN v_id:
END;
$$ LANGUAGE plpgsql;
And than on the Node.js-side (i'm using node-postgres in this example):
var pg = require('pg');
pg.connect('someConnectionString', function(connErr, client){
//do some errorchecking here
client.query('SELECT id FROM foo($1, $2);', ['foo', 'bar'], function(queryErr, result){
//errorchecking
var id = result.rows[0].id;
};
});
Something like this, if you are on PostgreSQL 9.1
with test_insert as (
insert into foo (id, col1, col2)
select 42, 'Foo', 'Bar'
where not exists (select * from foo where id = 42)
returning foo.id, foo.col1, foo.col2
)
select id, col1, col2
from test_insert
union
select id, col1, col2
from foo
where id = 42;
It's a bit longish and you need to repeat the id to test for several times, but I can't think of a different solution that involves a single SQL statement.
If a row with id=42 exists, the writeable CTE will not insert anything and thus the existing row will be returned by the second union part.
When testing this I actually thought the new row would be returned twice (therefor a union not a union all) but it turns out that the result of the second select statement is actually evaluated before the whole statement is run and it does not see the newly inserted row. So in case a new row is inserted, it will be taken from the "returning" part.
create table t (
id serial primary key,
a integer
)
;
insert into t (a)
select 2
from (
select count(*) as s
from t
where a = 2
) s
where s.s = 0
;
select id
from t
where a = 2
;
I have the statement below when creating a Oracle view. The statement is to return a particular value as MYVALUE column.
(SELECT myval
FROM (SELECT myval
FROM mytable
WHERE primary_key = /*CS.primary_key*/ 12345
ORDER BY table_primary_key ASC)
WHERE ROWNUM < 2) AS MYVALUE,
The inner query can return more than one row. I am only interested in the 1st record and it must be ordered by the table_primary_key, hence the use of the sub query to allow the ROWNUM selection.
When I create the query in its current state above, the view is created successfully. When I uncomment CS.primary_key and remove the hardcoded 12345, the view creation fails, with no description of why
(SQLDeveloper 2.1): "Failed: Warning: execution completed with warning".
Also, just to try and narrow down the problem, I removed the ORDER BY, and ROWNUM as below, and the same error occurs
(SELECT myval
FROM (SELECT myval
FROM mytable
WHERE primary_key = CS.primary_key)
) AS MYVALUE,
Lastly, I know CS.primary_key is a valid reference, as I use in in other parts of my view without issues.
Any idea why the reference is valid, or how to get a more detailed error message?
edit: updated starting opening bracket
EDIT2: Thanks for the responses so far. Here is a summary of the problem, I think the CS.PRIMARY key should be in scope, as I use it in other places in my query. The code below works, but if I replace the hardcoded 1 to CS.primary_key, it fails:
drop view myview;
drop table mytable;
drop table mytable_parent;
drop table proof_table;
-- ISSUE TABLES
create table mytable_parent ( primary_key number primary key );
create table mytable ( primary_key number, myval varchar(255), parent_primary_key number);
insert into mytable_parent values (1);
insert into mytable_parent values (2);
insert into mytable values (1, 'myval1-1', 1);
insert into mytable values (2, 'myval1-2', 1);
insert into mytable values (3, 'myval2-1', 2);
-- EXAMPLE TABLE TO PROVE CS.* WORKS
create table proof_table ( primary_key number primary key, parent_primary_key number, any_old_value varchar(255));
insert into proof_table values (1, 1, 'proofval1-1');
insert into proof_table values (2, 2, 'proofval1-2');
-- VIEW
CREATE OR REPLACE FORCE VIEW myview AS
SELECT
-- PROOF STATEMENT USING CS.primary_key SUCCESSFULLY
(SELECT any_old_value FROM proof_table WHERE parent_primary_key IN
(SELECT primary_key FROM proof_table WHERE parent_primary_key =
-- USING CS REFERENCE, NO PROBLEM
CS.primary_key)
) AS PROOF_VALUE,
-- PROBLEM STATEMENT
(SELECT myval FROM (SELECT myval FROM mytable
WHERE parent_primary_key = /*CS.primary_key*/ 1
ORDER BY primary_key ASC)
WHERE ROWNUM < 2) AS MYVALUE
-- DEFINE CS
FROM mytable_parent CS;
From the limited information you have supplied I'd suggest that CS.PRIMARY_KEY is out of scope for your subquery. Hence it compiles OK when you use a literal and doesn't compile when it has to resolve the reference to CS.PRIMARY_KEY.
Include whichever table CS refers to in the subquery (with the relevent criteria) to check that this is the case.
If it is then you'll need to rewrite your query to ensure the CS table is in scope for all its dependencies.
Hope it helps...
See the details in this link: http://etutorials.org/SQL/Mastering+Oracle+SQL/Chapter+5.+Subqueries/5.4+Inline+Views/
You will see that inline views are executed prior to the outer query and nesting the reference to the CS table too deeply will cause it to become out of scope.
Replace the nested subquery where you do your ordering:
(SELECT myval
FROM (SELECT myval
FROM mytable
WHERE parent_primary_key = CS.primary_key
ORDER BY primary_key ASC)
WHERE ROWNUM < 2) AS MYVALUE
with an analytic function or other way of limiting your rows and it will work fine
This is untested but replacing the above code with:
(SELECT DISTINCT
FIRST_VALUE(myval) OVER (PARTITION BY parent_primary_key ORDER BY primary_key)
FROM mytable
WHERE parent_primary_key = CS.primary_key) AS MYVALUE
might be close to what you want and the reference to CS is in scope.
I was wondering if it is possible to move all rows of data from one table to another, that match a certain query?
For example, I need to move all table rows from Table1 to Table2 where their username = 'X' and password = 'X', so that they will no longer appear in Table1.
I'm using SQL Server 2008 Management Studio.
Should be possible using two statements within one transaction, an insert and a delete:
BEGIN TRANSACTION;
INSERT INTO Table2 (<columns>)
SELECT <columns>
FROM Table1
WHERE <condition>;
DELETE FROM Table1
WHERE <condition>;
COMMIT;
This is the simplest form. If you have to worry about new matching records being inserted into table1 between the two statements, you can add an and exists <in table2>.
This is an ancient post, sorry, but I only came across it now and I wanted to give my solution to whoever might stumble upon this one day.
As some have mentioned, performing an INSERT and then a DELETE might lead to integrity issues, so perhaps a way to get around it, and to perform everything neatly in a single statement, is to take advantage of the [deleted] temporary table.
DELETE FROM [source]
OUTPUT [deleted].<column_list>
INTO [destination] (<column_list>)
All these answers run the same query for the INSERT and DELETE. As mentioned previously, this risks the DELETE picking up records inserted between statements and could be slow if the query is complex (although clever engines "should" make the second call fast).
The correct way (assuming the INSERT is into a fresh table) is to do the DELETE against table1 using the key field of table2.
The delete should be:
DELETE FROM tbl_OldTableName WHERE id in (SELECT id FROM tbl_NewTableName)
Excuse my syntax, I'm jumping between engines but you get the idea.
A cleaner representation of what some other answers have hinted at:
DELETE sourceTable
OUTPUT DELETED.*
INTO destTable (Comma, separated, list, of, columns)
WHERE <conditions (if any)>
Yes it is. First INSERT + SELECT and then DELETE orginals.
INSERT INTO Table2 (UserName,Password)
SELECT UserName,Password FROM Table1 WHERE UserName='X' AND Password='X'
then delete orginals
DELETE FROM Table1 WHERE UserName='X' AND Password='X'
you may want to preserve UserID or someother primary key, then you can use IDENTITY INSERT to preserve the key.
see more on SET IDENTITY_INSERT on MSDN
You should be able to with a subquery in the INSERT statement.
INSERT INTO table1(column1, column2) SELECT column1, column2 FROM table2 WHERE ...;
followed by deleting from table1.
Remember to run it as a single transaction so that if anything goes wrong you can roll the entire operation back.
Use this single sql statement which is safe no need of commit/rollback with multiple statements.
INSERT Table2 (
username,password
) SELECT username,password
FROM (
DELETE Table1
OUTPUT
DELETED.username,
DELETED.password
WHERE username = 'X' and password = 'X'
) AS RowsToMove ;
Works on SQL server make appropriate changes for MySql
Try this
INSERT INTO TABLE2 (Cols...) SELECT Cols... FROM TABLE1 WHERE Criteria
Then
DELETE FROM TABLE1 WHERE Criteria
You could try this:
SELECT * INTO tbl_NewTableName
FROM tbl_OldTableName
WHERE Condition1=#Condition1Value
Then run a simple delete:
DELETE FROM tbl_OldTableName
WHERE Condition1=#Condition1Value
You may use "Logical Partitioning" to switch data between tables:
By updating the Partition Column, data will be automatically moved to the other table:
here is the sample:
CREATE TABLE TBL_Part1
(id INT NOT NULL,
val VARCHAR(10) NULL,
PartitionColumn VARCHAR(10) CONSTRAINT CK_Part1 CHECK(PartitionColumn = 'TBL_Part1'),
CONSTRAINT TBL_Part1_PK PRIMARY KEY(PartitionColumn, id)
);
CREATE TABLE TBL_Part2
(id INT NOT NULL,
val VARCHAR(10) NULL,
PartitionColumn VARCHAR(10) CONSTRAINT CK_Part2 CHECK(PartitionColumn = 'TBL_Part2'),
CONSTRAINT TBL_Part2_PK PRIMARY KEY(PartitionColumn, id)
);
GO
CREATE VIEW TBL(id, val, PartitionColumn)
WITH SCHEMABINDING
AS
SELECT id, val, PartitionColumn FROM dbo.TBL_Part1
UNION ALL
SELECT id, val, PartitionColumn FROM dbo.TBL_Part2;
GO
--Insert sample to TBL ( will be inserted to Part1 )
INSERT INTO TBL
VALUES(1, 'rec1', 'TBL_Part1');
INSERT INTO TBL
VALUES(2, 'rec2', 'TBL_Part1');
GO
--Query sub table to verify
SELECT * FROM TBL_Part1
GO
--move the data to table TBL_Part2 by Logical Partition switching technique
UPDATE TBL
SET
PartitionColumn = 'TBL_Part2';
GO
--Query sub table to verify
SELECT * FROM TBL_Part2
Here is how do it with single statement
WITH deleted_rows AS (
DELETE FROM source_table WHERE id = 1
RETURNING *
)
INSERT INTO destination_table
SELECT * FROM deleted_rows;
EXAMPLE:
postgres=# select * from test1 ;
id | name
----+--------
1 | yogesh
2 | Raunak
3 | Varun
(3 rows)
postgres=# select * from test2;
id | name
----+------
(0 rows)
postgres=# WITH deleted_rows AS (
postgres(# DELETE FROM test1 WHERE id = 1
postgres(# RETURNING *
postgres(# )
postgres-# INSERT INTO test2
postgres-# SELECT * FROM deleted_rows;
INSERT 0 1
postgres=# select * from test2;
id | name
----+--------
1 | yogesh
(1 row)
postgres=# select * from test1;
id | name
----+--------
2 | Raunak
3 | Varun
If the two tables use the same ID or have a common UNIQUE key:
1) Insert the selected record in table 2
INSERT INTO table2 SELECT * FROM table1 WHERE (conditions)
2) delete the selected record from table1 if presents in table2
DELETE FROM table1 as A, table2 as B WHERE (A.conditions) AND (A.ID = B.ID)
It will create a table and copy all the data from old table to new table
SELECT * INTO event_log_temp FROM event_log
And you can clear the old table data.
DELETE FROM event_log
For some scenarios, it might be the easiest to script out Table1, rename the existing Table1 to Table2 and run the script to recreate Table1.