How can I copy a table to another while changing some values ?
I mean for example something like :
perso=# create table chk ( id serial primary key, Q text , R text not null unique, p text not null unique) ;
CREATE TABLE
perso=# select * from chk
perso-# ;
id | q | r | p
----+---+---+---
(0 rows)
as a destination
an I would use datas from
perso=# select * from tmp
;
idt | qt | rt | pt
-----+----+----+----
I want to copy data from tmp table to chk table and at once if possible update data on chk so
chk.p is equal to tmp.t with crypt('t', gen_salt('bf'))
I tried to copy with subselecting but I got so many different types of errors ... I just can't explain nor paste them all here.
short ask : update data from a table to store results in another
I guess the good way should be to do something like :
update chk
set
......
from tmp (colums, crypt('t', gen_salt('bf'))
....
but how ?
To copy rows from one table to another, you can use insert ... select. This is a very flexible syntax (the select part may use all the features that a regular select query can), which lets you do what you want in a simple manner:
insert into chk (id, q, r, p)
select idt, dt, rt, crypt('t', gen_salt('bf'))
from tmp
In case you wanted to update the original table:
update tmp set pt = crypt('t', gen_salt('bf')) ;
Related
I am trying to modify a table of my taking the id of another table being that this other table is with another column, taking only if the data of that column is 2, take the ID of that column and use in the one that I want to change, example:
UPDATE QuestData SET RepeatFinish = 100000
WHERE QuestID =
(
SELECT * FROM Quest WHERE QuestID = 2
);
But QuestData have so much data and table Quest too, how can i make this?
UPDATE QuestData SET RepeatFinish = 100000
WHERE QuestID in
(
SELECT id FROM Quest WHERE QuestID = 2
);
Change the id in select by yours. when using the in with an other request the select must return a one field that will be used in IN, you are using * so we don't know with what we should compare the QuestID
In my project I need to create a script that insert data with auto generate value for the primary key and then to reuse this number for foreign on other tables.
I'm trying to use the WITH statement in order to keep that value.
For instance, I'm trying to do this:
WITH tmp as (SELECT ID FROM (INSERT INTO A ... VALUES ...))
INSERT INTO B ... VALUES tmp.ID ...
But I can't make it work.
Is it at least possible to do it or am I completely wrong???
Thank you
Yes, it is possible, if your DB2-server version supports the syntax.
For example:
create table xemp(id bigint generated always as identity, other_stuff varchar(20));
create table othertab(xemp_id bigint);
SELECT id FROM FINAL TABLE
(INSERT INTO xemp(other_stuff)
values ('a'), ('b'), ('c'), ('d')
) ;
The above snippet of code gives the result below:
ID
--------------------
1
2
3
4
4 record(s) selected.
If you want to re-use the ID to populate another table:
with tmp1(id) as ( SELECT id FROM new TABLE (INSERT INTO xemp(other_stuff) values ('a1'), ('b1'), ('c1'), ('d1') ) tmp3 )
, tmp2 as (select * from new table (insert into othertab(xemp_id) select id from tmp1 ) tmp4 )
select * from othertab;
As per my understanding
You will have to create an auto-increment field with the sequence object (this object generates a number sequence).
You can CREATE SEQUENCE to achieve the auto increment value :
CREATE SEQUENCE seq_person
MINVALUE 1
START WITH 1
INCREMENT BY 1
CACHE 10
Similar to this, is it possible to convert a String field to Numeric in PostgreSQL. For instance,
create table test (name text);
insert into test (name) values ('amy');
insert into test (name) values ('bob');
insert into test (name) values ('bob');
insert into test (name) values ('celia');
and add a field that is
name | num
-------+-----
amy | 1
bob | 2
bob | 2
celia | 3
The most effective "hash"-function of all is a serial primary key - giving you a unique number like you wished for in the question.
I also deal with duplicates in this demo:
CREATE TEMP TABLE string (
string_id serial PRIMARY KEY
,string text NOT NULL UNIQUE -- no dupes
,ct int NOT NULL DEFAULT 1 -- count instead of dupe rows
);
Then you would enter new strings like this:
(Data-modifying CTE requires PostgreSQL 9.1 or later.)
WITH x AS (SELECT 'abc'::text AS nu)
, y AS (
UPDATE string s
SET ct = ct + 1
FROM x
WHERE s.string = x.nu
RETURNING TRUE
)
INSERT INTO string (string)
SELECT nu
FROM x
WHERE NOT EXISTS (SELECT 1 FROM y);
If the string nu already exists, the count (ct) is increased by 1. If not, a new row is inserted, starting with a count of 1.
The UNIQUE also adds an index on the column string.string automatically, which leads to optimal performance for this query.
Add additional logic (triggers ?) for UPDATE / DELETE to make this bullet-proof - if needed.
Note, there is a super-tiny race condition here, if two concurrent transactions try to add the same string at the same moment in time. To be absolutely sure, you could use SERIALIZABLE transactions. More info and links under this this related question.
Live demo at sqlfiddle.
How 'bout a hash, such as md5, of name?
create table test (name text, hash text);
-- later
update test set hash = md5(name);
If you need to convert that md5 text to a number: Hashing a String to a Numeric Value in PostgresSQL
If they are all single characters, you could do this:
ALTER TABLE test ADD COLUMN num int;
UPDATE test SET num = ascii(name);
Though that would only return the character for the first letter if the string was more than a single character.
The exact case shown in your request can be produced with the dense_rank window function:
regress=# SELECT name, dense_rank() OVER (ORDER BY name) FROM test;
name | dense_rank
-------+------------
amy | 1
bob | 2
bob | 2
celia | 3
(4 rows)
so if you were adding a number for each row, you'd be able to do something like:
ALTER TABLE test ADD COLUMN some_num integer;
WITH gen(gen_name, gen_num) AS
(SELECT name, dense_rank() OVER (ORDER BY name) FROM test GROUP BY name)
UPDATE test SET some_num = gen_num FROM gen WHERE name = gen_name;
ALTER TABLE test ALTER COLUMN some_num SET NOT NULL;
however I think it's much more sensible to use a hash or to assign generated keys. I'm just showing that your example can be achieved.
The biggest problem with this approach is that inserting new data is a pain. It's a ranking (like your example shows) so if you INSERT INTO test (name) VALUES ('billy'); then the ranking changes.
I am working on a stored procedure in SQL Server 2008 for resetting an integer column in a database table. This integer column stores or persists the display order of the item rows. Users are able to drag and drop items in a particular sort order and we persist that order in the database table using this "Order Rank Integer".
Display queries for items always append a "ORDER BY OrderRankInt" when retrieving data so the user sees the items in the order they previously specified.
The problem is that this integer column collects a lot of duplicate values after the table items are re-ordered a bit. Hence...
Table
--------
Name | OrderRankInt
a | 1
b | 2
c | 3
d | 4
e | 5
f | 6
After a lot of reordering by the user becomes....
Table
--------
Name | OrderRankInt
a | 1
b | 2
c | 2
d | 2
e | 2
f | 6
These duplicates are primarily because of insertions and user specified order numbers. We're not trying to prevent duplicate order ranks, but we'd like a way to 'Fix' the table on item inserts/modifies.
Is there a way I can reset the OrderRankInt column with a single UPDATE Query?
Or do I need to use a cursor? What would the syntax for that cursor look like?
Thanks,
Kervin
EDIT
Update with Remus Rusanu solution. Thanks!!
CREATE PROCEDURE EPC_FixTableOrder
#sectionId int = 0
AS
BEGIN
-- "Common Table Expression" to append a 'Row Number' to the table
WITH tempTable AS
(
SELECT OrderRankInt, ROW_NUMBER() OVER (ORDER BY OrderRankInt) AS rn
FROM dbo.[Table]
WHERE sectionId = #sectionId -- Fix for a specified section
)
UPDATE tempTable
SET OrderRankInt = rn; -- Set the Order number to the row number via CTE
END
GO
with cte as (
select OrderId, row_number() over (order by Name) as rn
from Table)
update cte
set OrderId = rn;
This doesn't account for any foreign key relationships, I hope you are taken care of those.
Fake it. Make the column nullable, set the values to NULL, alter it to be an autonumber, and then turn off autonumber and nullable.
(You could skip the nullable steps.)
I've got big (>Mil rows) MySQL database messed up by duplicates. I think it could be from 1/4 to 1/2 of the whole db filled with them.
I need to get rid of them quick (i mean query execution time).
Here's how it looks:
id (index) | text1 | text2 | text3
text1 & text2 combination should be unique,
if there are any duplicates, only one combination with text3 NOT NULL should remain. Example:
1 | abc | def | NULL
2 | abc | def | ghi
3 | abc | def | jkl
4 | aaa | bbb | NULL
5 | aaa | bbb | NULL
...becomes:
1 | abc | def | ghi #(doesn't realy matter id:2 or id:3 survives)
2 | aaa | bbb | NULL #(if there's no NOT NULL text3, NULL will do)
New ids cold be anything, they do not depend on old table ids.
I've tried things like:
CREATE TABLE tmp SELECT text1, text2, text3
FROM my_tbl;
GROUP BY text1, text2;
DROP TABLE my_tbl;
ALTER TABLE tmp RENAME TO my_tbl;
Or SELECT DISTINCT and other variations.
While they work on small databases, query execution time on mine is just huge (never got to the end, actually; > 20 min)
Is there any faster way to do that? Please help me solve this problem.
I believe this will do it, using on duplicate key + ifnull():
create table tmp like yourtable;
alter table tmp add unique (text1, text2);
insert into tmp select * from yourtable
on duplicate key update text3=ifnull(text3, values(text3));
rename table yourtable to deleteme, tmp to yourtable;
drop table deleteme;
Should be much faster than anything that requires group by or distinct or a subquery, or even order by. This doesn't even require a filesort, which is going to kill performance on a large temporary table. Will still require a full scan over the original table, but there's no avoiding that.
Found this simple 1-line code to do exactly what I needed:
ALTER IGNORE TABLE dupTest ADD UNIQUE INDEX(a,b);
Taken from:
http://mediakey.dk/~cc/mysql-remove-duplicate-entries/
DELETE FROM dups
WHERE id NOT IN(
SELECT id FROM (
SELECT DISTINCT id, text1, text2
FROM dups
GROUP BY text1, text2
ORDER BY text3 DESC
) as tmp
)
This queries all records, groups by the distinction fields and orders by ID (means we pick the first not null text3 record). Then we select the id's from that result (these are good ids...they wont be deleted) and delete all IDs that AREN'T those.
Any query like this affecting the entire table will be slow. You just need to run it and let it roll out so you can prevent it in the future.
After you have done this "fix" I would apply UNIQUE INDEX (text1, text2) to that table. To prevent the posibility of duplicates in the future.
If you want to go the "create a new table and replace the old one" route. You could use the very inner select statement to create your insert statement.
MySQL specific (assumes new table is named my_tbl2 and has exactly the same structure):
INSERT INTO my_tbl2
SELECT DISTINCT id, text1, text2, text3
FROM dups
GROUP BY text1, text2
ORDER BY text3 DESC
See MySQL INSERT ... SELECT for more information.
remove duplicates without removing foreign keys
create table tmp like mytable;
ALTER TABLE tmp ADD UNIQUE INDEX(text1, text2, text3, text4, text5, text6);
insert IGNORE into tmp select * from mytable;
delete from mytable where id not in ( select id from tmp);
If you can create a new table, do so with a unique key on the text1 + text2 fields. Then insert into the table ignoring errors (using the INSERT IGNORE syntax):
select * from my_tbl order by text3 desc
I think the order by text3 desc will put the NULLs last, but double check that.
Indexes on all those columns could help a lot, but creating them now could be pretty slow.
For large tables with few duplicates, you may want to avoid copying the whole table to another place. One way is to create a temporary table holding the rows you want to keep (for each key with duplicates), and then delete duplicates from the original table.
An example is given here.
I don't have much experience with MySQL. If it has analytic functions try:
delete from my_tbl
where id in (
select id
from (select id, row_number()
over (partition by text1, text2 order by text3 desc) as rn
from my_tbl
/* optional: where text1 like 'a%' */
) as t2
where rn > 1
)
the optional where clause makes the means you'll have to run it multiple times, one for each letter, etc. Create an index on text1?
Before running this, confirm that "text desc" will sort nulls last in MySQL.
I know this is an Old thread but I have a somewhat messy method that is much faster and customizable, in terms of speed I'd say 10sec instead of 100sec (10:1).
My method does required all that messy stuff you were trying to avoid:
Group by (and Having)
group concat with ORDER BY
2 temporary tables
using files on disk!
somehow (php?) deleting the file after
But when you are talking about MILLIONS (or in my case Tens of Millions) it's worth it.
anyway its not much because comment are in portuguese but here is my sample:
EDIT: if I get comments I'll explain further how it works :)
START TRANSACTION;
DROP temporary table if exists to_delete;
CREATE temporary table to_delete as (
SELECT
-- escolhe todos os IDs duplicados menos os que ficam na BD
-- A ordem de escolha dos IDs é dada por "ORDER BY campo_ordenacao DESC" em que o primeiro é o que fica
right(
group_concat(id ORDER BY campos_ordenacao DESC SEPARATOR ','),
length(group_concat(id ORDER BY campos_ordenacao DESC SEPARATOR ','))
- locate(",",group_concat(id ORDER BY campos_ordenacao DESC SEPARATOR ','))
) as ids,
count(*) as c
-- Tabela a eliminar duplicados
FROM teste_dup
-- campos a usar para identificar duplicados
group by test_campo1, test_campo2, teste_campoN
having count(*) > 1 -- é duplicado
);
-- aumenta o limite desta variável de sistema para o máx
SET SESSION group_concat_max_len=4294967295;
-- envia os ids todos a eliminar para um ficheiro
select group_concat(ids SEPARATOR ',') from to_delete INTO OUTFILE 'sql.dat';
DROP temporary table if exists del3;
create temporary table del3 as (select CAST(1 as signed) as ix LIMIT 0);
-- insere os ids a eliminar numa tabela temporaria a partir do ficheiro
load data infile 'sql.dat' INTO TABLE del3
LINES TERMINATED BY ',';
alter table del3 add index(ix);
-- elimina os ids seleccionados
DELETE teste_dup -- tabela
from teste_dup -- tabela
join del3 on id=ix;
COMMIT;
you can remove all the duplicate entries by using this simple query.
that will select all the duplicate records and remove them.
DELETE i1
FROM TABLE i1
LEFT JOIN TABLE i2
ON i1.id = i2.id
AND i1.colo = i2.customer_invoice_id
AND i1.id < i2.id
WHERE i2.customer_invoice_id IS NOT NULL