I have an identity field in one of my tables, specified as
User_Key BIGINT PRIMARY KEY IDENTITY(-1,1) NOT NULL
The seed is set to -1 to account for an unknown value (universal across all my tables) and the table in question is appended to rather than being dropped and recreated each time, but the process is set up to recreate the table if it is accidentally dropped.
Since RedShift doesn't support if statements (ideally, I'd only insert the unknown value if the table didn't already exist), my workaround (using EXCEPT) is shown below.
CREATE TABLE IF NOT EXISTS TBL
(User_Key BIGINT PRIMARY KEY IDENTITY(-1,1) NOT NULL
,Col1 VARCHAR(255) NOT NULL
,Col2 INT
,COL3 VARCHAR(255) NOT NULL
INSERT INTO TBL
(Col1
,Col2
,Col3)
SELECT
'Unknown'
NULL
'Etc'
EXCEPT
SELECT
Col1
,Col2
,Col3
FROM TBL
With the EXCEPT clause, the value in the User_Key field varies (but never is -1 as expected). Without the EXCEPT clause, the User_Key field works exactly as expected without fail. Testing this process has involved dropping and recreating this table with each iteration.
I've done the usual sweep of process documentation/forums/etc., but can't see this being reported elsewhere. Is this a known issue that can be fixed by specifying additional parameters? I have a workaround for this (enclosing the create statement and unknown value prior to the rest of the procedure in my application, but I'd rather keep the script in as few parts as possible.
Are you sure the remaining columns have the correct values? I suspect the error might be along those lines.
You can do this using not exists:
INSERT INTO TBL (Col1, Col2, Col3)
SELECT x.*
FROM (SELECT 'Unknown' as col1, NULL as col2, 'Etc' as col3) x
WHERE NOT EXISTS (SELECT 1 FROM tbl WHERE tbl.user_key = -1)
Related
In SQL (I'm using postgres, but am open to other variations), is it possible to update a value based on a row location and a column name when the table doesn't have unique rows or keys? ...without adding a column that contains unique values?
For example, consider the table:
col1
col2
col3
1
1
1
1
1
1
1
1
1
I would like to update the table based on the row number or numbers. For example, change the values of rows 1 and 3, col2 to 5 like so:
col1
col2
col3
1
5
1
1
1
1
1
5
1
I can start with the example table:
CREATE TABLE test_table (col1 int, col2 int, col3 int);
INSERT INTO test_table (col1, col2, col3) values(1,1,1);
INSERT INTO test_table (col1, col2, col3) values(1,1,1);
INSERT INTO test_table (col1, col2, col3) values(1,1,1);
Now, I could add an additional column, say "id" and simply:
UPDATE test_table SET col2 = 5 WHERE id = 1
UPDATE test_table SET col2 = 5 WHERE id = 3
But can this be done just based on row number?
I can select based on row number using something like:
SELECT * FROM (
SELECT *, ROW_NUMBER() OVER() FROM test_table
) as sub
WHERE row_number BETWEEN 1 AND 2
But this doesn't seem to play well with the update function (at least in postgres). Likewise, I have tried using some subsets or common table expressions, but again, I'm running into difficulties with the UPDATE aspect. How can I perform something that accomplishes something like this pseudo code?: UPDATE <my table> SET <col name> = <new value> WHERE row_number = 1 or 3, or... This is trivial other languages like R or python (e.g., using pandas's .iloc function). It would be interesting to know how to do this in SQL.
Edit: in my table example, I should have specified the column types to something like int.
This is one of the many instances where you should embrace the lesser evil that is Surrogate Keys. Whichever table has a primary key of (col1,col2,col3) should have an additional key created by the system, such as an identity or GUID.
You don't specify the data type of (col1,col2,col3), but if for some reason you're allergic to surrogate keys you can embrace the slightly greater evil of a "combined key", where instead of a database-created value your unique key field is derived from some other fields. (In this instance, it'd be something like CONCAT(col1, '-', col2, '-', col3) ).
Should neither of the above be practical, you will be left with the greatest evil of having to manually specify all three columns each time you query a record. Which means that any other object or table which references this one will need to have not one but three distinct fields to identify which record you're talking about.
Ideally, btw, you would have some business key in the actual data which you can guarantee by design will be unique, never-changing, and never-blank. (Or at least changing so infrequently that the db can handle cascade updates reasonably well.)
You may wind up using a surrogate key for performance in such a case anyway, but that's an implementation detail rather than a data modeling requirement.
I have two tables table1 and test_table1 which have the same schema.
Both tables have rows/data and pk id's starting from 1.
I would like to do:
insert into test_table1 select * from table1;
but this fails due to the pk values from table1 existing in test_table1.
Way around it would be to specify columns and leave the pk column out, but for some reason thats not working either:
e.g.
NOTE - no pk columns in query below
insert into test_table1 (col1, col2,..., coln) select col1,col2,...,coln from table1;
returns
ERROR: duplicate key value violates unique constraint "test_table1_pkey"
DETAIL: Key (id)=(1) already exists.
I know this works in MySql, is this just due to Postgresql? Anyway around it?
EDIT:
Both tables have primary keys and sequence set.
Since it wasn't clear - tables don't have the same data.
I would just like to add rows from table1 to test_table1.
For answers telling me to exclude primary key from the query - I did as I said before.
Just remove pk column from columns of query
insert into test_table1 (col2,..., coln) select col2,...,coln from table1;
If it still fails maybe you have not sequence on pk columns.
Create sequence on already existing pk column
create sequence test_table1_seq;
ALTER TABLE test_table1
ALTER COLUMN col1 SET DEFAULT nextval('test_table1_seq'::regclass);
And update sequence value to current
SELECT setval('test_table1_seq', (SELECT MAX(col1) FROM test_table1));
This post helped me solve my problem, not sure what went wrong:
How to fix PostgreSQL error "duplicate key violates unique constraint"
If you get this message when trying to insert data into a PostgreSQL database:
ERROR: duplicate key violates unique constraint
That likely means that the primary key sequence in the table you're working with has somehow become out of sync, likely because of a mass import process (or something along those lines). Call it a "bug by design", but it seems that you have to manually reset the a primary key index after restoring from a dump file. At any rate, to see if your values are out of sync, run these two commands:
SELECT MAX(the_primary_key) FROM the_table;
SELECT nextval('the_primary_key_sequence');
If the first value is higher than the second value, your sequence is out of sync. Back up your PG database (just in case), then run thisL
SELECT setval('the_primary_key_sequence', (SELECT MAX(the_primary_key) FROM the_table)+1);
That will set the sequence to the next available value that's higher than any existing primary key in the sequence.
You rather would want to do a UPDATE JOIN like
UPDATE test_table1 AS v
SET col1 = s.col1,
col2 = s.col2,
col3 = s.col3,
.....
colN = s.colN
FROM table1 AS s
WHERE v.id = s.id;
what you want to do is an upsert.
with upsert as (
update test_table1 tt
set col1 = t.col1,
col2 = t.col2,
col3 = t.col3
from table1 t
where t.id = tt.id
returning *
)
insert into test_table1(id, col1, col2, col3)
select id, col1,col2,col3
from table1
where not exists (select * from upsert)
Table named T1 with following values
Col1 Col2 Col3
Rs1 S S2
Rs2 SX S3
Rs3 S S2
From a csv, I need to insert some values into the table, having values Rs4, SX and S3 respectively to each column.
I need to apply a check with following constraints.
One S3 can belong to only one SX, but S3 and SX as pair can belong can belong to multiple columns1's values.
What will be the oracle query for this? And if the above condition is true then I need to run an insertion query which is prepared. How can it validated?
PS: we can't create another table.
Had to do a little discovery after I was informed that I totally missed the ORACLE tag. Knowing what you do not know is very important to me. This post should be sufficiently different.
THE BASIC PROBLEM WITH ORACLE'S CHECK
A check constraint can NOT be defined on a SQL View. The check constraint defined on a table must refer to only columns in that
table. It can not refer to columns in other tables.
A check constraint can NOT include a SQL Subquery.
A check constraint can be defined in either a SQL CREATE TABLE statement or a SQL ALTER TABLE statement.
REVISITING THE PROBLEM
We know that (Col2,Col3)| #(Col2,COl3) >= 1.
We know that {Col1}∩(Col2,Col3)
However, the #Cardinality of Col1? Can it be more than 1?
Clearly, the business requirements are not fully explained.
REVISITING THE SOLUTIONS
Adding Objects to the database.
While adding additional tables has been voted down, is it possible to add an ID column? Assuming Col1 is NOT unique to the subsets of (Col2,COl3), then you can add a true ID Column that fulfills the need for normalization while providing true indexing power in your query.
Col1 Col2 Col3 Col4
Rs1 S S2 1
Rs2 SX S3 2
Rs3 S S2 1
To be clear, Col4 would still be an ID since the values of Col2, Col3 are determined by Col4. (Col2,Col3) 1:1 Col4.
CHECKS
Multiple CHECK constraints, each with a simple condition enforcing a
single business rule, are preferable to a single CHECK constraint with
a complicated condition enforcing multiple business rules ORACLE - Constraint
A single column can have multiple CHECK constraints that reference the
column in its definition. There is no limit to the number of CHECK
constraints that you can define on a column. ORACLE - Data Integrity
If you can add a column...by the love of monkeys, please do...not only will it make your life much easier, but you can also make QUERYING the table very efficient. However, for the rest of this post, I will assume you cannot add columns:
RESTATING THE PROBLEM IN CONSTRAINTS
Col2 may not appear with a different Col3. Vice Versa.
(Col2,Col3) may have multiple Co1...what is the possible Cardinality of Col1? Can it be repetitive? I read no.
WRITING OUT THE THEORY ON CHECKS
IF Col1 truly is unique in {(col2,col3)}, then the following already works:
ALTER TABLE EXAMPLE3
ADD CONSTRAINT ch_example3_3way UNIQUE (C, D, X) --only works if these valus never repeat.
The other main constraint #(Col2,Col3) > 1 simply cannot work unless you knew what value was being entered so as to enforce a real SARG. Any Col1 = Col1 or Col1 IN Col1 is the same thing as writing 1 = 1.
ON TRIGGERS
As tempting as the idea sounds, a quick glance through ORACLE lane left me warning against the use. Some reasons from ORACLE:
ORACLE - USING TRIGGERS
Do not create recursive triggers.
For example, if you create an AFTER UPDATE statement trigger on the
employees table, and the trigger itself issues an UPDATE statement on
the employees table, the trigger fires recursively until it runs out
of memory.
Use triggers on DATABASE judiciously. They are executed for every user every time the event occurs on which the trigger is created
Other problems include: TOADWORLD - ORACLE WIKI
Not Compiled -STORED PROCs can reuse a cached plan
No SELECT Trigger Support
Complete Trigger Failure
Disabled Triggers
No Version Control
Update OF COLUMN
No Support of SYS Table Triggers
Mutating Triggers
Hidden Behavior
Still, there are advantages of TRIGGERs, and you could still enforce data integrity by using a query where the first result of
SELECT Col2, Col3 FROM T1 WHERE ROWNUM = 1
Is compared to the inserted value *new.*Col2, *new.*Col3, but this would require the trigger to fire EVERY TIME a row was inserted...recompiled and everything,..I STRONGLY URGE AVOIDANCE.
STORED PROCS
Whatever you may think of STORED PROCEDURES, I suggest you consider them again. Everything from Functions, DML, DDL, database management, RECURSIVE LOGIC, sp_executesql, and beyond can be accomplished through a PROC.
Easily managed, provides encapsulation from accidental or malicious disabling or mutilization of coding.
PROCs are compiled once and can be reuse query plan caches, providing improved performances.
Provides superior portability, can be embedded into TRIGGERS, ORM framework, applications and beyond.
Can literally automate almost any function in a database including ETL, Resource management, security, and discovery. Views are commonly run through stored Procs.
THE UNIQUE ADVANTAGE OF ORACLE
Perhaps forgotten, consider that this is ORACLE which allows you to suspend CONSTRAINTS by inserting in the CONSTRAINT DEFFERABLE. From an ETL specialist perspective, this is essentially making a staging table out of your only table...which is pretty sweet in your predicament of having limited DDL rights.
CONCLUDING COMMENTS
There are a few efficient methods to delete duplicates in your data.
DELETE FROM T1
WHERE rowid NOT IN
(SELECT MAX(rowid)
FROM T1
GROUP BY Col1, Col2, Col3);
NOTE: rowid is the physical location of the row, while rownum represents the logical position in the query.
Lastly, my last attempt at rowid. Unfortunately, time is running late, and the free COMPILER from ORACLE is unhelpful. But I think the idea is what is important.
CREATE TABLE Example3 (MUT VARCHAR(50), D VARCHAR(50), X VARCHAR(50) );
INSERT INTO Example3 (MUT, D, X) VALUES('MUT', 'T', 'M' );
INSERT INTO Example3 (MUT, D, X) VALUES('MUT', 'T', 'P' );
INSERT INTO Example3 (MUT, D, X) VALUES('MUT', 'X', 'LP');
INSERT INTO Example3 (MUT, D, X) VALUES('MUT', 'X', 'Z');
INSERT INTO Example3 (MUT, D, X) VALUES('MUT', 'Y', 'POP');
SELECT C.D, B.X, B.rowid
FROM EXAMPLE3 A
LEFT OUTER JOIN (
SELECT DISTINCT X, C.rowid
FROM EXAMPLE3) B ON B.rowid = A.rowid
LEFT OUTER JOIN (
SELECT DISTINCT D, MAX(rowid) AS [rowid]
FROM Example3) C ON C.rowid = B.rowid
Finally, I'm able to resolve the question with a some select queries and few if conditions being applied. I have done this in a stored procedure.
SELECT count(col3)
INTO V_exist_value
FROM T3
WHERE col3's value = Variable_col3
AND col1's value <> Variable_col1
AND col2's value = Variable_col2;
IF (V_exist_value >= 1) THEN
INSERT INTO T3 (col1, col2, col3)
VALUES (Variable_col1, Variable_col2, Variable_col3);
ELSE
SELECT count(col3)
INTO V_exist_value1
FROM T3
WHERE col3's value = Variable_col3;
IF (V_exist_value1 = 0) THEN
INSERT INTO T3 (col1, col2, col3)
VALUES (Variable_col1, Variable_col2, Variable_col3);
ELSE
RAISE Exception_col3_value_exists;
END IF;
END IF;
If you don't want to use a trigger then you must normalize your tables.
Create a second table - say T1_PAIRS - that will store all permitted pairs of (col2, col3).
Create an unique constraint on col2 column in table T1_PAIRS - this constraint allows only for unique values of COL2 - for example no more than one S3 value can be used in all pairs ==> this enforces the rule: "One S3 can belong to only one SX"
Create a primary key on ( col2, col3 ) columns in this table T1_PAIRS.
Create a foreign key constraint on ( col2, col3 ) in T1 table that references the primary key of T1_PAIRS table.
In the end create an unique constraint on (col1, col2, col3) columnt to enforce a rule ==> S3 and SX as pair can belong can belong to multiple columns1's values (but no more than one value of column1)"
An example:
CREATE TABLE T1_PAIRS (
Col2 varchar2(10), Col3 varchar2(10),
CONSTRAINT T1_PAIRS_PK PRIMARY KEY( col2, col3 ),
CONSTRAINT T1_col2_UQ UNIQUE( col2 )
);
INSERT ALL
INTO T1_PAIRS( col2, col3 ) VALUES( 'S', 'S2' )
INTO T1_PAIRS( col2, col3 ) VALUES( 'SX', 'S3' )
SELECT 1 FROM dual;
ALTER TABLE T1
ADD CONSTRAINT col2_col3_pair_fk
FOREIGN KEY ( col2, col3 ) REFERENCES T1_pairs( col2, col3 );
ALTER TABLE T1
ADD CONSTRAINT pair_can_belong_to_multi_col1 UNIQUE( col1, col2, col3 );
I want to define a column in my table with following requirements:
The column should be insertable. If the value is provided in the INSERT statement, then it should be inserted.
If the column is not referenced in the INSERT statement, then it should be set to the sum of two other columns.
Because of the first requirement, I cannot user computed columns, since they are not insertable. Because of the second, I cannot use DEFAULT, because it doesn't allow referencing other columns in the definition. What other options do I have?
BTW, the column should be NOT NULL.
Here you go, I'm demonstrating this with an example schema since you've not provided your real table/column names.
Table:
CREATE TABLE test
(
id INT NOT NULL PRIMARY KEY IDENTITY, --made up key
col1 INT, --first column to add, wasn't sure if this was nullable or not
col2 INT, --second column to add, wasn't sure if this was nullable or not
col3 INT NOT NULL --this is the column to optionally insert into
)
Here is the trigger definition:
CREATE TRIGGER demo
ON test
INSTEAD OF INSERT
AS
INSERT INTO test (col1,col2,col3)
SELECT inserted.col1,
inserted.col2,
CASE
WHEN inserted.col3 IS NULL THEN COALESCE(inserted.col1, 0) + COALESCE(inserted.col2, 0)
ELSE inserted.col3
END
FROM inserted
Basically it replaces any insert statement done on the table with the one in the trigger, so I check using the inserted temporary table to see if the value that is trying to be inserted into our non-nullable optional column, col3, is NULL. If it is, I replace it with the addition of col1 and col2 (I'm coalescing with zero as you didn't mention if the two source columns are nullable or not).
You can then run insert statements which either include it or not, despite the fact col3 is not nullable:
INSERT INTO test(col1,col2)
SELECT 12, 31
GO
INSERT INTO test(col1, col2, col3)
SELECT 1, 2, 89
GO
Results are:
ID COL1 COL2 COL3
------------------
1 12 31 43
2 1 2 89
If the trigger wasn't there, you could have got an error trying to run that first insert statement, telling you it couldn't insert NULL into col3.
Notice also that the second insert statement that specifies a value has not been replaced by the addition, as requested.
Here's a working SQL Fiddle.
Im trying to copy the contents from a column in one table to another and at the same time want to populate the primary key column with an incrementing number for each row created:
I have tried doing the following:
INSERT INTO Table1 (col1, col2) VALUES((SELECT col1 FROM table2), (SELECT NEXTVAL FOR col2_SEQ FROM sysibm.sysdummy1));
but get the following error:
DB21034E The command was processed as an SQL statement because it was not a
valid Command Line Processor command. During SQL processing it returned:
SQL0348N "NEXTVAL FOR col2_SEQ" cannot be specified in this
context. SQLSTATE=428F
It seems that i cant use the sequence value in this way, is there any other way I can achieve what I'm trying to do? I just need col2 in table1 to be populated with a unique BIGINT for each new entry from col1 from table2
If you're on Linux/Unix/Windows (and probably for others), I think you just want NEXT VALUE FOR sequence. You don't need the extra select from sysdummy in this context.
INSERT INTO table1 (col1, col2)
SELECT col1, NEXT VALUE FOR col2_SEQ
FROM table2
There are 3 methods in which unique values can be generated in DB2.
GENERATE_UNIQUE function
IDENTITY column
SEQUENCE object
Assuming col2_SEQ is created similar to below statement:
CREATE SEQUENCE col2_SEQ
AS INTEGER
START WITH 1
INCREMENT BY 1
NO MINVALUE
NO MAXVALUE
NO CYCLE
ORDER
The insert statement can be written as follows:
INSERT INTO Table1 (col1, col2)
VALUES ((SELECT col1 FROM table2),
NEXT VALUE FOR col2_SEQ)
More information, on each of the three methods mentioned above, can be found here
There is also alternative syntax now, which worked for me in DB2 10.x
INSERT INTO table1 (col1, col2)
SELECT col1, schema.seq_name.nextval
FROM table2;
Maybe you should specify the columns as:
col2 smallint not null
generated by default as identity (start with 1, increment by 1)
and insert into table1 select col1, default from table2