Let's say for instance:
I have two tables: old_data and new_data.
Both old_data and new_data have one column called this_is_col.
Both old_data and new_data have various (hundreds) of rows of dates (2010-02-06, 2010-01-09, 2007-06-02, etc.). Both tables don't necessarily have the same dates, but they both have the same format.
The fields of both tables are various integers.
My task:
Copy the fields from old_data to new_data.
If a date exists in both tables, the field in new_data will be replaced.
If the date doesn't exist in new_data, then the correct row will be added and the field will be copied over.
Here is how far I've gotten:
Create a temporary column:
ALTER TABLE `new_data` ADD `tempColumn` TEXT NULL;
Copy over data from old_data:
INSERT INTO `new_data` (`tempColumn`) SELECT `this_is_col` FROM `old_data`;
Combine temporary column and new_data . this_is_col. (I haven't really figured this step out since I haven't gotten this far).
MERGE? `tempColumn` `this_is_col`;
Delete temporary table
ALTER TABLE `new_data` DROP `tempColumn`;
Upon performing the second action (transferring the data over to the temporary column) I get this error:
#1062 - Duplicate entry '0000-00-00' for key 1
And now I'm stuck. Any help would be appreciated. I'm using MySQL and phpMyAdmin to test the SQL commands.
Assuming your dates are indexed as unique keys:
INSERT INTO newtable
SELECT *
FROM oldtable
ON DUPLICATE KEY column1=oldcolumn1, ...
you want INSERT ... ON DUPLICATE KEY UPDATE. your solution already satisfies steps 1 and 3 of your task, ON DUPLICATE KEY UPDATE will take care of step 2.
If you'd rather delete the row first, instead of updating: REPLACE
It'd be just one line too, so: REPLACE data SELECT, you wouldn't have to do the weirdness with adding a text column.
How about just doing an UPDATE and INSERT?
UPDATE new_data SET col=col
FROM new_data a join old_data b on a.this_is_col = b.this_is_col
Then
INSERT INTO new_data (cols) SELECT cols
FROM old_data WHERE this_is_col NOT IN (SELECT this_is_col FROM new_data)
Unless I misunderstood...
Related
I found the similar question and solution for the SQL server. I want to replace all my null values with zero or empty strings. I can not use the update statement because my table has 255 columns and using the update for all columns will consume lots of time.
Can anyone suggest to me, how to update all the null values from all columns at once in PostgreSQL?
If you want to replace the data on the fly while selecting the rows you need:
SELECT COALESCE(maybe_null_column, 0)
If you want the change to be saved on the table you need to use an UPDATE. If you have a lot of rows you can use a tool like pg-batch
You can also create a new table and then swap the old one and the new one:
# Create new table with updated values
CREATE TABLE new_table AS
SELECT COALESCE(maybe_null_column, 0), COALESCE(maybe_null_column2, '')
FROM my_table;
# Swap table
ALTER TABLE my_table RENAME TO obsolete_table;
ALTER TABLE new_table RENAME TO my_table;
I have a table that I'm trying to append unique values to. Every month I get list of user logins to import into this table. I would like to keep all the original values and just append the new and unique values onto the existing table. Both the table and the flatfile have a single column, with unique values, built like this:
_____
login
abcde001
abcde002
...
_____
I'm bulk ingesting the flat file into a temp table, with this:
IF OBJECT_ID('tempdb..#FLAT_FILE_TBL') IS NOT NULL
DROP TABLE #FLAT_FILE_TBL
CREATE TABLE #FLAT_FILE_TBL
(
ntlogin2 nvarchar(15)
)
BULK INSERT #FLAT_FILE_TBL
FROM 'C:\ImportFiles\logins_Dec2021.csv'
WITH (FIELDTERMINATOR = ' ');
Is there a join that would give me the table with existing values + new unique values appended? I'd rather not hard code a loop to evaluate it line by line.
Something like (pseudocode):
append unique {login} from temp_tbl into original_tbl
Hopefully it's an easy answer for someone out there.
Thanks!
Poster on Reddit r/sql provided this answer, which I'm pursuing:
Merge statement?
It looks like using a merge statement will do exactly what I want. Thanks for those who already posted replies.
You can check if a record exists using 'EXISTS' clause and insert if it doesn't exist in the target table. You can also use MERGE statement to achieve the same. Depending on what you want to do to the existing records in the target table, you can modify the Merge statement. Here since you only want to insert new records, you need to specify only what you want to do when a new record comes in. Here is an example
MERGE original_tbl T
USING temp_tbl S
ON T.login = S.login
WHEN NOT MATCHED THEN
INSERT (login)
VALUES(S.login)
Another solution would be to left join the target table to the temp table and insert only when the record doesn't exist.
INSERT INTO original_tbl(login)
SELECT S.Login
FROM temp_tbl S
LEFT JOIN original_tbl T
ON S.Login = T.Login
WHERE T.Login IS NULL
I want to merge the IndirectFlights table to the PriceTable.
I do not have IDs entered in the SourceTable (IndirectFlights) and I haven't set a PK for it yet.
The ID column for the PriceTable is an Identity (1,1) column and is also the Primary Key.
Qs1 How do I enter IDs in Source column so that they dont clash with target table (PriceTable) IDs? I was thinking of using a sequence but It potentially could clash in future.
Qs2 Can I choose what columns to merge or must I merge all the columns from the Source table?
Target Table (PriceTable) Columns
IDAirport_ICAO_Code,Airline_ICAO_Code,Departure,Price,RouteStatus,DateRowModified
Source Table (IndirectFlights) Columns
IDAirport_ICAO_Code,Destination,Airline,Airline_ICAO_Code,RouteStatus,Connecting Airport
Edit: I have just run the following Union All statement as an alternative to using Merge.
Select ID,Airport_ICAO_Code,Airline_ICAO_Code,RouteStatus
From RoughworkPriceTable
Union All
Select ID,Airport_ICAO_Code,Airline_ICAO_Code,RouteStatus
From RoughworkIndirectFlights;
The code worked but i noticed that the ID column accepted the Null values from IndirectFlights.ID eventhough I have the ID columns set to Not Null.
Can anyone explain this.
Also can someone expalin how I could create a new permanent table from this Union All statement.
You can create a new table with something like
Select * into newTmpTable from (
Select ID,Airport_ICAO_Code,Airline_ICAO_Code,RouteStatus From RoughworkPriceTable
Union All
Select ID,Airport_ICAO_Code,Airline_ICAO_Code,RouteStatus From RoughworkIndirectFlights)
as mergedData;
I have a "source data" table with columns A,B,C,D,E,F
I use this table to populate a live table by using
INSERT INTO LIVETABLE
SELECT *
FROM SOURCEDATATABLE
Recently, a new column (C1) was added to the LIVETABLE
All I want to do is insert a C1 column into my SOURCEDATATABLE between C and D so that it now is A,B,C,C1,D,E,F. There is no need to populate with data as the LIVETABLE accepts NULLs
Is there any easy solution?
EDIT - MISSING INFORMATION
This table is one of many and my approach to using the INSERT INTO is due to having to use dynamic SQL (for various other reasons) so I cannot specify the column names
There is a reason for the Mantra "I shall not use SELECT *" and you ran straight into it. Add the column to SOURCEDATATABLE (if necessary) and enumerate the columns in the SELECT clause using NULL for the new one.
The only way to insert a new column between two columns is to create a new table with the columns in the order you want, copy the data into it, drop the old table and rename the new table with the old name. Make sure you remove primary key identities to maintain the identity column.
I have a sql table that I am trying to add a column from another table to. Only when I execute the alter table query it does not pull the values out of the table to match the column where I am trying to make the connection.
For example I have column A from table 1 and column A from table 2, they are supposed to coincide. ColumnATable1 being an identification number and ColumnATable2 being the description.
I tried this but got an error...
alter table dbo.CommittedTbl
add V_VendorName nvarchar(200)
where v_venkey = v_vendorno
It tells me that I have incorrect syntax... Anyone know how to accomplish this?
alter table dbo.CommittedTbl
add V_VendorName nvarchar(200);
go
update c
set c.V_VendorName = a.V_VendorName
from CommittedTbl c
join TableA a
on c.v_venkey = a.v_vendorno;
go
I'm just guessing at your structure here.
alter table 2 add column A <some_type>;
update table2 set column A = (select column_A from table2 where v_venkey = v_vendorno);
Your names for tables and columns are a bit confusing but I think that should do it.
There is no WHERE clause for an ALTER TABLE statement. You will need to add the column (your first two lines), and then insert rows based upon a relationship you define between the two tables.
ALTER TABLE syntax:
http://msdn.microsoft.com/en-us/library/ms190273%28v=sql.90%29.aspx
There are several languages within SQL:
DDL: Data Definition Language - this defines the schema (the structure of tables, columns, data types) - adding a column to a table affects the table definitions and all rows will have that new column (not just some rows according to a criteria)
DML: Data Manipulation Language - this affects data within a table, and inserting, updating or other changes fall into this and you can update some data according to criteria (and this is where a WHERE clause would come in)
ALTER is a DDL statement, while INSERT and UPDATE are DML statements.
The two cannot really be mixed as you are doing.
You should ALTER your table to add the column, then INSERT or UPDATE the column to include appropriate data.
Is it possible that you want a JOIN query instead? If you want to join two tables or parts of two tables you should use JOIN.
have a look at this for a start if you need to know more LINK
hope that helps!