Bulk inserting 3 text files each containing 1 lac records into test1 table.
Each of the 3 files has companycode and folio. If compcode and folio already exist in the test1 table, I have to update the table with that particular record from text file else insert it.
But my query is taking lot of time. test1 table has 70 columns
Mmy logic:
import data in dummy table
compare each row of dummy with test1 table
if exists ( select * from #dummy , test1 where condition )
begin
update test1
set col = (#dummy.col)..
inner join #dummy on (condition)
end
else insert
Since the records are in lacs more than 30 min are taken..how can I improve the query?
I assume you're using BULK INSERT to insert data in to temporary table or you could import it using Import Wizard in SQL Server. After that you could use below queries.
Even though you've if(exist) it will update only rows that exists in both the table. So remove if else and write queries directly as shown below:
update test1
set col = (#dummy.col)..
from test1
inner join #dummy on (test1.companycode =#dummy.companycode and test1.folio = #dummy.companycode)
For inserting records that doesn't exits in test1, you could use left join as shown below:
Insert into test1
select column names
from
#dummy left join test1
on test1.companycode =#dummy.companycode and test1.folio = #dummy.companycode
where test1.companycode is null
and test1.folio is null
Related
I have a table A with records with 1 column only that are random alphanumerical characters. That table has say 10 rows.
I have another table B with 10 rows also that I want to take a value from a row from table A and apply it to a row in table B.
So basically, take a value from Table A and assign it to a row in table B. Preferably, take the value from table A row 1 and assign it to table B row 1, etc...
I am using SQL Server.
We can take any value from table B to assign to a row in table A. We just can't re-use a value from table B.
Here are the 2 tables in it's simplest form for this example:
CREATE TableA ([Value] NVARCHAR(50))
CREATE TableB ([Value] NVARCHAR(50))
Given the following table structure:
CREATE TABLE #tempA (stringEntry NVARCHAR(50));
INSERT INTO #tempA (stringEntry) VALUES ('abcd'), ('efgh'), ('ijkl');
CREATE TABLE #tempB (stringEntry NVARCHAR(50));
INSERT INTO #tempB (stringEntry) VALUES ('mnop'), ('qrst'), ('uvwx');
You can do the following:
SELECT
ROW_NUMBER() OVER(ORDER BY #tempA.stringEntry) AS RowNumber,
#tempA.stringEntry AS entryA
INTO #tempA2
FROM #tempA;
SELECT
ROW_NUMBER() OVER(ORDER BY #tempB.stringEntry) AS RowNumber,
#tempB.stringEntry AS entryB
INTO #tempB2
FROM #tempB;
UPDATE #tempA
SET #tempA.stringEntry = #tempB2.entryB
FROM #tempA
INNER JOIN #tempA2 ON #tempA.stringEntry = #tempA2.entryA
INNER JOIN #tempB2 ON #tempB2.RowNumber = #tempA2.RowNumber;
This assumes that you have equal number of rows in each table, as you indicated, or are okay with having the "excess" entries in your first table not being updated.
I need to execute insertions for around 10 tables, before inserting I have to check for a condition, condition remains the same for each of the tables, instead of giving that condition within insert query, I wish I could give in if condition (a select query), if satisfied then execute insert statements, is there a way to give if condition in Vertica SQL file ? If condition is not satisfied I dont want to execute any of the insert queries.
If the condition is, for example, that you only insert the data on a Sunday, try this:
a) a test table:
CREATE LOCAL TEMPORARY TABLE input(id,name)
ON COMMIT PRESERVE ROWS AS
SELECT 42,'Arthur Dent'
UNION ALL SELECT 43,'Ford Prefect'
UNION ALL SELECT 44,'Tricia McMillan'
KSAFE 0;
From this table, select with a WHERE condition that tests whether it's sunday -- that's all, see here:
SELECT
*
FROM input
WHERE TRIM(TO_CHAR(CURRENT_DATE,'Day'))='Sunday'
;
id|name
42|Arthur Dent
43|Ford Prefect
44|Tricia McMillan
With a different value for the week day (I'm writing this on a Sunday...), you get this:
SELECT
*
FROM input
WHERE TRIM(TO_CHAR(CURRENT_DATE,'Day'))='Monday'
;
id|name
select succeeded; 0 rows fetched
I use this technique in SQL generating SQL to create a script or an empty file determining on circumstances, and then to call that script (full or empty), implementing a conditional SQL execution that way....
I know it is an old post, but i want to share what i recently solved my problem.
I need to insert in one table or another based on some condition. You first should have a field or value what will be your search condition.
create table tmp1 (
Col1 int null
,Col2 varchar(100) null
)
--Insert values
insert into tmp (Col1,Col2) Values
(1,'Text1')
,(2,'Text2')
--Insert into table001
insert into table001
select
t.field1
,t.field2
,......
from table1 t
inner join tmp t2
on t2.col1 = t.ColX
where 1 = case when t2.Col2 = 'Text1' then 1 else 0 end --Search condition; if 1<>0 then it doesn't do anything; otherwise insert.
--Insert into table002
insert into table002
select
t.field1
,t.field2
,......
from table2 t
inner join tmp t2
on t2.col1 = t.ColX
where 1 = case when t2.Col2 = 'Text2' then 1 else 0 end --Search condition; if 1<>0 then it doesn't do anything; otherwise insert.
Or you can use an UNION/UNION ALL based on this if it is the same working table.
Regards!
How do I update all columns at once without specifying the column name.
Say I have table A
ID NAME AGE
21 MATT 45
Table B
ID NAME AGE
21 SCOTT 24
What I expect
update table A
set A.columns=B.columns
Basically I'm trying to sync two tables.
As far as I know, there are no ways to update the table column values without specifying column name. Your purpose is to make the table A has the same values of table B, then you can delete all rows from the table A, and copy the data to table A.
delete from A;
insert into A select * from B;
commit;
If you have some reasons only to use update statement, and there are lots of columns, then you can generate the update statement by using dictionary.
select 'A.'||column_name||'=B.'||column_name||','
from all_tab_columns
where owner = 'your schema name'
and table_name = 'A';
You execute the query and copy the result and edit.
update A
set <paste the result of previous query here>
;
You can do it in PLSQL:
for src in ( select * from B ) loop
update A set ROW = src where A.id = src.id;
end loop;
or for insert
for src in ( select * from B ) loop
insert into A values src;
end loop;
I have a scenario whereby I have a table of data I wish to import into my schema. The key field on the import t\ble AND my target table is a field called MEMBERNO
Before importing, I check if any of the rows in the import table have a MEMBERNO that currently exists within the target table. If so, I wish to amend the MEMBERNO on the import table to the next available value before starting the import (a simple INSERT INTO targetTable SELECT * FROM importTable)
The next available MEMBERNO is currently identified by adding 1 to SELECT MAX(MEMBERNO) FROM targetTable - can I apply this function to several rows in one SQL statement or will I have to create a cursor to loop through? Ideally, I would like to have one UPDATE statement where the value to set is "moving" but I'm not sure this is possible
So something like...
UPDATE importTable SET MEMBERNO = nextavailablenumber
WHERE MEMBERNO IN (SELECT MEMBERNO FROM targetTable)
where nextAvailableNumber is computed for each UPDATE
You don't even need to do the update, you can do the check and alter the MemberNo all within the insert to the Target table
INSERT TargetTable (MemberNo, Column1, Column2, Column3)
SELECT MemberNo = CASE WHEN t.MemberNoExists IS NULL THEN i.MemberNo
ELSE n.NextMemberNo + ROW_NUMBER() OVER(PARTITION BY t.MemberNoExists ORDER BY i.MemberNo)
END,
i.Column1,
i.Column2,
i.Column3
FROM ImportTable i
OUTER APPLY
( SELECT TOP 1 MemberNoExists = 1
FROM TargetTable t
WHERE t.MemberNo = i.MemberNo
) t
CROSS JOIN
( SELECT NextMemberNo = MAX(MemberNo)
FROM TargetTable
) n;
The subquery t simply generates 1 when the MemberNo already exists, and NULL when it doesn't. If it doesn't exist then the MemberNo from the import table is used.
The subquery n gets the maximum MemberNo in the target table. This is only used where the MemberNo from the import table already exists. Then, so that you don't need a loop it uses ROW_NUMBER to increment this maximum MemberNo by one for each new row to avoid duplicates.
Example on SQL Fiddle (Only with SELECT for demonstration purposes)
If you do still need to do the update you can use a similar query:
WITH Import AS
( SELECT NewMemberNo = n.NextMemberNo + ROW_NUMBER() OVER(ORDER BY i.MemberNo),
i.MemberNo
FROM ImportTable i
CROSS JOIN
( SELECT NextMemberNo = MAX(MemberNo)
FROM TargetTable
) n
WHERE EXISTS
( SELECT 1
FROM TargetTable t
WHERE t.MemberNo = i.MemberNo
)
)
UPDATE Import
SET MemberNo = NewMemberNo;
UPDATE example on SQL Fiddle
I need to write a procedure that will allow me to select x amount of rows and at the same time update those rows so the calling application will know those records are locked and in use. I have a column in the table named "locked". The next time the procedure is called it will only pull the next x amount of records that do not have the "locked" column checked. I have read a little about the OUTPUT method for SQL server, but not sure that is what I want to do.
As you suggested, you can use the OUTPUT clause effectively:
Live demo: https://data.stackexchange.com/stackoverflow/query/8058/so3319842
UPDATE #tbl
SET locked = 1
OUTPUT INSERTED.*
WHERE id IN (
SELECT TOP 1 id
FROM #tbl
WHERE locked = 0
ORDER BY id
)
Also see this article:
http://www.sqlmag.com/article/tsql3/more-top-troubles-using-top-with-insert-update-and-delete.aspx
Vote for Cade Roux's answer, using OUTPUT:
UPDATE #tbl
SET locked = 1
OUTPUT INSERTED.*
WHERE id IN (SELECT TOP 1 id
FROM #tbl
WHERE locked = 0
ORDER BY id)
Previously:
This is one of the few times I can think of using a temp table:
ALTER PROCEDURE temp_table_test
AS
BEGIN
SELECT TOP 5000 *
INTO #temp_test
FROM your_table
WHERE locked != 1
ORDER BY ?
UPDATE your_table
SET locked = 1
WHERE id IN (SELECT id FROM #temp_test)
SELECT *
FROM #temp_test
IF EXISTS (SELECT NULL
FROM tempdb.dbo.sysobjects
WHERE ID = OBJECT_ID(N'tempdb..#temp_test'))
BEGIN
DROP TABLE #temp_test
END
END
This:
Fetches the rows you want, stuffs them into a local temp table
Uses the temp table to update the rows to be "locked"
SELECTs from the temp table to give you your resultset output
Drops the temp table because they live for the session