SQL Server 2008 R2: Multiple UNION on different databases table's - sql

I have two databases namely db1 and db2.
The database db1 contain 1 table namely test1 and database db2 contain two tables namely test2 and test3.
Here is the following table's with some demo records.
Database: db1
Table: test1
Create table test1
(
ID int,
hrs int,
dates date,
st_date date,
ed_date date
);
insert into test1 values(1,10,'2000-01-01','1900-01-01','2016-01-01');
insert into test1 values(2,20,'2000-02-01','1900-01-01','2016-01-01');
insert into test1 values(3,30,'2000-03-01','1900-01-01','2016-01-01');
insert into test1 values(4,40,'2000-04-01','1900-01-01','2016-01-01');
Database: db2
Table: test2
create table test2
(
ID int,
ID2 int
);
insert into test2 values(1,11);
insert into test2 values(2,22);
insert into test2 values(3,33);
insert into test2 values(4,44);
Database: db2
Table: test3
create table test3
(
ID int,
date date
);
insert into test3 values(1,'2000-01-01');
insert into test3 values(2,'2000-02-02');
insert into test3 values(3,'2000-03-03');
insert into test3 values(4,'2000-04-04');
Note: Now i am executing following query by union all three table's but getting performance issue. There are also test2,test3 tables are present in the db1
select nm,sum(x.avghrs)
from
(
select t1.nm, sum(hrs) as avghrs
from db2.test3 t3,
db2.test2 t2,
db1.test1 t1
where t1.id = t2.id
t3.id = t2.id
group by t1.nm
union
select t1.nm, sum(hrs) as avghrs
from db1.test3 t3,
db1.test2 t2,
db1.test1 t1
where t1.id = t2.id
t3.id = t2.id
group by t1.nm
union
select nm, 0 as avghrs
from test1
where dates between st_date and ed_date
) x
group by nm;
Please tell me if there is any need of modification?

I think the problem is related to JOINs between columns from tables residing in different databases. You can try the following:
1) mirror the tables in a single database (replicate schema and data)
2) apply the appropriate indexes (at least to contain ids used in JOINs)
3) Change your query to SELECT only from mirrored tables
Also, do you need to perform distinct for your unioned results? If not, UNION should be replaced with UNION ALL to avoid the implicit DISTINCT.

UNION ALL will perform better than UNION when you're not concerned about eliminating duplicate records because you're avoiding an expensive distinct sort operation.

Related

Find the difference of values between 2 columns after joining 2 tables on ms sql server

I have 2 tables in MS SQL Server 2019 - test1 and test2. Below are the table creation and insert statements for the 2 tables :
create table test2 (id nvarchar(10) , code nvarchar(5) , all_names nvarchar(80))
create table test3 (code nvarchar(5), name1 nvarchar(18) )
insert into test2 values ('A01', '50493', '12A2S0403-Buffalo;13A1T0101-Boston;13A2C0304-Miami')
insert into test2 values ('A02', '31278', '12A1S0205-Detroit')
insert into test2 values ('A03', '49218', '12A2S0403-Buffalo;12A1M0208-Manhattan')
insert into test3 values ('50493', 'T0101-Boston')
insert into test3 values ('49218', 'S0403-Buffalo')
insert into test3 values ('31278', 'S0205-Detroit')
I can join the 2 tables on the code column. Task is to find difference of test2.all_names and test3.name1. For example 'A01' should display the result as '12A2S0403-Buffalo;13A2C0304-Miami'.
A02 should not come as output.
The output should be :
Id | Diff_of_name
----------------------------------------
A01 | 12A2S0403-Buffalo;13A2C0304-Miami
A03 | 12A1M0208-Manhattan
Here's one possible solution, first using openjson to split your source string into rows, then using exists to check for matching values in table test3 and finally string_agg to provide the final result:
select Id, String_Agg(j.[value], ';') within group (order by j.seq) Diff_Of_Name
from test2 t2
cross apply (
select j.[value], Convert(tinyint,j.[key]) Seq
from OpenJson(Concat('["',replace(all_names,';', '","'),'"]')) j
where not exists (
select * from test3 t3
where t3.code = t2.code and j.[value] like Concat('%',t3.name1,'%')
)
)j
group by t2.Id;
Demo Fiddle
I don't like the need to normalize. However, if one must normalize, STRING_SPLIT is handy.
When done with the real work, STRING_AGG can de-normalize the data.
WITH normalized as ( -- normalize all_names in test2 to column name1
SELECT t2.id, t2.code, t2.all_names, n.value as [name1]
FROM test2 t2
OUTER APPLY STRING_SPLIT(t2.all_names, ';') n
) select * from normalized;
WITH normalized as ( -- normalize all_names in test2 to column name1
SELECT t2.id, t2.code, t2.all_names, n.value as [name1]
FROM test2 t2
OUTER APPLY STRING_SPLIT(t2.all_names, ';') n
), differenced as ( -- exclude name1 values listed in test3, ignoring leading characters
SELECT n.*
FROM normalized n
WHERE NOT EXISTS(SELECT * FROM test3 t3 WHERE t3.code = n.code AND n.name1 LIKE '%' + t3.name1)
) -- denormalize
SELECT id, STRING_AGG(name1, ';') as [Diff_of_name]
FROM differenced
group by id
order by id
id Diff_of_name
---------- ---------------------------------
A01 12A2S0403-Buffalo;13A2C0304-Miami
A03 12A1M0208-Manhattan

UNION two SELECT queries but result set is smaller than one of them

In a SQL Server statement there is
SELECT id, book, acnt, prod, category from Table1 <where clause...>
UNION
SELECT id, book, acnt, prod, category from Table2 <where clause...>
The first query returned 131,972 lines of data; the 2nd one, 147,692 lines. I didn't notice there is any commonly shared line of data from these two tables, so I expect the result set after UNION should be the same as the sum of 131972 + 147692 = 279,384.
However the result set after UNION is 133,857. Even though they might have overlapped lines that I accidently missed, the result should be at least the same as the larger result set of those two. I can't figure how the number 133,857 came from.
Is my understanding about SQL UNION correct? I use SQL server in this case.
To expand comment given under the question, which I think states what you already know:
UNION takes care of duplicates also within one table as well.
Just take a look at a example:
SETUP:
create table tbl1 (col1 int, col2 int);
insert into tbl1 values
(1,2),
(3,4);
create table tbl2 (col1 int, col2 int);
insert into tbl1 values
(1,2),
(1,2),
(1,2),
(3,4);
Query
select * from tbl1
union
select * from tbl2;
will produce output
col1 | col2
-----|------
1 | 2
3 | 4
DB fiddle

In Clause With Subquery In Oracle

I have two tables say table1 and table2 with below details
create table test1(id number, name varchar2(20));
insert into test1 values(11,'micro');
insert into test1 values(22,'soft');
create table test2(id number, name varchar2(20));
insert into test2 values(77,'micro,soft');
1) if I use below query I am getting no rows selectd
select * from test1 t1 where t1.name in ( select ''''||replace(t2.name,',',''',''')||'''' from test2;
2) if I fire subquery alone output I am getting is : 'micro','soft'
select ''''||replace(t2.name,',',''',''')||'''' from test2;
but if I fire query(1) I need the result
id name
------------
11 micro
22 soft
can some one please help me to get the same result with query (1).
You have a very strange data layout and should probably change it.
You can do what you want with a join or with a correlated subquery using like:
select *
from test1 t1
where exists (select 1
from test2 t2
where ','||t2.name||',' like '%,'||t1.name||',%'
);
You version doesn't work because the expression:
where x in ('a,b,c')
tests where x is equal to the string value 'a,b,c', not whether it is equal to one of the three values.
Use REGEXP_SUBSTR function to get this done.
SELECT *
FROM test1 t1
WHERE t1.NAME IN(SELECT regexp_substr(t2.NAME, '[^,]+', 1, ROWNUM)
FROM test2 t2
CONNECT BY LEVEL <= LENGTH (regexp_replace (t2.NAME, '[^,]+')) + 1
);

SQL Import Data - Insert Only if it doesn't exists

I am using SQL server management tool 2008 to import data to the web host database. I have tables with primary keys. (Id for each row) Now I can import data normally. But when I am importing data for the second time..I need to make sure only those rows that doesn't currently exists only then it's inserted. If there's a to do this using the wizard? If not, then what's the best practice?
Insert the data into a temp table
use left join with main table to identify which records to insert
--
CREATE TABLE T1(col1 int)
go
CREATE TABLE Temp(col1 int )
go
INSERT INTO T1
SELECT 1
UNION
SELECT 2
INSERT INTO TEMP
SELECT 1
UNION
SELECT 2
UNION
SELECT 3
UNION
SELECT 4
INSERT INTO T1
SELECT TEMP.col1
FROM Temp
LEFT JOIN T1
ON TEMP.col1 = T1.col1
WHERE T1.col1 IS NULL
I've used this some time ago, maybe it can help:
insert into timestudio.dbo.seccoes (Seccao,Descricao,IdRegiao,IdEmpresa)
select distinct CCT_COD_CENTRO_CUSTO, CCT_DESIGNACAO, '9', '10' from rhxxi.dbo.RH_CCTT0
where CCT_COD_CENTRO_CUSTO not in (select Seccao from TimeStudio.dbo.Seccoes where idempresa = '10')
Or, just use simple a IF Statement.

Comparison Query to Compare Two SQL Server Tables [duplicate]

This question already has answers here:
sql query to return differences between two tables
(14 answers)
Closed 6 years ago.
I would like to know how to compare two different database table records. What I mean is I will compare two database tables which may have different column names but same data. But one of them may have more records than the other one so I want to see what the difference is between those two tables. To do that how to write the sql query ? FYI : these two databases are under the same SQL Server instance.
Table1
------+---------
|name |lastname|
------+---------
|John |rose |
------+---------
|Demy |Sanches |
------+---------
Table2
------+----------
|name2|lastname2|
------+----------
|John |rose |
------+----------
|Demy |Sanches |
------+----------
|Ruby |Core |
------+----------
Then when after comparing table 1 and table 2, it should return Ruby Core from Table2.
Select * from Table1
Except
Select * from Table2
It will show all mismatch records between table1 and table2
Late answer but can be useful to other readers of this thread
Beside other solutions, I can recommend SQL comparison tool called ApexSQL Data Diff.
I know you'd prefer the solution not based on the software, but for other visitors, who may want to do this in an easier way, I strongly suggest reading this article: http://solutioncenter.apexsql.com/how-to-compare-sql-server-database-tables-with-different-names/
The article explains how to use the Object mapping feature in ApexSQL Data Diff, which is particularly useful in situations where two tables share the same name, but their column names are different.
To handle such a case - each column pair needs to be mapped manually in order for the data stored within them to be included when comparing SQL database tables for differences.
If you do an outer join from T1 to T2 you can find rows in the former that are not in the latter by looking for nulls in the T2 values, similarly an outer join of T2 to T1 will give you rows in T2. Union the two together and you get the lot... something like:
SELECT 'Table1' AS TableName, name, lastname FROM
Table1 OUTER JOIN Table2 ON Table1.name = Table2.name2
AND Table1.lastname = Table2.lastname
WHERE Table2.name2 IS NULL
UNION
SELECT 'Table2' AS TableName, name2 as name, lastname2 as lastname FROM
Table2 OUTER JOIN Table1 ON Table2.name2 = Table1.name
AND Table2.lastname2 = Table1.lastname
WHERE Table1.name IS NULL
That's off the top of my head - and I'm a bit rusty :)
If you are using Sql server use a full join. it does exactly the same as Murph said but in one command.
SELECT 'Table1' AS TableName, name, lastname
FROM Table1
FULL JOIN Table2 ON Table1.name = Table2.name2
AND Table1.lastname = Table2.lastname
You could use the CHECKSUM function if you're confident that the data is expressed identically.
Example:
if not OBJECT_ID('Table1', 'Table') is null drop table Table1
if not OBJECT_ID('Table2', 'Table') is null drop table Table2
create table table1
( id int identity(0, 1),
name varchar(128),
lastname varchar(128)
)
create table table2
( id int identity(0, 1),
name varchar(128),
lastname varchar(128)
)
insert into table1 (name, lastname) values ('John', 'rose')
insert into table1 (name, lastname) values ('Demy', 'Sanches')
insert into table2 (name, lastname) values ('John', 'rose')
insert into table2 (name, lastname) values ('Demy', 'Sanches')
insert into table2 (name, lastname) values ('Ruby', 'Core')
select
table2.*
from table1
right outer join table2 on CHECKSUM(table1.name, table1.lastname) = CHECKSUM(table2.name, table2.lastname)
where table1.id is null
See the CHECKSUM MSDN topic for more information.
Try dbForge Data Compare for SQL Server. It can compare and synchronize any database data. Quick, easy, always delivering a correct result. See how it flies on your database!
create table #test
(
Sno INT IDENTITY(1,1),
ExpDate VARCHAR(50),
Amt INT,
Amt1 INT,
Amt2 INT,
SumoAmt INT
)
create table #test1
(
Sno INT IDENTITY(1,1),
ExpDate VARCHAR(50),
Amt INT,
Amt1 INT,
Amt2 INT,
SumoAmt INT
)
INSERT INTO #test(Expdate,Amt,Amt1,Amt2,SumoAmt) values ('30-07-2012',10,20,10,40)
INSERT INTO #test(Expdate,Amt,Amt1,Amt2,SumoAmt) values ('30-07-2012',10,20,20,50)
INSERT INTO #test(Expdate,Amt,Amt1,Amt2,SumoAmt) values ('30-07-2012',10,20,30,60)
INSERT INTO #test(Expdate,Amt,Amt1,Amt2,SumoAmt) values ('30-07-2012',NULL,20,40,70)
INSERT INTO #test1(Expdate,Amt,Amt1,Amt2,SumoAmt) values ('30-07-2012',10,20,10,40)
INSERT INTO #test1(Expdate,Amt,Amt1,Amt2,SumoAmt) values ('30-07-2012',10,20,20,50)
INSERT INTO #test1(Expdate,Amt,Amt1,Amt2,SumoAmt) values ('30-07-2012',10,20,30,60)
INSERT INTO #test1(Expdate,Amt,Amt1,Amt2,SumoAmt) values ('30-07-2012',NULL,20,40,70)
SELECT MIN(TableName) as TableName, Sno,Expdate,Amt,Amt1,Amt2,SumoAmt
FROM
(
SELECT '#test' as TableName,Sno,Expdate,Amt,Amt1,Amt2,SumoAmt
FROM #test
UNION ALL
SELECT '#test1' as TableName,Sno,Expdate,Amt,Amt1,Amt2,SumoAmt
FROM #test1
) tmp
GROUP BY Sno,Expdate,Amt,Amt1,Amt2,SumoAmt
HAVING COUNT(*) = 1
ORDER BY sno
If you want the differences from both the table.
(SELECT *, 'in Table1' AS Comments
FROM Table1
EXCEPT
SELECT * , 'in Table1' AS Comments
FROM Table2)
UNION
(SELECT *, 'in Table2' AS Comments
FROM Table2
EXCEPT
SELECT *, 'in Table2' AS Comments
FROM Table1)
Firefly will do exactly what you're looking for. It lets you build two sql statements then compare the results of the sql queries showing missing rows and data differences. Each query can even come from a different database like oracle / sql server.
http://download.cnet.com/Firefly-Data-Compare-Tool/3000-10254_4-10633690.html?tag=mncol