I have two (linked) sql servers with basically the same setup but they differ in the content. What I want to do is take 2 tables (one from each server) and merge them, so that there are no duplicates ID (e.g., no duplicate fname) and that the the count are added together
They may look like
SERV1.DB1.dbo.Table:
| fname | count |
----------------------
| 'file1.txt' | 10 |
| 'file2.txt' | 5 |
| 'file3.txt' | 35 |
SERV2.DB2.dbo.Table:
| fname | count |
----------------------
| 'file1.txt' | 40 |
| 'file2.txt' | 150 |
And I want to write a select that outputs
| fname | count |
----------------------
| 'file1.txt' | 50 |
| 'file2.txt' | 155 |
| 'file3.txt' | 35 |
I don't want a join and a union doesn't merge them the way I want.
edit
It needs to be case-insensitive as the fname may (read: will) vary in case
fname has different collations (it's a no problem but worth mentioning)
I get these two tables by doing a similar select on each server. I could create temporary tables but I'd prefer if I didn't have too.
You can do it easly with a JOIN
SELECT table1.fname fname, table1.Count + table2.count Count
FROM SERV1.DB1.dbo.Table table1
FULL OUTER JOIN SERV2.DB2.dbo.Table table2 ON table1.fname=table2.fname
EDIT: case-sensitive/-insensitive depends on the column collation
This is my own recommendation, based on giammin's answer:
SELECT
COALESCE(t1.fname,t2.fname) fname,
COALESCE(t1.Count,0) + COALESCE(t2.count,0) Count
FROM
SERV1.DB1.dbo.Table t1
FULL OUTER JOIN
SERV2.DB2.dbo.Table t2
ON t1.fname=t2.fname
which ensures that all fname values from both tables appear in the result, even those that only appear in SERV2's table.
Related
I am working on a project where I have to solve the following problem.
Goal:
If there are two rows that same the same identifier, but additional data that is different, how can I combine all of that data into one row with individual columns?
Example:
DateBase:
| ID | Rating | Rating Provider|
--------------------------------
| 5055 | A+ | Moodys |
---------------------------------
| 5055 | Bb+ | SNP |
Desired End Result:
| ID | Moodys | SNP |
--------------------
| 5005 | A+ | Bb+ |
I believe you simply need a Pivot -
SELECT *
FROM YOUR_TABLE
PIVOT(MAX(Rating)
FOR Rating_Provider IN (Moodys AS 'Moodys', SNP AS 'SNP'));
Quantnesto, i believe that what you are looking for it's the JOIN function. You have the information in different databases, right?
You SELECT all the fields that you want from the different tables
SELECT a.ID,a.Moodys,B.SNP
FROM DataBase a
JOIN Database b on a.ID = b.ID
And that's it.
There are different kinds of JOIN's, for further information let me know, i can explain each type.
I'm struggling to find a value that might be in different tables but using UNION is a pain as there are a lot of tables.
[Different table that contains the suffixes from the TestTable_]
| ID | Name|
| -------- | -----------|
| 1 | TestTable1 |
| 2 | TestTable2 |
| 3 | TestTable3 |
| 4 | TestTable4 |
TestTable1 content:
| id | Name | q1 | a1 |
| -------- | ---------------------------------------- |
| 1 | goose | withFeather? |featherID |
| 2 | rooster| withoutFeather?|shinyfeatherID |
| 3 | rooster| age | 20 |
TestTable2 content:
| id | Name | q1 | a1 |
| -------- | ---------------------------------------------------|
| 1 | brazilian_goose | withFeather? |featherID |
| 2 | annoying_rooster | withoutFeather?|shinyfeatherID |
| 3 | annoying_rooster | no_legs? |dead |
TestTable3 content:
| id | Name | q1 | a1 |
| -------- | ---------------------------------------- |
| 1 | goose | withFeather? |featherID |
| 2 | rooster| withoutFeather?|shinyfeatherID |
| 3 | rooster| age | 15 |
Common columns: q1 and a1
Is there a way to parse through all of them to lookup for a specific value without using UNION because some of them might have different columns?
Something like: check if "q1='age'" exists in all those tables (from 1 to 50)
Select q1,*
from (something)
where q1 exists in (TestTable_*)... or something like that.
If not possible, not a problem.
You could use dynamic SQL but something I do in situations like this where I have a list of tables that I want to quickly perform the same actions on is to either use a spreadsheet to paste the list of tables into and type a query into the cell with something like #table then use the substitute function to replace it.
Alternative I just paste the list into SSMS and use SHIFT+ALT+ArrowKey to select the column and start typing stuff out.
So here is my list of tables
Then I use that key combo. As you can see my cursor has now selected all those rows.
Now I can start typing and all rows selected will get the input.
Then I just go to the other side of the table names and repeat the action
It's not a perfect solution but it's quick a quick and dirty way of doing something repetitive quickly.
If you want to find all the tables with that column name you can use information schema.
Select table_name from INFORMATION_SCHEMA.COLUMNS where COLUMN_NAME = 'q1'
Given the type of solution you are after I can offer a method that I've had to use on legacy systems.
You can query sys.columns for the name of the column(s) you need to find in N tables and join using object_id to sys.tables where type='U'. This will give you a list of table names.
From this list you can then build a working query for each table, and depending on your requirements (is this ad-hoc?) either just manually execute it yourself of build a procedure that will do it for you using sp_executesql
Eg
select t.name, c.name
into #workingtable
from sys.columns c
join sys.tables t on t.object_id=c.object_id
where c.name in .....
psudocode:
begin loop while rows exist in #working table
select top 1 row from #workingtable
set #sql=your query specific to that table and column(s)
exec(#sql) / sp_executesql / try/catch as necessary
delete row from working table
end loop
Hopefully that give ideas at least for how you might implement your requirements.
I want to get all rows in a table where one column matches a relationship with the value of the column in the row in a different table that has the same value of another column.
Concretely, I have two tables, orders and product_info that I'm accessing through Amazon Redshift
Orders
| ID | Date | Amount | Region |
=====================================
| 1 | 2019/4/1 | $120 | A |
| 1 | 2019/4/4 | $100 | A |
| 2 | 2019/4/2 | $50 | A |
| 3 | 2019/4/6 | $70 | B |
The partition keys of order are region and date.
Product Information
| ID | Release Date | Region |
| ---- | ------------ | ------ |
| 1 | 2019/4/2 | A |
| 2 | 2019/4/3 | A |
| 3 | 2019/4/5 | B |
The primary key of product information is id, and the partition key is region.
I want to get all rows from Orders in region A where the date of the row is greater than the release date value in product information for that ID.
So in this case it should return just one row,
| 1 | 2019/4/4 | $100 | A |
I tried doing
select *
from orders
INNER JOIN product_info ON orders.date>product_info.release_date
AND orders.id=product_info.id
AND orders.region=A
AND product_info.region=A
limit 10
The problem is that this query was absurdly slow (cancelled it after 10 minutes). The tables are extremely large, and I have a feeling it was scanning the entire table without restricting it to region first (in reality I have other filters in addition to region that I want to apply to the list of IDs before I do the inner join, but I've limited it to only region for the sake of simplifying the question).
How can I efficiently write this type of query?
The best way to make an SQL query faster is to exclude rows as soon as possible.
So, rather than putting conditions like orders.region=A in the JOIN statement, you should move them to a WHERE statement. This will eliminate rows before they are joined.
Also, make the JOIN condition as simple as possible so that the database can optimize the comparison.
Try something like this:
SELECT *
FROM orders
INNER JOIN product_info ON orders.id = product_info.id
WHERE orders.region = 'A'
AND product_info.region = 'A'
AND orders.date > product_info.release_date
Any further optimization would require consideration of the DISTKEY and SORTKEY on the Redshift tables. (Preferably a DISTKEY of id and a SORTKEY of date).
I have two tables I am using at work to help me gain experience in writing SQL queries. One table contains a list of Applications and has three columns -
Application_Name, Application_Contact_ID and Business_Contact_ID. I then have a separate table called Contacts with two columns - Contact_ID and Contact_Name. I am trying to write a query that will list the Application_Name and Contact_Name for both the Applications_Contact_ID and Business_Contact_ID columns instead of the ID number itself.
I understand I need to JOIN the two tables but I haven't quite figured out how to formulate the correct statement. Help Please!
APPLICATIONS TABLE:
+------------------+------------------------+---------------------+
| Application_Name | Application_Contact_ID | Business_Contact_ID |
+------------------+------------------------+---------------------+
| Adobe | 23 | 23 |
| Word | 52 | 14 |
| NotePad++ | 44 | 989 |
+------------------+------------------------+---------------------+
CONTACTS TABLE:
+------------+--------------+
| Contact_ID | Contact_Name |
+------------+--------------+
| 23 | Tim |
| 52 | John |
| 14 | Jen |
| 44 | Carl |
| 989 | Sam |
+------------+--------------+
What I am trying to get is:
+------------------+--------------------------+-----------------------+
| Application_Name | Application_Contact_Name | Business_Contact_Name |
+------------------+--------------------------+-----------------------+
| Adobe | Tim | Tim |
| Word | John | Jen |
| NotePad++ | Carl | Sam |
+------------------+--------------------------+-----------------------+
I've tried the below but it is only returning the name for one of the columns:
SELECT Application_Name, Application_Contact_ID, Business_Contact_ID, Contact_Name
FROM Applications
JOIN Contact ON Contact_ID = Application_Contact_ID
This is a pretty critical and 101 part of SQL. Consider reading this other answer on a different question, which explains the joins in more depth. The trick to your query, is that you have to join the CONTACTS table twice, which is a bit hard to visualize, because you have to go there for both the application_contact_id and business_contact_id.
There are many flavors of joins (INNER, LEFT, RIGHT, etc.), which you'll want to familiarize yourself with for the future reference. Consider reading this article at the very least: https://www.techonthenet.com/sql_server/joins.php.
SELECT t1.application_name Application_Name,
t2.contact_name Application_Contact_name,
t3.contact_name Business_Contact_name
FROM applications t1
INNER JOIN contacts ON t2 t1.Application_Contact_ID = t2.contact_id -- join contacts for appName
INNER JOIN contacts ON t3 t1.business_Contact_ID = t3.contact_id; -- join contacts for busName
Problem: SQL Query that looks at the values in the "Many" relationship, and doesn't return values from the "1" relationship.
Tables Example: (this shows two different tables).
+---------------+----------------------------+-------+
| Unique Number | <-- Table 1 -- Table 2 --> | Roles |
+---------------+----------------------------+-------+
| 1 | | A |
| 2 | | B |
| 3 | | C |
| 4 | | D |
| 5 | | |
| 6 | | |
| 7 | | |
| 8 | | |
| 9 | | |
| 10 | | |
+---------------+----------------------------+-------+
When I run my query, I get multiple, unique numbers that show all of the roles associated to each number like so.
+---------------+-------+
| Unique Number | Roles |
+---------------+-------+
| 1 | C |
| 1 | D |
| 2 | A |
| 2 | B |
| 3 | A |
| 3 | B |
| 4 | C |
| 4 | A |
| 5 | B |
| 5 | C |
| 5 | D |
| 6 | D |
| 6 | A |
+---------------+-------+
I would like to be able to run my query and be able to say, "When the role of A is present, don't even show me the unique numbers that have the role of A".
Maybe if SQL could look at the roles and say, WHEN role A comes up, grab unique number and remove it from column 1.
Based on what I would "like" to happen (I put that in quotations as this might not even be possible) the following is what I would expect my query to return.
+---------------+-------+
| Unique Number | Roles |
+---------------+-------+
| 1 | C |
| 1 | D |
| 5 | B |
| 5 | C |
| 5 | D |
+---------------+-------+
UPDATE:
Query Example: I am querying 8 tables, but I condensed it to 4 for simplicity.
SELECT
c.UniqueNumber,
cp.pType,
p.pRole,
a.aRole
FROM c
JOIN cp ON cp.uniqueVal = c.uniqueVal
JOIN p ON p.uniqueVal = cp.uniqueVal
LEFT OUTER JOIN a.uniqueVal = p.uniqueVal
WHERE
--I do some basic filtering to get to the relevant clients data but nothing more than that.
ORDER BY
c.uniqueNumber
Table sizes: these tables can have anywhere from 50,000 rows to 500,000+
Pretending the table name is t and the column names are alpha and numb:
SELECT t.numb, t.alpha
FROM t
LEFT JOIN t AS s ON t.numb = s.numb
AND s.alpha = 'A'
WHERE s.numb IS NULL;
You can also do a subselect:
SELECT numb, alpha
FROM t
WHERE numb NOT IN (SELECT numb FROM t WHERE alpha = 'A');
Or one of the following if the subselect is materializing more than once (pick the one that is faster, ie, the one with the smaller subtable size):
SELECT t.numb, t.alpha
FROM t
JOIN (SELECT numb FROM t GROUP BY numb HAVING SUM(alpha = 'A') = 0) AS s USING (numb);
SELECT t.numb, t.alpha
FROM t
LEFT JOIN (SELECT numb FROM t GROUP BY numb HAVING SUM(alpha = 'A') > 0) AS s USING (numb)
WHERE s.numb IS NULL;
But the first one is probably faster and better[1]. Any of these methods can be folded into a larger query with multiple additional tables being joined in.
[1] Straight joins tend to be easier to read and faster to execute than queries involving subselects and the common exceptions are exceptionally rare for self-referential joins as they require a large mismatch in the size of the tables. You might hit those exceptions though, if the number of rows that reference the 'A' alpha value is exceptionally small and it is indexed properly.
There are many ways to do it, and the trade-offs depend on factors such as the size of the tables involved and what indexes are available. On general principles, my first instinct is to avoid a correlated subquery such as another, now-deleted answer proposed, but if the relationship table is small then it probably doesn't matter.
This version instead uses an uncorrelated subquery in the where clause, in conjunction with the not in operator:
select num, role
from one_to_many
where num not in (select otm2.num from one_to_many otm2 where otm2.role = 'A')
That form might be particularly effective if there are many rows in one_to_many, but only a small proportion have role A. Of course you can add an order by clause if the order in which result rows are returned is important.
There are also alternatives involving joining inline views or CTEs, and some of those might have advantages under particular circumstances.