I have two tables that I would like to combine into an unduplicated list. I have a 'SUB' table that has a 1 column named 'ID' containing a unique identifier for 11,000 some records. I also have another table with 75,000 rows called 'MASTER'. It contains two columns, 'ID' which has the same unique identifier and 'CODE' which contains a unique code for each ID. I want to create a new table that has the 11,000 IDs from the 'SUB' table with the corresponding 'CODE's that match the 'SUB' IDs from the 'MASTER' table. I have used a basic UNION Query, but the results had duplication in the 'ID' column. I tried to consolidate the table produced by that query using Excel, but the list was too long to crunch. Any help? I know this is a basic, but I am not a database person... What would the SQL code look like to achieve this?
Thanks!
You should use JOIN instead of UNION to achieve what you want.
Something like that should work:
SELECT SUB.ID, MASTER.CODE
FROM SUB
JOIN MASTER
ON SUB.ID = MASTER.ID
In general, JOIN allows you to match some rows from one table to the rows from other table according to the value(s) in these rows (think of it as "gluing" rows together along the vertical axis to form longer rows), while what UNION does is just adding all rows from one table below other table (i.e. appends one table to the end of other table along the horizontal axis).
Related
Prolog:
I have two tables in two different databases, one is an updated version of the other. For example we could imagine that one year ago I duplicated table 1 in the new db (say, table 2), and from then I started working on table 2 never updating table 1.
I would like to compare the two tables, to get the differences that have grown in this period of time (the tables has preserved the structure, so that comparison has meaning)
My way of proceeding was to create a third table, in which I would like to copy both table 1 and table 2, and then count the number of repetitions of every entry.
In my opinion, this, added to a new attribute that specifies for every entry the table where he cames from would do the job.
Problem:
Copying the two tables into the third table I get the (obvious) error to have two duplicate key values in a unique or primary key costraint.
How could I bypass the error or how could do the same job better? Any idea is appreciated
Something like this should do what you want if A and B have the same structure, otherwise just select and rename the columns you want to confront....
SELECT
*
FROM
B
WHERE NOT EXISTS (SELECT * FROM A)
if NOT EXISTS doesn't work in your DBMS you could also use a left outer join comparing the rows columns values.
SELECT
A.*
from
A left outer join B
on A.col = B.col and ....
Trying to create relationships (joins) between tables in power pivot.
Got 2 tables I wold like to join together, connected with a common column = CustomerID.
One is a Fact Table the other Dim table (look up).
I have run the "remove duplicates" on both tables without any problem.
But I still get an error saying : "the relationship cannot be created because each column contains duplicate values. Select at least one column that contains only unique values".
The Fact Table contains duplicates (as it should?) and the Dim Table do not, why do I get this error?
Help much appreciated
Created an appended table with both columns "CustomerID". After the columns where appended together I could "remove duplicates" and connect the tables together through the newly created appended table.
Don't know if this causes another problem later however.
You can also check for duplicate id values in a column by using the group by feature.
Remove all columns except ID, add a column that consists only of the number 1.
Group by ID, summing the content of the added column and filter out IDs whose total equals 1. What's left are duplicated IDs.
I have two large Hive tables, say TableA and TableB (which get loaded from different sources).
These two tables have almost identical table structure / columns with same partition column, a date stored as string.
I need to filter records from each table based on certain (identical) filter criteria.
These tables have some columns containing "codes", which need to be looked up to get its corresponding "values".
There are eight to ten such lookup tables, say, LookupA, LookupB, LookupC, etc.,
Now, I need to:
do a union of those filtered records from TableA and TableB.
do a lookup into the lookup tables and replace those "codes" from the filtered records with their respective "values". If a "code" or "value" is unavailable in the filtered records or lookup table respectively, I need to substitute it with zero or an empty string
transform the dates in the filtered records from one format to another
I am a beginner in Hive. Please let know how I can do it. Thanks.
Note: I can manage till union of the tables. Need some guidance on lookup and transformation.
To basically do a lookup Please follow these steps below,
You have to create a custom User Defined function(UDF) which basically does the look up work,meaning you have to create a Java Program internally for looking up, jar it and add it to Hive something like below:
ADD JAR /home/ubuntu/lookup.jar
You then have to add lookup file containing keyvalue pair as follows:
ADD FILE /home/ubuntu/lookupA;
You then have to create a temporary lookup function such as
CREATE TEMPORARY FUNCTION getLookupValueA AS 'com.LookupA';
Finally you have to call this lookup function in the Select query which will basically populate lookup value for the given lookup key.
Same thing can be achieved using JOIN but that will take a hit on the performance.
Taking a join approach you can very well join by the lookupcode for source and lookup tables something like
select a.key,b.lookupvalue
table a join lookuptable b
where a.key=b.lookupKey
Now for Date Transformation, you can use Date functions in Hive.
For the above problem follow the following steps:
Use union schema to union two tables(schema must be same).
For the above scenario you can try pig script.
script would look like(jn table A and tableB with lookup table and generate the appropriate columns):
a = join TableA by codesA left outer, lookupA by codesA.
b = join a by codesB left outer, lookupB by codesB.
Similarly for Table B.
Suppose some value of codesA does not have a value in the lookup table, then:
z = foreach b generate codesA as codesA, valueA is null ? '0' as valuesA.
(will replace all null values from value with 0).
If you are using Pig 0.12 or later, you can use ToString(CurrentTime(),'yyyy-MM-dd')
I hope it will solve your problem. Let me know in case of any concern.
I want only the unique records in same or new table. And I want to do this with different tables (having duplicate records) in access database through same code.
The flow should be like:
input table ------VBA MODULE------> table with unique records
I am able to do this group by function but for that i have to use field names in query. But field names will differ from table to table.
Please help!
Just use query
SELECT DISTINCT * FROM MyAnyTable
Here's the problem:
I have an Access query that feeds a report, which sometimes doesn't return any records for certain criteria. I would like to display zeros in the report instead of an empty line (an empty recordset is currently being returned).
Is there an SQL solution that (perhaps using some kind of union statement and/or nested SQL) always returns one record (with zeros) if there are not matching records from the initial query?
One possible solution would be to create a second table with the same primary key, and add just one record. In your query, choose as join type all records in the second table, including those with no matching records in the first one. Select as output all fields in the first table.
You can materialize a one-row table with zero for all columns. This is a slight pain to achieve in Access (ACE, Jet, whatever) because it doesn't support row constructors and the FROM must resolve to a base table. In other words, you'll need a table that is guaranteed to always contain at least one row.
This isn't a problem for me because my databases always include auxilliary tables e.g. a calendar table, a sequence table of integers, etc. For exmaple, to materialize a table one-row, all-zeros table using my 3000 row Calendar table:
SELECT DISTINCT 0 AS c
FROM Calendar;
I can then UNION my query with my materialized table but include an antijoin to ensure the all-zeros row only appears in the resultset when my query is the empty set:
SELECT c
FROM T
UNION
SELECT 0
FROM Calendar
WHERE NOT EXISTS (
SELECT c
FROM T
);
Note the use of UNION allows me to remove the DISTINCT keyword and the AS clause ("column alias") from the materialized table.