Combining data from 2 tables in to 1 dynamic query - sql

I have two tables:
table 1
id item itemType
-----------------------
1 book1 1
2 book2 1
3 laptop1 2
table 2
id itemId name value
------------------------------------------
1 1 author enid blyton
2 1 title five 1
3 2 author enid blyton
4 2 title five 2
5 3 cpu i7-940
6 3 ram 4 GB
7 3 vcard nvidia quadro
When I query with filter itemType = 1, the result should be:
query 1
id item author title
--------------------------------------------------------
1 book1 enid blyton five 1
2 book2 enid blyton five 2
and with filter itemType = 2
query 2
id item cpu ram vcard
----------------------------------------------
1 laptop1 i7-940 4 GB nvidia quadro
and without filter
query 3
id item author title cpu ram vcard
---------------------------------------------------------------------------
1 book1 enid blyton five 1
2 book2 enid blyton five 2
1 laptop1 i7-940 4 GB nvidia quadro
The reason I use table 2 is because the parameter of each itemType is created during the fly, so it is not possible to have a table like in query 3.
At this moment I can solve this in C# by rebuilding the table programmatically (using a lot of linq call). With a small size of table 1 (1K rows) and 2 (10K rows), the performance is good, but now the size of table 1 is already more than 100K rows and table 2 is more than 1M rows, and the performance is very low.
Is there any function using SQL query that can solve this problem?

Not exactly dynamic but if your name's are all known upfront, you can use PIVOT to retrieve your data.
PIVOT rotates a table-valued expression by turning the unique values
from one column in the expression into multiple columns in the output,
and performs aggregations where they are required on any remaining
column values that are wanted in the final output.
SQL Statement
SELECT t1.Id
, t1.item
, t2.author
, t2.title
, t2.cpu
, t2.ram
, t2.vcard
FROM table1 t1
INNER JOIN (
SELECT *
FROM (
SELECT itemId
, name
, value
FROM table2
) s
PIVOT (
MAX(Value)
FOR name IN (title, author, cpu, ram, vcard)
) p
) t2 ON t2.itemId = t1.Id
Test script
;WITH table1 (id, item, itemtype) AS (
SELECT 1, 'book1', 1
UNION ALL SELECT 2, 'book2', 1
UNION ALL SELECT 3, 'laptop1', 2
)
, table2 (id, itemId, name, value) AS (
SELECT 1, 1, 'author', 'enid blyton'
UNION ALL SELECT 2, 1, 'title', 'five 1'
UNION ALL SELECT 3, 2, 'author', 'enid blyton'
UNION ALL SELECT 4, 2, 'title', 'five 2'
UNION ALL SELECT 5, 3, 'cpu', 'i7 940'
UNION ALL SELECT 6, 3, 'ram', '4 GB'
UNION ALL SELECT 7, 3, 'vcard', 'nvidia quadro'
)
SELECT t1.Id
, t1.item
, t2.author
, t2.title
, t2.cpu
, t2.ram
, t2.vcard
FROM table1 t1
INNER JOIN (
SELECT *
FROM (
SELECT itemId
, name
, value
FROM table2
) s
PIVOT (
MAX(Value)
FOR name IN (title, author, cpu, ram, vcard)
) p
) t2 ON t2.itemId = t1.Id

I suggest running a query to return all possible names from table2 for the specified itemtype, like so:
select distinct name
from table2 t2
where exists (select null
from table1 t1
where t1.itemtype = #itemtype and
t1.id = t2.item_id)
In C#, concatenate the names into a single comma-separated string, then construct a new query string similar to Lieven's answer, like so:
SELECT t1.item
, t2.*
FROM table1 t1
INNER JOIN (SELECT *
FROM (SELECT itemId,
name,
value
FROM table2) s
PIVOT (MAX(Value)
FOR name IN (/*insert names string here*/)) p
) t2 ON t2.itemId = t1.Id
WHERE t1.itemtype = #itemtype;
(with the names string replacing the comment inside the brackets).
Incidentally, if possible, I suggest separating the names from Table 2 into a separate lookup table, like so:
name_table
----------
name_id
name
itemtype
- this would mean that the first query would only have to query a small lookup table rather than all of table 2; it could also be used for consistency in name values at data entry.

Related

SQL for Exclude

I have a table which is a simple lists of ID numbers and NAMES - I am trying to write a SQL which only returns rows where the NAME does not have particular IDs.
This has been stumping me - the query below returns all as they have other IDs from the exclude lists (large range of IDs). How to structure a query where only those who don't have ID 2 or 3 are returned -- i.e. only returns 'bob' for table below.
select * from TABLE where ID not in (2, 3)
ID NAMES
1 bob
1 alice
2 alice
1 dave
2 dave
3 dave
4 dave
Thank you.
One method is group by and having:
select name
from t
group by name
having sum(case when ID in (2, 3) then 1 else 0 end) = 0;
If you want the original ids, you can add listagg(id, ',') within group (order by id) to the select. Or use not exists:
select t.*
from t
where not exists (select 1
from t t2
where t2.name = t.name and
t2.id in (2, 3)
);

SQL grouping by distinct values in a multi-value string column

(I want to perform a group-by based on the distinct values in a string column that has multiple values
The said column has a list of strings in a standard format separated by commas. The potential values are only a,b,c,d.
For example the column collection (type: String) contains:
Row 1: ["a","b"]
Row 2: ["b","c"]
Row 3: ["b","c","a"]
Row 4: ["d"]`
The expected output is a count of unique values:
collection | count
a | 2
b | 3
c | 2
d | 1
For all the below i used this table:
create table tmp (
id INT auto_increment,
test VARCHAR(255),
PRIMARY KEY (id)
);
insert into tmp (test) values
("a,b"),
("b,c"),
("b,c,a"),
("d")
;
If the possible values are only a,b,c,d you can try one of this:
Tke note that this will only works if you have not so similar values like test and test_new, because then the test would be joined also with all test_new rows and the count would not match
select collection, COUNT(*) as count from tmp JOIN (
select CONCAT("%", tb.collection, "%") as like_collection, collection from (
select "a" COLLATE utf8_general_ci as collection
union select "b" COLLATE utf8_general_ci as collection
union select "c" COLLATE utf8_general_ci as collection
union select "d" COLLATE utf8_general_ci as collection
) tb
) tb1
ON tmp.test LIKE tb1.like_collection
GROUP BY tb1.collection;
Which will give you the result you want
collection | count
a | 2
b | 3
c | 2
d | 1
or you can try this one
SELECT
(SELECT COUNT(*) FROM tmp WHERE test LIKE '%a%') as a_count,
(SELECT COUNT(*) FROM tmp WHERE test LIKE '%b%') as b_count,
(SELECT COUNT(*) FROM tmp WHERE test LIKE '%c%') as c_count,
(SELECT COUNT(*) FROM tmp WHERE test LIKE '%d%') as d_count
;
The result would be like this
a_count | b_count | c_count | d_count
2 | 3 | 2 | 1
What you need to do is to first explode the collection column into separate rows (like a flatMap operation). In redshift the only way to generate new rows is to JOIN - so let's CROSS JOIN your input table with a static table having consecutive numbers, and take only ones having id less or equal to number of elements in the collection. Then we'll use split_part function to read the item at correct index. Once we have the exploaded table, we'll do a simple GROUP BY.
If your items are stored as JSON array strings ('["a", "b", "c"]') then you can use JSON_ARRAY_LENGTH and JSON_EXTRACT_ARRAY_ELEMENT_TEXT instead of REGEXP_COUNT and SPLIT_PART respectively.
with
index as (
select 1 as i
union all select 2
union all select 3
union all select 4 -- could be substituted with 'select row_number() over () as i from arbitrary_table limit 4'
),
agg as (
select 'a,b' as collection
union all select 'b,c'
union all select 'b,c,a'
union all select 'd'
)
select
split_part(collection, ',', i) as item,
count(*)
from index,agg
where regexp_count(agg.collection, ',') + 1 >= index.i -- only get rows where number of items matches
group by 1

Where clause between union all in sql?

I have a query that vertically expands data by using Union condition. Below are the 2 sample tables:
create table #temp1(_row_ord int,CID int,_data varchar(10))
insert #temp1
values
(1,1001,'text1'),
(2,1001,'text2'),
(4,1002,'text1'),
(5,1002,'text2')
create table #temp2(_row_ord int,CID int,_data varchar(10))
insert #temp2
values
(1,1001,'sample1'),
(2,1001,'sample2'),
(4,1002,'sample1'),
(5,1002,'sample2')
--My query
select * from #temp1
union
select * from #temp2 where CID in (select CID from #temp1)
order by _row_ord,CID
drop table #temp1,#temp2
So my current output is:
I want to group the details of every client together for which I am unable to use 'where' clause across Union condition.
My desired output:
Any help?! Order by is also not helping me.
I can imagine you want all of the rows for a CID sorted by _row_ord from the first table before the ones from the second table. And the CID should be the outermost sort criteria.
If that's right, you can select literals from your tables. Let the literal for the first table be less than that of the second table. Then first sort by CID, then that literal and finally by _row_ord.
SELECT cid,
_data
FROM (SELECT 1 s,
_row_ord,
cid,
_data
FROM #temp1
UNION ALL
SELECT 2 s,
_row_ord,
cid,
_data
FROM #temp2) x
ORDER BY cid,
s,
_row_ord;
db<>fiddle
If I correctly understand your need, you need the output to be sorted the way that #temp1 rows appear before #temp2 rows for each cid value.
What you could do is generate additional column ordnum assigning values for each table, just for sorting purposes, and then get rid of it in the outer select statement.
select cid, _data
from (
select 1 as ordnum, *
from #temp1
union all
select 2 as ordnum, *
from #temp2 t2
where exists (
select 1
from #temp1 t1
where t1.cid = t2.cid
)
) q
order by cid, ordnum
I have also rewritten your where condition for an equivalent which should work faster using exists operator.
Live DEMO - click me!
Output
cid _data
1001 text1
1001 text2
1001 sample1
1001 sample2
1002 text1
1002 text2
1002 sample1
1002 sample2
Use With. here is my first try with your sql
create table #temp1(_row_ord int,CID int,_data varchar(10))
insert #temp1
values
(1,1001,'text1'),
(2,1001,'text2'),
(4,1002,'text1'),
(5,1002,'text2')
create table #temp2(_row_ord int,CID int,_data varchar(10))
insert #temp2
values
(1,1001,'sample1'),
(2,1001,'sample2'),
(4,1002,'sample1'),
(5,1002,'sample2');
WITH result( _row_ord, CID,_data) AS
(
--My query
select * from #temp1
union
select * from #temp2 where CID in (select CID from #temp1)
)
select * from tmp order by CID ,_data
drop table #temp1,#temp2
result
_row_ord CID _data
1 1001 sample1
2 1001 sample2
1 1001 text1
2 1001 text2
4 1002 sample1
5 1002 sample2
4 1002 text1
5 1002 text2
Union is placed between two result set blocks and forms a single result set block. If you want a where clause on a particular block you can put it:
select a from a where a = 1
union
select z from z
select a from a
union
select z from z where z = 1
select a from a where a = 1
union
select z from z where z = 1
The first query in a union defines column names in the output. You can wrap an output in brackets, alias it and do a where on the whole lot:
select * from
(
select a as newname from a where a = 1
union
select z from z where z = 2
) o
where o.newname = 3
It is important to note that a.a and z.z will combine into a new column, o.newname. As a result, saying where o.newname will filter on all rows from both a and z (the rows from z are also stacked into the newname column). The outer query knows only about o.newname, it knows nothing of a or z
Side note, the query above produces nothing because we know that only rows where a.a is 1 and z.z is 2 are output by the union as o.newname. This o.newname is then filtered to only output rows that are 3, but no rows are 3
select * from
(
select a as newname from a
union
select z from z
) o
where o.newname = 3
This query will pick up any rows in a or z where a.a is 3 or z.z is 3, thanks to the filtering of the resulting union

edit and Update records using reference id

i have table with multiple records in a field name Comments... with my aspx code the data in comments column gets inserted in three rows with different requirementcommentid but the field comment will remain same
to retrieve distinct i used this query
SELECT distinct (
select top 1 requirementcommentid
from Requirementcomment
where requirementcomment=rc.requirementcomment
and fcr.SectionID in(
SELECT sectionid
FROM [dbo].udfGetSectionID_allComYear(2151)
)
AND fcr.FirmID = 20057
),
rc.IsRejected,
fcr.SectionID,
rc.UserID,
rc.RequirementComment,
convert(varchar(25), dateadd(hour, -5, rc.InsertDate),101) as InsertDate,
Department.DeptName,
FirmUser.DepartmentID,
rc.FirmComplianceYearID
FROM RequirementComment rc
INNER JOIN FirmComplianceRequirement fcr ON fcr.FirmComplianceRequirementID = rc.FirmComplianceRequirementID
INNER JOIN FirmUser ON FirmUser.FirmUserID =rc.UserID
INNER JOIN Department ON Department.DeptID = FirmUser.DepartmentID WHERE rc.IsRejected = 1
AND fcr.SectionID in(SELECT sectionid FROM [dbo].udfGetSectionID_allComYear (2151))
AND fcr.FirmID = 20057 AND rc.RequirementComment!=''
if i want to edit this distinct comment and update it.how can i do this... as only one comment row get edited remaining two rows value in field comment remain the same...!
i want remaining data to be updated automatically if i clicked on edit and updated only single record
If you can not solve this with a procedure when storing, or in .NET, consider to use a trigger. I have made a generic example, since your example code is a bit complex :)
CREATE TABLE TMP_TriggerTable
(
ID INT IDENTITY(1,1) PRIMARY KEY
, ID2 INT NOT NULL
, Comment VARCHAR(255) NOT NULL
)
GO
INSERT INTO TMP_TriggerTable
SELECT 1, 'asd'
UNION ALL
SELECT 1, 'asd'
UNION ALL
SELECT 1, 'asd'
UNION ALL
SELECT 2, 'asd'
UNION ALL
SELECT 2, 'asd'
UNION ALL
SELECT 2, 'asd'
GO
CREATE TRIGGER TRG_TMP_TriggerTable ON TMP_TriggerTable
AFTER UPDATE
AS
BEGIN
WITH InsertedIDPriority AS
(
--Handle if more than one related comment was updated
SELECT Prio = ROW_NUMBER() OVER (PARTITION BY ID2 ORDER BY ID)
, ID
, ID2
, Comment
FROM INSERTED
)
UPDATE t SET Comment = i.Comment FROM TMP_TriggerTable t
JOIN InsertedIDPriority i ON
t.ID2 = i.ID2 --Select all related comments
AND t.ID != i.ID2 --No need to update main column two times
AND i.Prio = 1 --Handle if more than one related comment was updated
END
GO
UPDATE TMP_TriggerTable SET Comment = 'asd2' WHERE ID = 1
/*
SELECT * FROM TMP_TriggerTable
--Returns--
ID ID2 Comment
1 1 asd2
2 1 asd2
3 1 asd2
4 2 asd
5 2 asd
6 2 asd
*/

Comparing two tables and get the values that dont match

I have two tables with articles.
table 1 article and table 2 articlefm
both tables have one field with artnr.
'table 1' has 2192 artnr and 'table 2' has 2195 artnr.
I want in my query to find out whats the artnr of the 3 articles that is not matched.
If 'table 2' has more articles then 'table 1' then I need a list with those artnr.
How can I make this?
You can do this using a FULL JOIN:
SELECT COALESCE(t1.Artnr, t2.Artnr) AS Artnr,
CASE WHEN t1.Artnr IS NULL THEN 'Table1' ELSE 'Table2' END AS MissingFrom
FROM Table1 AS t1
FULL JOIN Table2 AS t2
ON t1.Artnr = t2.Artnr
WHERE t1.Artnr IS NULL
OR t2.Artnr IS NULL;
Note, that just because there is a difference in the count of 3, it does not necessarily mean that there are only 3 records in one table missing from the other. Imagine the following:
Table1 Table2
------ -------
1 2
2 4
3 6
4
The difference in count is 1, but there are actually 2 records present in table1 that aren't in table2, and 1 in table2 that isn't in table1. Using the above full join method you would get a result like:
Artnr | MissingFrom
------+-------------
1 | Table1
3 | Table1
6 | Table2
In most databases you can use except (SQL standard) or minus (Oracle specific):
select artnr
from articlefm -- table 2
except
select artnr
from article -- table 1
Else you could try a not in:
select atrnr
from articlefm -- table 2
where atrnr not in
( select artnr
from article -- table 1
)
This will give you the article numbers that exist in 2, but not in 1.