Advanced SQL Select and Union Statements - sql

I've seen other similar questions and I have tried implementing many solutions, but to to no avail so far. This specific questions involves a little more complexity. What I need to do is create a table and join columns to the right side depending on certain criterion. It seems simple enough, but there are a few bumps that I am encountering.
The tables are as follows:
ADC_DATA_COLLECTION_HEADER
(PK)Transaction_ID | BEMSID | DEVICE | TIMESTAMP | CONFIG_NAME
ADC_DATA_COLLECTION_APPS
(FK)CONFIG_NAME | NUM_DATA_ELEMENTS | DATA_ELEMENT1 | DATA_ELEMENT2 | DATA_ELEMENT3 | DATA_ELEMENT4
ADC_DATA_COLLECTION_DATA
(FK)TRANSACTION_ID | DATA_ELEMENT_NUMBER | DATA
I want my final output to look like:
TRANSACTION_ID | DEVICE | CONFIG_NAME | DATA | DATA | DATA | DATA
The "data" column is filled in using the table ADC_DATA_COLLECTION_DATA. The first instance of "data" would be the "data" field in ADC_DATA_COLLECTION_DATA where DATA_ELEMENT_NUMBER = 1. The second instance of "data" would be the "data" field in ADC_DATA_COLLECTION_DATA where DATA_ELEMENT_NUMBER = 2... And so on.
The furthest I have gotten is by using a join statement, except I have nulls in places I do not want them. The code I have used and the results are posted below. So far I only wrote code for the first two columns of data.
SELECT
ADC_Data_Collection_header.BEMSID,
ADC_Data_Collection_header.DEVICE,
ADC_Data_Collection_header.CONFIG_NAME,
null AS locationlabel,
null AS partno
/*null AS partno2,
null AS DE4,
null AS DE5,
null AS DE6 */
FROM
ADC_Data_Collection_header,
ADC_Data_Collection_apps,
ADC_Data_Collection_data
WHERE
ADC_Data_Collection_header.CONFIG_NAME = 'mobileScanning'
AND ADC_Data_Collection_header.BEMSID = '2386531'
AND ADC_Data_Collection_header.CONFIG_NAME = ADC_Data_Collection_apps.CONFIG_NAME
AND (TO_DATE('7/19/2013','MM/DD/YYYY') <= timestamp AND TO_DATE('7/27/2013','MM/DD/YYYY') >= timestamp)
AND ADC_DATA_COLLECTION_HEADER.transaction_ID = ADC_DATA_COLLECTION_DATA.Transaction_ID
UNION
SELECT
null as BEMSID,
null as DEVICE,
null as CONFIG_NAME,
ADC_Data_Collection_DATA.DATA AS locationlabel,
null as partno
FROM
ADC_DATA_COLLECTION_DATA,
ADC_Data_Collection_header,
ADC_Data_Collection_apps
WHERE
ADC_DATA_COLLECTION_DATA.DATA_ELEMENT_NUMBER = 3
AND ADC_Data_Collection_header.CONFIG_NAME = 'mobileScanning'
AND (TO_DATE('7/19/2013','MM/DD/YYYY') <= timestamp AND TO_DATE('7/27/2013','MM/DD/YYYY') >= timestamp)
AND ADC_DATA_COLLECTION_HEADER.transaction_ID = ADC_DATA_COLLECTION_DATA.Transaction_ID
UNION
SELECT
null as BEMSID,
null as DEVICE,
null as CONFIG_NAME,
null as locationlabel,
ADC_Data_Collection_DATA.DATA AS partno
FROM
ADC_DATA_COLLECTION_DATA,
ADC_Data_Collection_header,
ADC_Data_Collection_apps
WHERE
ADC_DATA_COLLECTION_DATA.DATA_ELEMENT_NUMBER = 4
AND ADC_Data_Collection_header.CONFIG_NAME = 'mobileScanning'
AND (TO_DATE('7/19/2013','MM/DD/YYYY') <= timestamp AND TO_DATE('7/27/2013','MM/DD/YYYY') >= timestamp)
AND ADC_DATA_COLLECTION_HEADER.transaction_ID = ADC_DATA_COLLECTION_DATA.Transaction_ID
The result from this appears with null values which I do not want to have.
If you can offer an explicit solution using a join statement or a fix to this union approach, it would be much appreciated. Thank you in advance!

UNION gives you additional rows so it's not the right tool for this situation.
Here's an abbreviated version that uses your ADC_DATA_COLLECTION_DATA table only; you should be able to incorporate this into your query:
SELECT
Transaction_ID,
MAX(CASE WHEN Data_Element_Number = 1 THEN Data END) AS Data1,
MAX(CASE WHEN Data_Element_Number = 2 THEN Data END) AS Data2,
MAX(CASE WHEN Data_Element_Number = 3 THEN Data END) AS Data3,
MAX(CASE WHEN Data_Element_Number = 4 THEN Data END) AS Data4
FROM ADC_DATA_COLLECTION_DATA
GROUP BY Transaction_ID
This is a fairly common "Pivot Table" hack for Oracle (and MySQL and SQL Server). Oracle also supports PIVOT queries but I'm not that good with them.
Note that once you put your final query together with the Device and Config_Name columns, you'll need to add those columns to your GROUP BY.

I would use pivot for this:
select
h.transaction_id,
h.device,
h.config_name,
d.data1,
d.data2,
d.data3,
d.data4
from
ADC_DATA_COLLECTION_HEADER h
inner join (
select *
from ADC_DATA_COLLECTION_DATA
pivot
(
max(data)
for data_element_number in (1 as data1, 2 as data2, 3 as data3, 4 as data4)
)
) d
on d.transaction_id = h.transaction_id
where
(TO_DATE('7/19/2013','MM/DD/YYYY') <= timestamp AND TO_DATE('7/27/2013','MM/DD/YYYY') >= timestamp);
I put together an example SQL Fiddle at: http://www.sqlfiddle.com/#!4/fe1c94/9/0

Related

postgresql total column sum

SELECT
SELECT pp.id, TO_CHAR(pp.created_dt::date, 'dd.mm.yyyy') AS "Date", CAST(pp.created_dt AS time(0)) AS "Time",
au.username AS "User", ss.name AS "Service", pp.amount, REPLACE(pp.status, 'SUCCESS', ' ') AS "Status",
pp.account AS "Props", pp.external_id AS "External", COALESCE(pp.external_status, null, 'indefined') AS "External status"
FROM payment AS pp
INNER JOIN auth_user AS au ON au.id = pp.creator_id
INNER JOIN services_service AS ss ON ss.id = pp.service_id
WHERE pp.created_dt::date = (CURRENT_DATE - INTERVAL '1' day)::date
AND ss.name = 'Some Name' AND pp.status = 'SUCCESS'
id | Date | Time | Service |amount | Status |
------+-----------+-----------+------------+-------+--------+---
9 | 2021.11.1 | 12:20:01 | some serv | 100 | stat |
10 | 2021.12.1 | 12:20:01 | some serv | 89 | stat |
------+-----------+-----------+------------+-------+--------+-----
Total | | | | 189 | |
I have a SELECT like this. I need to get something like the one shown above. That is, I need to get the total of one column. I've tried a lot of things already, but nothing works out for me.
If I understand correctly you want a result where extra row with aggregated value is appended after result of original query. You can achieve it multiple ways:
1. (recommended) the simplest way is probably to union your original query with helper query:
with t(id,other_column1,other_column2,amount) as (values
(9,'some serv','stat',100),
(10,'some serv','stat',89)
)
select t.id::text, t.other_column1, t.other_column2, t.amount from t
union all
select 'Total', null, null, sum(amount) from t
2. you can also use group by rollup clause whose purpose is exactly this. Your case makes it harder since your query contains many columns uninvolved in aggregation. Hence it is better to compute aggregation aside and join unimportant data later:
with t(id,other_column1,other_column2,amount) as (values
(9,'some serv','stat',100),
(10,'some serv','stat',89)
)
select case when t.id is null then 'Total' else t.id::text end as id
, t.other_column1
, t.other_column2
, case when t.id is null then ext.sum else t.amount end as amount
from (
select t.id, sum(amount) as sum
from t
group by rollup(t.id)
) ext
left join t on ext.id = t.id
order by ext.id
3. For completeness I just show you what should be done to avoid join. In that case group by clause would have to use all columns except amount (to preserve original rows) plus the aggregation (to get the sum row) hence the grouping sets clause with 2 sets is handy. (The rollup clause is special case of grouping sets after all.) The obvious drawback is repeating case grouping... expression for each column uninvolved in aggregation.
with t(id,other_column1,other_column2,amount) as (values
(9,'some serv','stat',100),
(10,'some serv2','stat',89)
)
select case grouping(t.id) when 0 then t.id::text else 'Total' end as id
, case grouping(t.id) when 0 then t.other_column1 end as other_column1
, case grouping(t.id) when 0 then t.other_column2 end as other_column2
, sum(t.amount) as amount
from t
group by grouping sets((t.id, t.other_column1, t.other_column2), ())
order by t.id
See example (db fiddle):
(To be frank, I can hardly imagine any purpose other than plain reporting where a column mixes id of number type with label Total of text type.)

Return count of total group membership when providers are part of a group

TABLE A: Pre-joined table - Holds a list of providers who belong to a group and the group the provider belongs to. Columns are something like this:
ProviderID (PK, FK) | ProviderName | GroupID | GroupName
1234 | LocalDoctor | 987 | LocalDoctorsUnited
5678 | Physican82 | 987 | LocalDoctorsUnited
9012 | Dentist13 | 153 | DentistryToday
0506 | EyeSpecial | 759 | OphtaSpecialist
TABLE B: Another pre-joined table, holds a list of providers and their demographic information. Columns as such:
ProviderID (PK,FK) | ProviderName | G_or_I | OtherColumnsThatArentInUse
1234 | LocalDoctor | G | Etc.
5678 | Physican82 | G | Etc.
9012 | Dentist13 | I | Etc.
0506 | EyeSpecial | I | Etc.
The expected result is something like this:
ProviderID | ProviderName | ProviderStatus | GroupCount
1234 | LocalDoctor | Group | 2
5678 | Physican82 | Group | 2
9012 | Dentist13 | Individual | N/A
0506 | EyeSpecial | Individual | N/A
The goal is to determine whether or not a provider belongs to a group or operates individually, by the G_or_I column. If the provider belongs to a group, I need to include an additional column that provides the count of total providers in that group.
The Group/Individual portion is relatively easy - I've done something like this:
SELECT DISTINCT
A.ProviderID,
A.ProviderName,
CASE
WHEN B.G_or_I = 'G'
THEN 'Group'
WHEN B.G_or_I = 'I'
THEN 'Individual' END AS ProviderStatus
FROM
TableA A
LEFT OUTER JOIN TableB B
ON A.ProviderID = B.ProviderID;
So far so good, this returns the expected results based on the G_or_I flag.
However, I can't seem to wrap my head around how to complete the COUNT portion. I feel like I may be overthinking it, and stuck in a loop of errors. Some things I've tried:
Add a second CASE STATEMENT:
CASE
WHEN B.G_or_I = 'G'
THEN (
SELECT CountedGroups
FROM (
SELECT ProviderID, count(GroupID) AS CountedGroups
FROM TableA
WHERE A.ProviderID = B.ProviderID
GROUP BY ProviderID --originally had this as ORDER BY, but that was a mis-type on my part
)
)
ELSE 'N/A' END
This returns an error stating that a single row sub-query is returning more than one row. If I limit the number of rows returned to 1, the CountedGroups column returns 1 for every row. This makes me think that its not performing the count function as I expect it to.
I've also tried including a direct count of TableA as a factored sub-query:
WITH CountedGroups AS
( SELECT Provider ID, count(GroupID) As GroupSum
FROM TableA
GROUP BY ProviderID --originally had this as ORDER BY, but that was a mis-type on my part
) --This as a standalone query works just fine
SELECT DISTINCT
A.ProviderID,
A.ProviderName,
CASE
WHEN B.G_or_I = 'G'
THEN 'Group'
WHEN B.G_or_I = 'I'
THEN 'Individual' END AS ProviderStatus,
CASE
WHEN B.G_or_I = 'G'
THEN GroupSum
ELSE 'N/A' END
FROM
CountedGroups CG
JOIN TableA A
ON CG.ProviderID = A.ProviderID
LEFT OUTER JOIN TableB
ON A.ProviderID = B.ProviderID
This returns either null or completely incorrect column values
Other attempts have been a number of variations of this, with a mix of bad results or Oracle errors. As I mentioned above, I'm probably way overthinking it and the solution could be rather simple. Apologies if the information is confusing or I've not provided enough detail. The real tables have a lot of private medical information, and I tried to translate the essence of the issue as best I could.
Thank you.
You can use the CASE..WHEN and analytical function COUNT as follows:
SELECT
A.PROVIDERID,
A.PROVIDERNAME,
CASE
WHEN B.G_OR_I = 'G' THEN 'Group'
ELSE 'Individual'
END AS PROVIDERSTATUS,
CASE
WHEN B.G_OR_I = 'G' THEN TO_CHAR(COUNT(1) OVER(
PARTITION BY A.GROUPID
))
ELSE 'N/A'
END AS GROUPCOUNT
FROM
TABLE_A A
JOIN TABLE_B B ON A.PROVIDERID = B.PROVIDERID;
TO_CHAR is needed on COUNT as output expression must be of the same data type in CASE..WHEN
Your problem seems to be that you are missing a column. You need to add group name, otherwise you won't be able to differentiate rows for the same practitioner who works under multiple business entities (groups). This is probably why you have a DISTINCT on your query. Things looked like duplicates which weren't. Once you've done that, just use an analytic function to figure out the rest:
SELECT ta.providerid,
ta.providername,
DECODE(tb.g_or_i, 'G', 'Group', 'I', 'Individual') AS ProviderStatus,
ta.group_name,
CASE
WHEN tb.g_or_i = 'G' THEN COUNT(DISTINCT ta.provider_id) OVER (PARTITION BY ta.group_id)
ELSE 'N/A'
END AS GROUP_COUNT
FROM table_a ta
INNER JOIN table_b tb ON ta.providerid = tb.providerid
Is it possible that your LEFT JOIN was going the wrong direction? It makes more sense that your base demographic table would have all practitioners in it and then the Group table might be missing some records. For instance if the solo prac was operating under their own SSN and Type I NPI without applying for a separate Type II NPI or TIN.

How to efficiently get a value from the last row in bulk on SQL Server

I have a table like so
Id | Type | Value
--------------------
0 | Big | 2
1 | Big | 3
2 | Small | 3
3 | Small | 3
I would like to get a table like this
Type | Last Value
--------------------
Small | 3
Big | 3
How can I do this. I understand there is an SQL Server method called LAST_VALUE(...) OVER .(..) but I can't get this to work with GROUP BY.
I've also tried using SELECT MAX(ID) & SELECT TOP 1.. but this seems a bit inefficient since there would be a subquery for each value. The queries take too long when the table has a few million rows in it.
Is there a way to quickly get the last value for these, perhaps using LAST_VALUE?
You can do it using rownumber:
select
type,
value
from
(
select
type,
value,
rownumber() over (partition by type order by id desc) as RN
) TMP
where RN = 1
Can't test this now since SQL Fiddle doesn't seem to work, but hopefully that's ok.
The most efficient method might be not exists, which uses an anti-join for the underlying operator:
select type, value
from likeso l
where not exists (select 1 from likeso l2 where l2.type = l.type and l2.id > l.id)
For performance, you want an index on likeso(type, id).
I really wonder if there is more efficent solution but, I use following query on such needs;
Select Id, Type, Value
From ( Select *, Max (Id) Over (Partition By Type) As LastId
From #Table) T
Where Id = LastId

Normalising a table in Oracle SQL

Was wondering if someone could assist in providing some guidance as to how I could most efficiently normalize the following table so that I can create a refreshable view / table.
Table1:
SYSTEM_KEY | ID | ORDER | ORDER_STATUS | SYSTEM_Actions
A 1 Pencil Open Shipped
B 1 Pencil Open Tested
C 1 Pencil Open Shipped
A 1 Paper Closed Delivered
I'am looking to normalize this table in a repeatable way to something like this:
RESULT:
ID | ORDER | Order Status | A_actions | B_Actions | C_Actions
1 Pencil OPEN Shipped Tested Delivered
1 Paper Closed Delivered null null
I was able to achieve this by doing something similar to this
Select full.ID, full.order, full.orderstatus, case when system_ID = 'A' then sysa.system_actions as A_actions, ....{for B, C}
from table1 full
left join table1 sysa on full.id = sysa.id and full.order = sysa.order
left join table1 sysb on full.id = sysb.id and full.order = sysb.order
Whilst this appeared to work, it was quite clunky in terms of being repeatable having to use several staging tables.
Does anyone know if a good way I can achieve this?
try using group by clause
select id,order,order_status,
case when system_ID = 'A' then system_actions as A_actions,
case when system_ID = 'B' then system_actions as B_actions,
case when system_ID = 'C' then system_actions as C_actions
from table1
group by id,order,order_status

How can I select unique rows in a database over two columns?

I have found similar solutions online but none that I've been able to apply to my specific problem.
I'm trying to "unique-ify" data from one table to another. In my original table, data looks like the following:
USERIDP1 USERIDP2 QUALIFIER DATA
1 2 TRUE AB
1 2 CD
1 3 EF
1 3 GH
The user IDs are composed of two parts, USERIDP1 and USERIDP2 concatenated. I want to transfer all the rows that correspond to a user who has QUALIFIER=TRUE in ANY row they own, but ignore users who do not have a TRUE QUALIFIER in any of their rows.
To clarify, all of User 12's rows would be transferred, but not User 13's. The output would then look like:
USERIDP1 USERIDP2 QUALIFIER DATA
1 2 TRUE AB
1 2 CD
So basically, I need to find rows with distinct user ID components (involving two unique fields) that also possess a row with QUALIFIER=TRUE and copy all and only all of those users' rows.
Although this nested query will be very slow for large tables, this could do it.
SELECT DISTINCT X.USERIDP1, X.USERIDP2, X.QUALIFIER, X.DATA
FROM YOUR_TABLE_NAME AS X
WHERE EXISTS (SELECT 1 FROM YOUR_TABLE_NAME AS Y WHERE Y.USERIDP1 = X.USERIDP1
AND Y.USERIDP2 = X.USERIDP2 AND Y.QUALIFIER = TRUE)
It could be written as an inner join with itself too:
SELECT DISTINCT X.USERIDP1, X.USERIDP2, X.QUALIFIER, X.DATA
FROM YOUR_TABLE_NAME AS X
INNER JOIN YOUR_TABLE_NAME AS Y ON Y.USERIDP1 = X.USERIDP1
AND Y.USERIDP2 = X.USERIDP2 AND Y.QUALIFIER = TRUE
For a large table, create a new auxiliary table containing only USERIDP1 and USERIDP2 columns for rows that have QUALIFIER = TRUE and then join this table with your original table using inner join similar to the second option above. Remember to create appropriate indexes.
This should do the trick - if the id fields are stored as integers then you will need to convert / cast into Varchars
SELECT 1 as id1,2 as id2,'TRUE' as qualifier,'AB' as data into #sampled
UNION ALL SELECT 1,2,NULL,'CD'
UNION ALL SELECT 1,3,NULL,'EF'
UNION ALL SELECT 1,3,NULL,'GH'
;WITH data as
(
SELECT
id1
,id2
,qualifier
,data
,SUM(CASE WHEN qualifier = 'TRUE' THEN 1 ELSE 0 END)
OVER (PARTITION BY id1 + '' + id2) as num_qualifier
from #sampled
)
SELECT
id1
,id2
,qualifier
,data
from data
where num_qualifier > 0
Select *
from yourTable
INNER JOIN (Select UserIDP1, UserIDP2 FROM yourTable WHERE Qualifier=TRUE) B
ON yourTable.UserIDP1 = B.UserIDP1 and YourTable.UserIDP2 = B.UserIDP2
How about a subquery as a where clause?
SELECT *
FROM theTable t1
WHERE CAST(t1.useridp1 AS VARCHAR) + CAST(t1.useridp2 AS VARCHAR) IN
(SELECT CAST(t2.useridp1 AS VARCHAR) + CAST(t.useridp2 AS VARCHAR)
FROM theTable t2
WHERE t2.qualified
);
This is a solution in mysql, but I believe it should transfer to sql server pretty easily. Use a subquery to pick out groups of (id1, id2) combinations with at least one True 'qualifier' row; then join that to the original table on (id1, id2).
mysql> SELECT u1.*
FROM users u1
JOIN (SELECT id1,id2
FROM users
WHERE qualifier
GROUP BY id1, id2) u2
USING(id1, id2);
+------+------+-----------+------+
| id1 | id2 | qualifier | data |
+------+------+-----------+------+
| 1 | 2 | 1 | aa |
| 1 | 2 | 0 | bb |
+------+------+-----------+------+
2 rows in set (0.00 sec)