What is the best way to apply SQL Pivot to a table? - sql

I'm trying to use the pivot function but not sure how to replace the Letters in the bi-weekly table to a persons name instead of the query result. Here is my query.
SELECT a.OCD, a.[f_name],a.[l_name],a.[ads_phone], b.wk1m, b.wk1t, b.wk1w, b.wk1r, b.wk1f, b.wk2m, b.wk2t, b.wk2w, b.wk2r, b.wk2f
FROM [dbo].[tbl_teleworkers] as a
inner join [dbo].[tbl_scheduled] as b ON a.pin = b.pin
where ocd = '022'
here is my result:
OCD|f_name|l_name|ads_phone|wk1m|wk1t|wk1w|wk1r|wk1f|wk2m|wk2t|wk2w|wk2r|wk2f
022|John |smith |111-1111 |M | | | R | | | | | |
022|Jane |smith |222-2222 | | | | | | | | W | |
022|Joe |smith |333-3333 |M | | | | F | | | | |
022|Jim |smith |444-4444 | | T | | | | M | | | |
022|Jill |smith |555-5555 |M | | W | | | | | | R |
Here is what I'm looking to get: So instead of the letters of the week. I'm trying to display the person's name and phone number.
wk1m |wk1t|wk1w|wk1r |wk1f|wk2m|wk2t|wk2w |wk2r|wk2f
John | | |John | | | | | |
phone| | |phone| | | | | |
| | | | | | |Jane S| |
| | | | | | |phone | |
Joe | |
phone| |
Any help would be greatly appreciated.
Thanks!


I'm going to ignore the question in your subject line because your sample data and desired output do not require a PIVOT at all.
You can get your desired output by making each column a CASE expression. In each wk** column, if the value is not empty, return the value of the first name, last initial (if desired) and phone number concatenated together.

Related

Sql server : select members having 2 or more records on different date of same class

I am trying to find out all the member ids who have more than 2 records on a different dates in the same class.
+----------+------------+-------+-----------+
| MemberId | Date | Class | |
+----------+------------+-------+-----------+
| 118111 | 2/18/2020 | A | Valid |
| 118111 | 10/15/2020 | A | Valid |
| 118216 | 1/31/2020 | B | Valid |
| 118216 | 5/16/1981 | B | Valid |
| 118291 | 6/9/2020 | A | Valid |
| 118291 | 12/5/2020 | A | Valid |
| 118533 | 4/9/2020 | A | Not valid |
| 118533 | 11/11/2020 | B | Not valid |
| 118533 | 7/22/2020 | C | Valid |
| 118533 | 10/25/2020 | C | Valid |
| 118293 | 3/30/2020 | A | Not valid |
| 118293 | 3/30/2020 | A | Not valid |
| 118499 | 4/16/2020 | B | Valid |
| 118499 | 7/26/2020 | B | Valid |
| 118499 | 3/25/2020 | A | Not valid |
+----------+------------+-------+-----------+
I have made a query which checks only 2 records but unable to find a solution for checking more than 2 records.
select mc.*
FROM table1 AS mc
JOIN table1 AS ma ON ma.memberid = mc.memberid
AND ma.date != mc.date
AND ma.class = mc.class

Assuming you just want the memberid you can us a HAVING:
SELECT memberid
FROM dbo.YourTable
GROUP BY memberid
HAVING COUNT(DISTINCT [Date]) > 2;

How to get the number of empty cells for each column in spark dataframe

I'd like to get the number of each column's empty value, so I tried
ele_df.where(ele_df['Shipment_ID'].isNotNull()).select('Shipment_ID').show()
But it returns me the empty value, it seems it consider the empty value as a non-null value.
+------------------+
|Shipment_ID|
+------------------+
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
+------------------+
Could you guys help me with this?

Splitting a table on comma separated emails in Big Query

I have a table with following columns (The email address are comma separated):
+---------+----------+------------+---------------------------------------------+---------+
| Sr. No. | Domain | Store Name | Email | Country |
+---------+----------+------------+---------------------------------------------+---------+
| 1. | kkp.com | KKP | den#kkp.com, info#kkp.com, reno#kkp.com | US |
| 2. | lln.com | LLN | silo#lln.com | UK |
| 3. | ddr.com | DDR | info#ddr.com, dave#ddr.com | US |
| 4. | cpp.com | CPP | hello#ccp.com, info#ccp.com, stelo#ccp.com | CN |
+---------+----------+------------+---------------------------------------------+---------+
I want the output with Email in separate Columns:
+---------+----------+------------+---------------+---------------+---------------+---------+---------+
| Sr. No. | Domain | Store Name | Email 1 | Email 2 | Email 3 | Email N | Country |
|---------+----------+------------+---------------+---------------+---------------+---------+---------+
| 1. | kkp.com | KKP | den#kkp.com | info#kkp.com | reno#kkp.com | ....... | US |
| 2. | lln.com | LLN | silo#lln.com | | | ....... | UK |
| 3. | ddr.com | DDR | info#ddr.com | dave#ddr.com | | ....... | US |
| 4. | cpp.com | CPP | hello#ccp.com | info#ccp.com | stelo#ccp.com | ....... | CN |
+---------+----------+------------+---------------+---------------+---------------+---------+---------+
Can someone please help a beginner in SQL and BigQuery.

Counting based on group of 1st column

I am using following query to count how many Bill_date each BAN have
select replace(c.usertoken, '-', '') as BAN
, to_char(to_date(bi.name,'YYYY-MM-DD'),'dd-mm-yy') as Billdate_dmy
, (replace(c.usertoken, '-', '') ||':'|| to_char(to_date(bi.name,'YYYY-MM-DD'),'dd-mm-yy')) as BAN_Billdate_dmy
, count(c.usertoken) as Number_Of_Bills
from customer c
, service s
, document d
, bill bi
, batch ba
, billrun br
where c.ID = s.CUSTOMER_SERVICE_ID
and s.ID = d.SERVICE_DOCUMENT_ID
and bi.ID = d.BILL_DOCUMENT_ID
and d.BATCH = ba.ID
and ba.BILLRUN = br.ID
and br.STATUS = 'APPROVED'
and c.brand='rogers'
and d.VERSIONEDCONTENTFOLDER='cbu'
group by c.usertoken, bi.name
order by c.usertoken
Output of the above query
+-----------+----------+--------------------+--------------+--+-------+
| BAN | Bill_date | BAN_Billdate | Count |
+-----------+----------+--------------------+--------------+--+-------+
| 100001247 | 25-09-19 | 100001247:25-09-19 | 1 | | |
| 100001247 | 25-10-19 | 100001247:25-10-19 | 1 | | |
| 100002583 | 15-10-19 | 100002583:15-10-19 | 1 | | |
| 100004753 | 25-09-19 | 100004753:25-09-19 | 1 | | |
| 100004753 | 25-10-19 | 100004753:25-10-19 | 1 | | |
| 100005719 | 25-09-19 | 100005719:25-09-19 | 1 | | |
| 100005719 | 25-10-19 | 100005719:25-10-19 | 1 | | |
| 100006311 | 06-09-19 | 100006311:06-09-19 | 1 | | |
| 100009596 | 25-09-19 | 100009596:25-09-19 | 1 | | |
| 100009596 | 25-10-19 | 100009596:25-10-19 | 1 | | |
+-----------+----------+--------------------+--------------+--+-------+
However I was expecting the following output
+-----------+----------+--------------------+--------------+--+-------+
| BAN | Billdate | BAN_Billdate | | Count |
+-----------+----------+--------------------+--------------+--+-------+
| 100001247 | 25-09-19 | 100001247:25-09-19 | 2 | | |
| 100001247 | 25-10-19 | 100001247:25-10-19 | 2 | | |
| 100002583 | 15-10-19 | 100002583:15-10-19 | 3 | | |
| 100004753 | 25-09-19 | 100004753:25-09-19 | 3 | | |
| 100004753 | 25-10-19 | 100004753:25-10-19 | 3 | | |
| 100005719 | 25-09-19 | 100005719:25-09-19 | 2 | | |
| 100005719 | 25-10-19 | 100005719:25-10-19 | 2 | | |
| 100006311 | 06-09-19 | 100006311:06-09-19 | 1 | | |
| 100009596 | 25-09-19 | 100009596:25-09-19 | 2 | | |
| 100009596 | 25-10-19 | 100009596:25-10-19 | 2 | | |
+-----------+----------+--------------------+--------------+--+-------+
Please advise what changes should I do in the query to have the count column reflecting the expected values.

I don't want to touch your query and the archaic join syntax. Please learn proper SQL grammar with JOIN and ON clauses for joins.
That said, you seem to want a window function to sum the counts:
select sum(count(*)) over (partition by ban, to_date(bi.name, 'YYYY-MM-DD'))
I'm not sure that aggregation is really useful, if you are only getting one row per group. In that case, you might want to remove the group by and use:
select count(*) over (partition by ban, to_date(bi.name, 'YYYY-MM-DD'))

Inefficient SQL Search Query - Oracle DB

My logic in this query is right (well im 80% sure it is). but its been running for 2h 23min and still going, was wondering if some one could maybe help me make this run a bit more efficiently as i don't think its that intense of a query
SELECT b.bridge_no, COUNT(*) AS comment_cnt
FROM iacd_asset b INNER JOIN iacd_note c
ON REGEXP_LIKE(c.comments, '(^|\W)BN' || b.bridge_no || '(\W|$)', 'i')
inner join ncr_note e on c.note_id=e.note_id
inner join ncr f on e.ncr_id=f.ncr_id
inner join ncr_iac g on f.ncr_id=g.ncr_id
WHERE c.create_dt >= date'2015-01-01'
AND c.create_dt < date'2015-03-12'
AND length(b.bridge_no) > 1
AND g.scheme in (1, 3, 5, 6, 7, 8, 9, 9, and about 10 more values)
GROUP BY b.bridge_no
ORDER BY comment_cnt;
in short the query should be making a bunch of joins, and then filtering the joined table by schemes (g.scheme in....) , and then parsing the notes field for anything with BN in it.
PLAN TABLE, ok i have never used one before, but i believe this is the plan table
+------------------+----------------+--------------+------------------+--------------+-----------------+----------------+-----------+----+-----------+-------+----------+---------+-------------+-------------+-----------------+---------+------------+-----------------------------+---------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------+------+--------------+
| OPERATION | OPTIONS | OBJECT_OWNER | OBJECT_NAME | OBJECT_ALIAS | OBJECT_INSTANCE | OBJECT_TYPE | OPTIMIZER | ID | PARENT_ID | DEPTH | POSITION | COST | CARDINALITY | BYTES | CPU_COST | IO_COST | TEMP_SPACE | ACCESS_PREDICATES | FILTER_PREDICATES | PROJECTION | TIME | QBLOCK_NAME |
+------------------+----------------+--------------+------------------+--------------+-----------------+----------------+-----------+----+-----------+-------+----------+---------+-------------+-------------+-----------------+---------+------------+-----------------------------+---------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------+------+--------------+
| SELECT STATEMENT | | | | | | | ALL_ROWS | 0 | | 0 | 281,503 | 281,503 | 40 | 4,480 | 148,378,917,975 | 215,677 | | | | | 458 | |
+------------------+----------------+--------------+------------------+--------------+-----------------+----------------+-----------+----+-----------+-------+----------+---------+-------------+-------------+-----------------+---------+------------+-----------------------------+---------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------+------+--------------+
| SORT | ORDER BY | | | | | | | 1 | 0 | 1 | 1 | 281,503 | 40 | 4,480 | 148,378,917,975 | 215,677 | | | | (#keys=1) COUNT(*)[22], "B"."BRIDGE_NO"[NUMBER,22] | 458 | SEL$81719215 |
+------------------+----------------+--------------+------------------+--------------+-----------------+----------------+-----------+----+-----------+-------+----------+---------+-------------+-------------+-----------------+---------+------------+-----------------------------+---------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------+------+--------------+
| HASH | GROUP BY | | | | | | | 2 | 1 | 2 | 1 | 281,503 | 40 | 4,480 | 148,378,917,975 | 215,677 | | | | (#keys=1) "B"."BRIDGE_NO"[NUMBER,22], COUNT(*)[22] | 458 | |
+------------------+----------------+--------------+------------------+--------------+-----------------+----------------+-----------+----+-----------+-------+----------+---------+-------------+-------------+-----------------+---------+------------+-----------------------------+---------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------+------+--------------+
| HASH JOIN | | | | | | | | 3 | 2 | 3 | 1 | 281,497 | 16,084 | 1,801,408 | 148,366,537,976 | 215,677 | 24,126,000 | "G"."NCR_ID"="F"."NCR_ID" | | (#keys=1) "B"."BRIDGE_NO"[NUMBER,22] | 458 | |
+------------------+----------------+--------------+------------------+--------------+-----------------+----------------+-----------+----+-----------+-------+----------+---------+-------------+-------------+-----------------+---------+------------+-----------------------------+---------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------+------+--------------+
| HASH JOIN | | | | | | | | 4 | 3 | 4 | 1 | 96,996 | 209,778 | 21,607,134 | 13,549,630,814 | 90,985 | 22,725,000 | "E"."NCR_ID"="F"."NCR_ID" | | (#keys=1) "F"."NCR_ID"[NUMBER,22], "B"."BRIDGE_NO"[NUMBER,22] | 158 | |
+------------------+----------------+--------------+------------------+--------------+-----------------+----------------+-----------+----+-----------+-------+----------+---------+-------------+-------------+-----------------+---------+------------+-----------------------------+---------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------+------+--------------+
| HASH JOIN | | | | | | | | 5 | 4 | 5 | 1 | 42,595 | 208,419 | 20,216,643 | 5,484,063,163 | 40,162 | 9,839,000 | "C"."NOTE_ID"="E"."NOTE_ID" | REGEXP_LIKE ("C"."COMMENTS",'(^|\W)BN'||TO_CHAR("B"."BRIDGE_NO")||'(\W|$)','i') | (#keys=1) "B"."BRIDGE_NO"[NUMBER,22], "E"."NCR_ID"[NUMBER,22] | 70 | |
+------------------+----------------+--------------+------------------+--------------+-----------------+----------------+-----------+----+-----------+-------+----------+---------+-------------+-------------+-----------------+---------+------------+-----------------------------+---------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------+------+--------------+
| PARTITION RANGE | SINGLE | | | | | | | 6 | 5 | 6 | 1 | 1,039 | 104,603 | 8,577,446 | 62,280,224 | 1,011 | | | | "C"."NOTE_ID"[NUMBER,22], "C"."COMMENTS"[VARCHAR2,4000] | 2 | |
+------------------+----------------+--------------+------------------+--------------+-----------------+----------------+-----------+----+-----------+-------+----------+---------+-------------+-------------+-----------------+---------+------------+-----------------------------+---------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------+------+--------------+
| TABLE ACCESS | FULL | IACDB | IACD_NOTE | C#SEL$1 | 2 | TABLE | ANALYZED | 7 | 6 | 7 | 1 | 1,039 | 104,603 | 8,577,446 | 62,280,224 | 1,011 | | | "C"."CREATE_DATE"<TO_DATE(' 2014-12-31 00:00:00', 'syyyy-mm-dd hh24:mi:ss') | "C"."NOTE_ID"[NUMBER,22], "C"."COMMENTS"[VARCHAR2,4000] | 2 | SEL$81719215 |
+------------------+----------------+--------------+------------------+--------------+-----------------+----------------+-----------+----+-----------+-------+----------+---------+-------------+-------------+-----------------+---------+------------+-----------------------------+---------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------+------+--------------+
| MERGE JOIN | CARTESIAN | | | | | | | 8 | 5 | 6 | 2 | 24,267 | 12,268,270 | 184,024,050 | 2,780,501,758 | 23,033 | | | | (#keys=0) "B"."BRIDGE_NO"[NUMBER,22], "E"."NCR_ID"[NUMBER,22], "E"."NOTE_ID"[NUMBER,22] | 40 | |
+------------------+----------------+--------------+------------------+--------------+-----------------+----------------+-----------+----+-----------+-------+----------+---------+-------------+-------------+-----------------+---------+------------+-----------------------------+---------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------+------+--------------+
| TABLE ACCESS | FULL | IACDB | IACD_ASSET | B#SEL$1 | 1 | TABLE | ANALYZED | 9 | 8 | 7 | 1 | 7 | 40 | 160 | 560,542 | 7 | | | LENGTH(TO_CHAR("B"."BRIDGE_NO"))>1 | "B"."BRIDGE_NO"[NUMBER,22] | 1 | SEL$81719215 |
+------------------+----------------+--------------+------------------+--------------+-----------------+----------------+-----------+----+-----------+-------+----------+---------+-------------+-------------+-----------------+---------+------------+-----------------------------+---------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------+------+--------------+
| BUFFER | SORT | | | | | | | 10 | 8 | 7 | 2 | 24,259 | 308,248 | 3,390,728 | 2,779,941,216 | 23,026 | | | | (#keys=0) "E"."NCR_ID"[NUMBER,22], "E"."NOTE_ID"[NUMBER,22] | 40 | |
+------------------+----------------+--------------+------------------+--------------+-----------------+----------------+-----------+----+-----------+-------+----------+---------+-------------+-------------+-----------------+---------+------------+-----------------------------+---------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------+------+--------------+
| TABLE ACCESS | FULL | IACDB | IACD_NCR_NOTE | E#SEL$2 | 4 | TABLE | ANALYZED | 11 | 10 | 8 | 1 | 606 | 308,248 | 3,390,728 | 69,498,530 | 576 | | | | "E"."NCR_ID"[NUMBER,22], "E"."NOTE_ID"[NUMBER,22] | 1 | SEL$81719215 |
+------------------+----------------+--------------+------------------+--------------+-----------------+----------------+-----------+----+-----------+-------+----------+---------+-------------+-------------+-----------------+---------+------------+-----------------------------+---------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------+------+--------------+
| INDEX | FAST FULL SCAN | IACDB | PK_IACDNCR_NCRID | F#SEL$3 | | INDEX (UNIQUE) | ANALYZED | 12 | 4 | 5 | 2 | 31,763 | 22,838,996 | 137,033,976 | 3,248,120,913 | 30,322 | | | | "F"."NCR_ID"[NUMBER,22] | 52 | SEL$81719215 |
+------------------+----------------+--------------+------------------+--------------+-----------------+----------------+-----------+----+-----------+-------+----------+---------+-------------+-------------+-----------------+---------+------------+-----------------------------+---------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------+------+--------------+
| TABLE ACCESS | FULL | IACDB | IACD_NCR_IAC | G#SEL$4 | 8 | TABLE | ANALYZED | 13 | 3 | 4 | 2 | 181,461 | 1,731,062 | 15,579,558 | 134,407,812,606 | 121,833 | | | ALL THE SCHEMES CHCECKS | "G"."NCR_ID"[NUMBER,22] | 295 | SEL$81719215 |
+------------------+----------------+--------------+------------------+--------------+-----------------+----------------+-----------+----+-----------+-------+----------+---------+-------------+-------------+-----------------+---------+------------+-----------------------------+---------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------+------+--------------+
Hopefully thats legible enough
interms of indexes i assume only the fields that are being sorted is importent
crate_dt is not indexed
scheme id is indexed
Maybe my order in query is wrong...

The plan shows you're doing FULL TABLE SCAN of IACD_NOTE and IACD_ASSET, and then doing a CARETESIAN join of them, because you have provided no criteria for linking one record in IACD_ASSET to a set of records in IACD_NOTE.
That's not my definition of a non-intense query, and the eye-popping values for CPU cost bear that out.
You need to replace this ..,
FROM iacd_asset b INNER JOIN iacd_note c
ON REGEXP_LIKE(c.comments, '(^|\W)BN' || b.bridge_no || '(\W|$)', 'i')
... with an actual join on indexed columns. It would be helpful if Notes were linked to Assets by a foreign key of BRIDGE_NO or similar. I don't know your data model. Then you can use that regex as an additional filter in the WHERE clause.
Also you join to three further tables, to get to something which allows an additional filter on SCHEME. Again, I don't know your data model but this seems pretty inefficient.
Unfortunately this is the sort of tuning which relies on domain knowledge. Fixing this query requires understanding of the data - its volume, distribution and skew, the data model itself and the business logic your query implements. This is way beyond the scope of the advice we can offer in StackOverflow.
One thing to consider, but it is a big decision would be to index the comments with a free text index. However, that has lots of ramifications (especially space and database admin). Find out more.