I wrote following SQL query to select data from #tmp table variable.
SELECT #rowCount AS [row-count],
t.[row-no] AS [row-no],
t.[ServiceID] AS ServiceID,
t.ServiceName AS ServiceName,
t.[BranchServiceSortValue] AS SortValue,
(CASE WHEN t.OptIn = 1 THEN 'Yes' ELSE 'No' END) AS OptIn
FROM #tmp t
INNER JOIN dbo.Category
ON Category.CategoryId = t.FkCategoryId
INNER JOIN dbo.ServiceType
ON ServiceType.ServiceTypeId = t.FkServiceTypeId
WHERE t.[row-no] >= #startRow
AND t.[row-no] <= #endRow
ORDER BY t.BranchServiceSortValue,t.serviceName
According to the data in #tmp table,my above query return following output.
| row-count | row-no | ServiceID | ServiceName | SortValue | OptIn |
|-----------|--------|-----------|-------------|-----------|-------|
| 24 | 4 | 1088 | AAB | NULL | No |
| 24 | 5 | 1089 | AAC | NULL | No |
| 24 | 6 | 1090 | AAD | NULL | No |
| 24 | 1 | 1093 | GDGD | 0 | Yes |
| 24 | 7 | 1091 | EETETE | 1 | Yes |
| 24 | 8 | 1092 | CSCDF | 2 | Yes |
| 24 | 3 | 1086 | CXCX | 3 | Yes |
| 24 | 9 | 16 | ASA | 4 | Yes |
| 24 | 2 | 1087 | BFB | 5 | Yes |
| 24 | 10 | 7 | Mortgage | 6 | Yes |
| 24 | 11 | 17 | DDWW | 7 | Yes |
| 24 | 12 | 11 | IL | 8 | Yes |
| 24 | 13 | 5 | SAA | 9 | Yes |
| 24 | 14 | 9 | CD | 10 | Yes |
You can see according to my above query data rows are sorted by SortValue and when SortValue = NULL, those 3 rows sorted by its ServiceName,
But I need to displaySortValue = NULLrows at the bottom of the other rows.Its mean I need to display Null rows after the SortValue Not NULL data and SortValue = NULL should be display order by its ServiceName.
My Expected Output is:
| row-count | row-no | ServiceID | ServiceName | SortValue | OptIn |
|-----------|--------|-----------|-------------|-----------|-------|
| 14 | 1 | 1093 | GDGD | 0 | Yes |
| 14 | 7 | 1091 | EETETE | 1 | Yes |
| 14 | 8 | 1092 | CSCDF | 2 | Yes |
| 14 | 3 | 1086 | CXCX | 3 | Yes |
| 14 | 9 | 16 | ASA | 4 | Yes |
| 14 | 2 | 1087 | BFB | 5 | Yes |
| 14 | 10 | 7 | Mortgage | 6 | Yes |
| 14 | 11 | 17 | DDWW | 7 | Yes |
| 14 | 12 | 11 | IL | 8 | Yes |
| 14 | 13 | 5 | SAA | 9 | Yes |
| 14 | 14 | 9 | CD | 10 | Yes |
| 14 | 4 | 1088 | AAB | NULL | No |
| 14 | 5 | 1089 | AAC | NULL | No |
| 14 | 6 | 1090 | AAD | NULL | No |
How should I need to change my query to get above output? please help me
NULL has the lowest value, so you'll need to use a CASE to put NULL at the end, and then sort by SortValue:
ORDER BY CASE WHEN t.BranchServiceSortValue IS NULL THEN 1 ELSE 0 END,
t.BranchServiceSortValue,
t.serviceName;
Just add a key to the ORDER BY:
ORDER BY (CASE WHEN t.BranchServiceSortValue IS NOT NULL THEN 1 ELSE 2 END),
t.BranchServiceSortValue, t.serviceName
The SQL standard provides the options NULLS FIRST and NULLS LAST for ORDER BY clauses. SQL Server does not (yet) implement these.
Related
This is what I have right now:
| time | car_id | order | in_order |
|-------|--------|-------|----------|
| 12:31 | 32 | null | 0 |
| 12:33 | 32 | null | 0 |
| 12:35 | 32 | null | 0 |
| 12:37 | 32 | 123 | 1 |
| 12:38 | 32 | 123 | 1 |
| 12:39 | 32 | 123 | 1 |
| 12:41 | 32 | 123 | 1 |
| 12:43 | 32 | 123 | 1 |
| 12:45 | 32 | null | 0 |
| 12:47 | 32 | null | 0 |
| 12:49 | 32 | 321 | 1 |
| 12:51 | 32 | 321 | 1 |
I'm trying to rank orders, including those who have null values, in this case by car_id.
This is the result I'm looking for:
| time | car_id | order | in_order | row |
|-------|--------|-------|----------|-----|
| 12:31 | 32 | null | 0 | 1 |
| 12:33 | 32 | null | 0 | 1 |
| 12:35 | 32 | null | 0 | 1 |
| 12:37 | 32 | 123 | 1 | 2 |
| 12:38 | 32 | 123 | 1 | 2 |
| 12:39 | 32 | 123 | 1 | 2 |
| 12:41 | 32 | 123 | 1 | 2 |
| 12:43 | 32 | 123 | 1 | 2 |
| 12:45 | 32 | null | 0 | 3 |
| 12:47 | 32 | null | 0 | 3 |
| 12:49 | 32 | 321 | 1 | 4 |
| 12:51 | 32 | 321 | 1 | 4 |
I just don't know how to manage a count for the null values.
Thanks!
You can count the number of non-NULL values before each row and then use dense_rank():
select t.*,
dense_rank() over (partition by car_id order by grp) as row
from (select t.*,
count(order) over (partition by car_id order by time) as grp
from t
) t;
I have a hive external table with data say, (version less than 0.14)
+--------+------+------+------+
| id | A | B | C |
+--------+------+------+------+
| 10011 | 10 | 3 | 0 |
| 10012 | 9 | 0 | 40 |
| 10015 | 10 | 3 | 0 |
| 10017 | 9 | 0 | 40 |
+--------+------+------+------+
And I have a delta file having data given below.
+--------+------+------+------+
| id | A | B | C |
+--------+------+------+------+
| 10012 | 50 | 3 | 10 | --> update
| 10013 | 29 | 0 | 40 | --> insert
| 10014 | 10 | 3 | 0 | --> update
| 10013 | 19 | 0 | 40 | --> update
| 10015 | 70 | 3 | 0 | --> update
| 10016 | 17 | 0 | 40 | --> insert
+--------+------+------+------+
How can I update my hive table with the delta file, without using sqoop. Any help on how to proceed will be great! Thanks.
This is because there is duplicates in the file. How do you know which you should keep? The last one?
In that case you can use, for example, the row_number and then get the maximum value. Something like that.
SELECT coalesce(tmp.id,initial.id) as id,
coalesce(tmp.A, initial.A) as A,
coalesce(tmp.B,initial.B) as B,
coalesce(tmp.C, initial.C) as C
FROM
table_a initial
FULL OUTER JOIN
( SELECT *, row_number() over( partition by id ) as row_num
,COUNT(*) OVER (PARTITION BY id) AS cnt
FROM temp_table
) tmp
ON initial.id=tmp.id
WHERE row_num=cnt
OR row_num IS NULL;
Output:
+--------+-----+----+-----+--+
| id | a | b | c |
+--------+-----+----+-----+--+
| 10011 | 10 | 3 | 0 |
| 10012 | 50 | 3 | 10 |
| 10013 | 19 | 0 | 40 |
| 10014 | 10 | 3 | 0 |
| 10015 | 70 | 3 | 0 |
| 10016 | 17 | 0 | 40 |
| 10017 | 9 | 0 | 40 |
+--------+-----+----+-----+--+
You can load the file to a temporary table in hive and then execute a FULL OUTER JOIN between the two tables.
Query Example:
SELECT coalesce(tmp.id,initial.id) as id,
coalesce(tmp.A, initial.A) as A,
coalesce(tmp.B,initial.B) as B,
coalesce(tmp.C, initial.C) as C
FROM
table_a initial
FULL OUTER JOIN
temp_table tmp on initial.id=tmp.id;
Output
+--------+-----+----+-----+--+
| id | a | b | c |
+--------+-----+----+-----+--+
| 10011 | 10 | 3 | 0 |
| 10012 | 50 | 3 | 10 |
| 10013 | 29 | 0 | 40 |
| 10013 | 19 | 0 | 40 |
| 10014 | 10 | 3 | 0 |
| 10015 | 70 | 3 | 0 |
| 10016 | 17 | 0 | 40 |
| 10017 | 9 | 0 | 40 |
+--------+-----+----+-----+--+
I have this table
+----+--------+------------+-----------+
| Id | day_id | subject_id | period_Id |
+----+--------+------------+-----------+
| 1 | 1 | 1 | 1 |
| 2 | 1 | 2 | 2 |
| 8 | 2 | 6 | 1 |
| 9 | 2 | 7 | 2 |
| 15 | 3 | 3 | 1 |
| 16 | 3 | 4 | 2 |
| 22 | 4 | 5 | 1 |
| 23 | 4 | 5 | 2 |
| 24 | 4 | 6 | 3 |
| 29 | 5 | 8 | 1 |
| 30 | 5 | 1 | 2 |
to something like this
| Id | day_id | subject_id | period_Id |
| 1 | 1 | 1 | 1 |
| 8 | 2 | 6 | 1 |
| 15 | 3 | 3 | 1 |
| 22 | 4 | 5 | 1 |
| 29 | 5 | 8 | 1 |
| 2 | 1 | 2 | 2 |
| 2 | 1 | 2 | 2 |
| 16 | 3 | 4 | 2 |
| 23 | 4 | 5 | 2 |
| 30 | 5 | 1 | 2 |
+----+--------+------------+-----------+
SO, I want to choose one period with a different subject each day and doing this for number of weeks. so first subject dose not come until all subject have been chosen.
You can ORDER BY period_id first and then by day_id:
SELECT *
FROM your_table
ORDER BY period_Id, day_Id
LiveDemo
Hopefully what I have here is a simple question and explained to you in the correct manner.
I have the following Query:
--DECLARE DATES
DECLARE #Date datetime
DECLARE #DaysInMonth INT
DECLARE #i INT
--GIVE VALUES
SET #Date = Getdate()
SELECT #DaysInMonth = datepart(dd,dateadd(dd,-1,dateadd(mm,1,cast(cast(year(#Date) as varchar)+'-'+cast(month(#Date) as varchar)+'-01' as datetime))))
SET #i = 1
--MAKE TEMP TABLE
CREATE TABLE #TempDays
(
[days] VARCHAR(50)
)
WHILE #i <= #DaysInMonth
BEGIN
INSERT INTO #TempDays
VALUES(#i)
SET #i = #i + 1
END
SELECT #TempDays.days, DATEPART(dd, a.ActualDate) ActualDate, a.ActualAmount, (SELECT SUM(b.ActualAmount)
FROM UnpaidManagement..Actual b
WHERE b.ID <= a.ID) RunningTotal
FROM UnpaidManagement..Actual a
RIGHT JOIN #TempDays on a.ID = #TempDays.days
DROP TABLE #TempDays
Which produces the following output:
+------+------------+--------------+--------------+
| days | ActualDate | ActualAmount | RunningTotal |
+------+------------+--------------+--------------+
| 1 | 1 | 438706 | R 438 706 |
| 2 | 2 | 16239 | R 454 945 |
| 3 | 3 | 1611264 | R 2 066 209 |
| 4 | 4 | 1157777 | R 3 223 986 |
| 5 | 5 | 470662 | R 3 694 648 |
| 6 | 6 | 288628 | 3983276 |
| 7 | 7 | 245897 | 4229173 |
| 8 | 8 | 5235 | 4234408 |
| 9 | 10 | 375630 | 4610038 |
| 10 | 11 | 95610 | 4705648 |
| 11 | 12 | 87285 | 4792933 |
| 12 | 13 | 73399 | 4866332 |
| 13 | 14 | 59516 | 4925848 |
| 14 | 15 | 918915 | 5844763 |
| 15 | 17 | 1957285 | 7802048 |
| 16 | 18 | 489964 | 8292012 |
| 17 | 19 | 272304 | 8564316 |
| 18 | 20 | 378601 | 8942917 |
| 19 | 22 | 92374 | 9035291 |
| 20 | 23 | 198 | 9035489 |
| 21 | 24 | 1500820 | 10536309 |
| 22 | 25 | 2631057 | 13167366 |
| 23 | 26 | 6466505 | 19633871 |
| 24 | 27 | 3757350 | 23391221 |
| 25 | 28 | 3487466 | 26878687 |
| 26 | 29 | 160197 | 27038884 |
| 27 | 30 | 14000 | 27052884 |
| 28 | NULL | NULL | NULL |
| 29 | NULL | NULL | NULL |
| 30 | NULL | NULL | NULL |
| 31 | NULL | NULL | NULL |
+------+------------+--------------+--------------+
If you look closely at the table above, the "ActualDate" column is missing a few values, EG: 9, 16, etc.
And because of this, the rows are being pushed up instead of being grouped with their correct number? How would I accomplish a group by / anything to keep them in their correct row?
DESIRED OUTPUT:
+------+------------+--------------+--------------+
| days | ActualDate | ActualAmount | RunningTotal |
+------+------------+--------------+--------------+
| 1 | 1 | 438706 | R 438 706 |
| 2 | 2 | 16239 | R 454 945 |
| 3 | 3 | 1611264 | R 2 066 209 |
| 4 | 4 | 1157777 | R 3 223 986 |
| 5 | 5 | 470662 | R 3 694 648 |
| 6 | 6 | 288628 | 3983276 |
| 7 | 7 | 245897 | 4229173 |
| 8 | 8 | 5235 | 4234408 |
| 9 | NULL | NULL | NULL |
| 10 | 10 | 375630 | 4610038 |
| 11 | 11 | 95610 | 4705648 |
| 12 | 12 | 87285 | 4792933 |
| 13 | 13 | 73399 | 4866332 |
| 14 | 14 | 59516 | 4925848 |
| 15 | 15 | 918915 | 5844763 |
| 16 | NULL | NULL | NULL |
| 17 | 17 | 1957285 | 7802048 |
| 18 | 18 | 489964 | 8292012 |
| 19 | 19 | 272304 | 8564316 |
| 20 | 20 | 378601 | 8942917 |
| 21 | NULL | NULL | NULL |
| 22 | 22 | 92374 | 9035291 |
| 23 | 23 | 198 | 9035489 |
| 24 | 24 | 1500820 | 10536309 |
| 25 | 25 | 2631057 | 13167366 |
| 26 | 26 | 6466505 | 19633871 |
| 27 | 27 | 3757350 | 23391221 |
| 28 | 28 | 3487466 | 26878687 |
| 29 | 29 | 160197 | 27038884 |
| 30 | 30 | 14000 | 27052884 |
| 31 | NULL | NULL | NULL |
+------+------------+--------------+--------------+
I know this is a long one to read, but please let me know if I have explained this clearly enough. I have been trying to group by this whole morning, but I keep getting errors.
SELECT #TempDays.days, DATEPART(dd, a.ActualDate) ActualDate, a.ActualAmount, (SELECT SUM(b.ActualAmount)
FROM UnpaidManagement..Actual b
WHERE b.ID <= a.ID) RunningTotal
FROM UnpaidManagement..Actual a
RIGHT JOIN #TempDays on DATEPART(dd, a.ActualDate) = #TempDays.days
If you select the temp table as first table in the select and join to UnpaidManagement..Actual you have the days in correct row and order:
SELECT t.days
,DATEPART(dd, a.ActualDate) ActualDate
,a.ActualAmount
,(
SELECT SUM(b.ActualAmount)
FROM UnpaidManagement..Actual b
WHERE b.ID <= a.ID
) RunningTotal
FROM #TempDays AS t
INNER JOIN UnpaidManagement..Actual AS a ON a.IDENTITYCOL = t.days
ORDER BY t.days
After doing that, cou can add CASE WHEN to generate content for the NULL cells.
I have a table where I have a list of people, lets say i have 100 people listed in that table
I need to filter out the people using different criteria's and put them in groups, problem is when i start excluding on the 4th-5th level, performance issues come up and it becomes slow
with lst_tous_movements as (
select
t1.refid_eClinibase
t1.[dthrfinmouvement]
t1.[unite_service_id]
t1.[unite_service_suiv_id]
from sometable t1
)
,lst_patients_hospitalisés as (
select distinct
t1.refid_eClinibase
from lst_tous_movements t1
where
t1.[dthrfinmouvement] = '4000-01-01'
)
,lst_patients_admisUIB_transferes as (
select distinct
t1.refid_eClinibase
from lst_tous_movements t1
left join lst_patients_hospitalisés t2 on t1.refid_eClinibase = t2.refid_eClinibase
where
t1.[unite_service_id] = 4
and t1.[unite_service_suiv_id] <> 0
and t2.refid_eClinibase is null
)
,lst_patients_admisUIB_nonTransferes as (
select distinct
t1.refid_eClinibase
from lst_tous_movements t1
left join lst_patients_admisUIB_transferes t2 on t1.refid_eClinibase = t2.refid_eClinibase
left join lst_patients_hospitalisés t3 on t1.refid_eClinibase = t3.refid_eClinibase
where
t1.[unite_service_id] = 4
and t1.[unite_service_suiv_id] = 0
and t2.refid_eClinibase is null
and t3.refid_eClinibase is null
)
,lst_patients_autres as (
select distinct
t1.refid_eClinibase
from lst_patients t1
left join lst_patients_admisUIB_transferes t2 on t1.refid_eClinibase = t2.refid_eClinibase
left join lst_patients_hospitalisés t3 on t1.refid_eClinibase = t3.refid_eClinibase
left join lst_patients_admisUIB_nonTransferes t4 on t1.refid_eClinibase = t4.refid_eClinibase
where
t2.refid_eClinibase is null
and t3.refid_eClinibase is null
and t4.refid_eClinibase is null
)
as you can see i have a multi level filtering out going on here...
1st i get the people where t1.[dthrfinmouvement] = '4000-01-01'
2nd i get the people with another criteria EXCLUDING the 1st group
3rd i get the people with yet another criteria EXCLUDING the 1st and
the 2nd group
etc..
when i get to the 4th level, my query takes 6 - 10 seconds to complete
is there any way to speed this up ?
this is my dataset i'm working with:
+------------------+-------------------------------+------------------+------------------+-----------------------+
| refid_eClinibase | nodossierpermanent_eClinibase | dthrfinmouvement | unite_service_id | unite_service_suiv_id |
+------------------+-------------------------------+------------------+------------------+-----------------------+
| 25611 | P0017379 | 2013-04-27 | 58 | 0 |
| 25611 | P0017379 | 2013-05-02 | 4 | 2 |
| 25611 | P0017379 | 2013-05-18 | 2 | 0 |
| 85886 | P0077918 | 2013-04-10 | 58 | 0 |
| 85886 | P0077918 | 2013-05-06 | 6 | 12 |
| 85886 | P0077918 | 4000-01-01 | 12 | 0 |
| 91312 | P0083352 | 2013-07-24 | 3 | 14 |
| 91312 | P0083352 | 2013-07-24 | 14 | 3 |
| 91312 | P0083352 | 2013-07-30 | 3 | 8 |
| 91312 | P0083352 | 4000-01-01 | 8 | 0 |
| 93835 | P0085879 | 2013-04-30 | 58 | 0 |
| 93835 | P0085879 | 2013-05-07 | 4 | 2 |
| 93835 | P0085879 | 2013-05-16 | 2 | 0 |
| 93835 | P0085879 | 2013-05-22 | 58 | 0 |
| 93835 | P0085879 | 2013-05-24 | 4 | 0 |
| 93835 | P0085879 | 2013-05-31 | 58 | 0 |
| 93836 | P0085880 | 2013-05-20 | 58 | 0 |
| 93836 | P0085880 | 2013-05-22 | 4 | 2 |
| 93836 | P0085880 | 2013-05-31 | 2 | 0 |
| 97509 | P0089576 | 2013-04-09 | 58 | 0 |
| 97509 | P0089576 | 2013-04-11 | 4 | 0 |
| 102787 | P0094886 | 2013-04-08 | 58 | 0 |
| 102787 | P0094886 | 2013-04-11 | 4 | 2 |
| 102787 | P0094886 | 2013-05-21 | 2 | 0 |
| 103029 | P0095128 | 2013-04-04 | 58 | 0 |
| 103029 | P0095128 | 2013-04-10 | 4 | 1 |
| 103029 | P0095128 | 2013-05-03 | 1 | 0 |
| 103813 | P0095922 | 2013-07-02 | 58 | 0 |
| 103813 | P0095922 | 2013-07-03 | 4 | 6 |
| 103813 | P0095922 | 2013-08-14 | 6 | 0 |
| 105106 | P0097215 | 2013-08-09 | 58 | 0 |
| 105106 | P0097215 | 2013-08-13 | 4 | 0 |
| 105106 | P0097215 | 2013-08-14 | 58 | 0 |
| 105106 | P0097215 | 4000-01-01 | 4 | 0 |
| 106223 | P0098332 | 2013-06-11 | 1 | 0 |
| 106223 | P0098332 | 2013-08-01 | 58 | 0 |
| 106223 | P0098332 | 4000-01-01 | 1 | 0 |
| 106245 | P0098354 | 2013-04-02 | 58 | 0 |
| 106245 | P0098354 | 2013-05-24 | 58 | 0 |
| 106245 | P0098354 | 2013-05-29 | 4 | 1 |
| 106245 | P0098354 | 2013-07-12 | 1 | 0 |
| 106280 | P0098389 | 2013-04-07 | 58 | 0 |
| 106280 | P0098389 | 2013-04-09 | 4 | 0 |
| 106416 | P0098525 | 2013-04-19 | 58 | 0 |
| 106416 | P0098525 | 2013-04-23 | 4 | 0 |
| 106444 | P0098553 | 2013-04-22 | 58 | 0 |
| 106444 | P0098553 | 2013-04-25 | 4 | 0 |
| 106609 | P0098718 | 2013-05-08 | 58 | 0 |
| 106609 | P0098718 | 2013-05-10 | 4 | 11 |
| 106609 | P0098718 | 2013-07-24 | 11 | 12 |
| 106609 | P0098718 | 4000-01-01 | 12 | 0 |
| 106616 | P0098725 | 2013-05-09 | 58 | 0 |
| 106616 | P0098725 | 2013-05-09 | 4 | 1 |
| 106616 | P0098725 | 2013-07-27 | 1 | 0 |
| 106698 | P0098807 | 2013-05-16 | 58 | 0 |
| 106698 | P0098807 | 2013-05-22 | 4 | 6 |
| 106698 | P0098807 | 2013-06-14 | 6 | 1 |
| 106698 | P0098807 | 2013-06-28 | 1 | 0 |
| 106714 | P0098823 | 2013-05-20 | 58 | 0 |
| 106714 | P0098823 | 2013-05-21 | 58 | 0 |
| 106714 | P0098823 | 2013-05-24 | 58 | 0 |
| 106729 | P0098838 | 2013-05-21 | 58 | 0 |
| 106729 | P0098838 | 2013-05-23 | 4 | 1 |
| 106729 | P0098838 | 2013-06-03 | 1 | 0 |
| 107038 | P0099147 | 2013-06-25 | 58 | 0 |
| 107038 | P0099147 | 2013-06-28 | 4 | 1 |
| 107038 | P0099147 | 2013-07-04 | 1 | 0 |
| 107038 | P0099147 | 2013-08-13 | 58 | 0 |
| 107038 | P0099147 | 2013-08-15 | 4 | 6 |
| 107038 | P0099147 | 4000-01-01 | 6 | 0 |
| 107082 | P0099191 | 2013-06-29 | 58 | 0 |
| 107082 | P0099191 | 2013-07-04 | 4 | 6 |
| 107082 | P0099191 | 2013-07-19 | 6 | 0 |
| 107157 | P0099267 | 4000-01-01 | 13 | 0 |
| 107336 | P0099446 | 4000-01-01 | 6 | 0 |
+------------------+-------------------------------+------------------+------------------+-----------------------+
thanks.
It is hard to understand exactly what all your rules are from the question, but the general approach should be to add a "Grouping" column to a singl query that uses a CASE statement to categorize the people.
The conditions in a CASE are evaluated in order, so that if the first criteria is met, then the subsequent criteria are not even evaluated for that row.
Here is some code to get you started....
select t1.refid_eClinibase
,t1.[dthrfinmouvement]
,t1.[unite_service_id]
,t1.[unite_service_suiv_id]
CASE WHEN [dthrfinmouvement] = '4000-01-01' THEN 'Group1 Label'
WHEN condition2 = something THEN 'Group2 Label'
....
WHEN conditionN = something THEN 'GroupN Label'
ELSE 'Catch All Label'
END as person_category
from sometable t1