Hive Summing up data in the table based on the date range - sql

Have a table with the following schema design and the data residing inside it is like:
ID HITS MISS DDATE
1 10 3 20180101
1 33 21 20180122
1 84 11 20180901
1 11 2 20180405
1 54 23 20190203
1 33 43 20190102
4 54 22 20170305
4 56 88 20180115
5 87 22 20180809
5 66 48 20180617
5 91 53 20170606
DataTypes:
ID INT
HITS INT
MISS INT
DDATE STRING
The requirement is to calculate the total of the given (HITS and MISS) on yearly basis i.e 2017,2018,2019...
Written the following query:
SELECT ID,
SUM(HITS) AS HITS,SUM(MISS) AS MISS,
CASE
WHEN DDATE BETWEEN '201701' AND '201712' THEN '2017' ELSE
'NOTHING' END AS TTL_YR17_DATA
CASE
WHEN DDATE BETWEEN '201801' AND '201812' THEN '2018' ELSE
'NOTHING' END AS TTL_YR18_DATA
CASE
WHEN DDATE BETWEEN '201901' AND '201912' THEN '2019' ELSE
'NOTHING' END AS TTL_YR19_DATA
FROM
HST_TABLE
WHERE
DDATE BETWEEN '201801' AND '201812'
GROUP BY
ID,DDATE;
But, the query is not fetching the expected result.
Actual O/P:
1 10 3 2018
1 33 21 2018
1 84 11 2018
1 11 2 2018
1 54 23 2019
1 33 43 2019
4 54 22 2017
4 56 88 2018
5 87 22 2018
5 66 48 2018
5 91 53 2017
Expected O/P:
1 138 37 2018
4 56 88 2018
5 153 70 2018
1 87 66 2019
5 91 53 2017
Another related question:
Is there a way that I can avoid passing the DDATE range in the query? As this should be given by the user and shouldn't be hardcoded.
Any help/advice to achieve the above two requirements will be really helpful.

OK,it's easy to implement this with the substring function in HIVE, as below:
select
substring(dddate,0,4) as the_year,
id,
sum(hits) as hits_num,
sum(miss) as miss_num
from
hst_table
group by
substring(dddate,0,4),
id
order by
the_year,
id

The answer above by #Shawn.X is correct but has a logical flaw. Below is the corrected one:
select
substring(ddate,0,4) as the_year,
id,
sum(hits) as hits_num,
sum(miss) as miss_num
from
hst_table
group by
substring(ddate,0,4),
id
order by
the_year,
id;

Related

How to join two tables based on conditon in sql?

I need to Join two table with respective to two columns
Stu_id (Table 1) - Stu_id (Table 2)
Perf_yr(Table 1) - yr_month (Table 2)
perf_yr starts on every year Sept to Aug.
Perf_yr should match the yr_month based on Perf_yr start and end Month
Table 1
Stu_id Roll_No Avg_marks Perf_yr
1 100244 72 2017
2 200255 62 2018
3 100246 68 2019
Table 2
Stu_id Subject Marks yr_month
1 Maths 70 201609
1 Science 69 201701
1 Social 74 201712
2 Maths 60 201709
2 Science 61 201801
2 Social 62 201808
3 Maths 65 201810
3 Science 64 201912
3 Social 72 201902
Output
Stu_id Roll_No Avg_marks Perf_yr Subject Marks yr_month
1 100244 72 2017 Maths 70 201609
1 100244 72 2017 Science 70 201701
2 200255 62 2018 Maths 60 201709
2 200255 62 2018 Science 61 201801
2 200255 62 2018 Social 62 201808
3 100246 68 2019 Maths 65 201810
3 100246 68 2019 Science 64 201912
3 100246 68 2019 Social 72 201902
I TRIED :
SELECT A.*, B.* FROM
(SELECT * FROM TABLE1 )A
LEFT JOIN
(SELECT * FROM TABLE)B
ON
A.Stu_id = B.Stu_id
AND
A.Perf_yr = B.Yr_Month
BUT IT WONT GIVE THE DESIRED RESULT BECAUSE THE CONDITION IS NOT SATISFYING THE PERF YR START AND END DATE .
You need to parse eg 201609 as a date, add 4 months to it, then match it to the year from the other table. Adding 4 months converts a date range 201609-201708 into being 201701-201712 - we only care about the year part:
SELECT * FROM
t1
INNER JOIN t2
ON
t1.stu_id = t2.stu_id AND
t1.Perf_yr = EXTRACT(year FROM ADD_MONTHS(TO_DATE(t2.yr_month, 'YYYYMM'), 4))
This is oracle. The same logic will work for SQLS, you'll just need to adjust the functions used-
CONVERT(date, t2.yr_month+'01', 112) -- Convert yyyymmdd to a date
DATEADD(month, x, 4) -- add 4 months to x
YEAR(x) -- extract year from date x

Aggregate result from query by quarter SQL

Lets say I have a table which holds all exports for some time back in Microsoft SQL database:
Name:
ExportTable
Columns:
id - numeric(18)
exportdate - datetime
In order to get the number of exports per week I can run the following query:
SELECT DATEPART(ISO_WEEK,[exportdate]) as 'exportdate', count(exportdate) as 'totalExports'
FROM [ExportTable]
Group By DATEPART(ISO_WEEK,[exportdate])
order by exportdate;
Returns:
exportdate totalExports
---------- ------------
27 13
28 12
29 15
30 8
31 17
32 10
33 7
34 15
35 4
36 18
37 10
38 14
39 14
40 21
41 19
Would it be possible to aggregate the week results by quarter so the output becomes something like the bellow?
UPDATE
Sorry for not being crystal clear, I would like the current result to add upp with previous result up to a new quarter.
Note week 41 contains 21+19 = 40
Week 39 contains 157 (13+12+15+8+17+10+7+15+4+18+10+14+14)
exportdate totalExports Quarter
---------- ------------ -------
27 13 3
28 25 3
29 40 3
30 48 3
31 65 3
32 75 3
33 82 3
34 97 3
35 101 3
36 119 3
37 129 3
38 143 3
39 157 3 -- Sum of 3 Quarter values.
40 21 4 -- New Quarter show current week value
41 40 4 -- (21+19)
You can use this.
SELECT
DATEPART(ISO_WEEK,[exportdate]) as 'exportdate'
, SUM( count(exportdate) ) OVER ( PARTITION BY DATEPART(QUARTER,MIN([exportdate])) ORDER BY DATEPART(ISO_WEEK,[exportdate]) ROWS UNBOUNDED PRECEDING ) as 'totalExports'
, DATEPART(QUARTER,MIN([exportdate])) [Quarter]
FROM [ExportTable]
Group By DATEPART(ISO_WEEK,[exportdate])
order by exportdate;
You could use a case statement to separate the dates into quarters.
e.g.
CASE
WHEN EXPORT_DATE BETWEEN '1' AND '4' THEN 1
WHEN Export_Date BETWEEN '5' and '9' THEN 2
ELSE 0 AS [Quarter]
END
Its just an example but you get the idea.
You could then use the alias from the case
SELECT DATEPART(ISO_WEEK,[exportdate]) as 'exportdate', count(exportdate) as 'totalExports', DATEPART(quarter,[exportdate]) as quarter FROM [ExportTable] Group By DATEPART(ISO_WEEK,[exportdate]), DATEPART(quarter,[exportdate]) order by exportdate;

Custom sorting by month name in SQL Server

I have a table where for some dates a certain number of entries are placed. Here is the table structure :
ID EntryName Entries DateOfEntry
1 A 20 2016-01-17
2 B 22 2016-01-29
3 C 23 2016-02-17
4 D 19 2016-02-17
5 E 29 2016-03-17
6 F 30 2016-03-17
7 G 43 2016-04-17
8 H 10 2016-04-17
9 I 5 2016-05-17
10 J 120 2016-05-17
11 K 220 2016-06-17
12 L 210 2016-06-17
13 M 10 2016-07-17
14 N 20 2016-07-17
15 O 15 2016-08-17
16 P 17 2016-08-17
17 Q 19 2016-09-17
18 R 23 2016-09-17
19 S 43 2016-10-17
20 T 56 2016-10-17
21 U 65 2016-11-17
22 V 78 2016-11-17
23 W 12 2016-12-17
24 X 23 2016-12-17
25 Y 43 2016-02-17
26 Z 67 2016-03-17
27 AA 35 2015-01-17
28 AB 23 2015-01-29
29 AC 43 2015-02-17
30 AD 35 2015-02-17
31 AE 45 2015-03-17
32 AF 23 2015-03-17
33 AG 43 2015-04-17
34 AH 19 2015-04-17
35 AI 21 2015-05-17
36 AJ 13 2015-05-17
37 AK 22 2015-06-17
38 AL 45 2015-06-17
39 AM 66 2015-07-17
40 AN 77 2015-07-17
41 AO 89 2015-08-17
42 AP 127 2015-08-17
43 AQ 19 2015-09-17
44 AR 223 2015-09-17
45 AS 143 2015-10-17
46 AT 36 2015-10-17
47 AU 45 2015-11-17
48 AV 28 2015-11-17
49 AW 72 2015-12-17
50 AX 24 2015-12-17
51 AY 46 2015-02-17
52 AZ 62 2015-03-17
The column EntryName is the entry identifier, the column Entries has the total number of entries for the date specified in the column DateOfEntry.
I am trying to formulate a query where the total number of entries are displayed on a month-wise basis. I currently have this query :
SELECT DateName(MONTH, e.DateOfEntry) AS MonthOfEntry,
MONTH(e.DateOfEntry) AS MonthNumber,
SUM(e.Entries) AS TotalEntries
FROM #entry e
GROUP BY MONTH(e.DateOfEntry), DateName(MONTH,e.DateOfEntry)
ORDER BY MONTH(e.DateOfEntry) ASC
which works fine as far as displaying the results are concerned. However, my issue here is that I need to sort the results on a month-wise basis where the starting month would be dynamic i.e. arising from a parameter (supplied by the user).
This means that if the user selects May of 2015 the results should be sorted from May 2015 to April 2016. Similarly, if the user selects October 2015, the results would be displayed from October 2015 to September 2016.
How would I go about getting this condition within the ORDER BY clause ?
You can put an offset into the ORDER BY using modulo arithmetic. For April:
ORDER BY (MONTH(e.DateOfEntry) + 12 - 4) % 12
--------------------------------------^ month number to start with
(The + 12 is simply so I don't have to remember if % returns negative numbers with negative operands.)
If you want the results chronologically, you can instead do:
ORDER BY MIN(e.DateOfEntry)
You could use the belosw in order by
ORDER BY YEAR(e.DATEOFENTRY),
DATEPART(MM,e.DAREOFENTRY)
This will sort the result first for Year and next for month.
Here you need to specify these same columns in Select.
If I understood you correctly
"This means that if the user selects May of 2015 the results should be sorted from May 2015 to April 2016. Similarly, if the user selects October 2015, the results would be displayed from October 2015 to September 2016."
this should work:
SAMPLE DATA:
IF OBJECT_ID('tempdb..#entry') IS NOT NULL
DROP TABLE #entry;
CREATE TABLE #entry(ID INT ,EntryName VARCHAR(10) , Entries INT , DateOfEntry DATE);
INSERT INTO #entry (ID ,EntryName ,Entries ,DateOfEntry)
VALUES
(1 ,'A', 20 ,'2016-01-17'),
(2 ,'B', 22 ,'2016-01-29'),
(3 ,'C', 23 ,'2016-02-17'),
(4 ,'D', 19 ,'2016-02-17'),
(5 ,'E', 29 ,'2016-03-17'),
(6 ,'F', 30 ,'2016-03-17'),
(7 ,'G', 43 ,'2016-04-17'),
(8 ,'H', 10 ,'2016-04-17'),
(9 ,'I', 5 ,'2016-05-17'),
(10,'J', 120 ,'2016-05-17'),
(11,'K', 220 ,'2016-06-17'),
(12,'L', 210 ,'2016-06-17'),
(13,'M', 10 ,'2016-07-17'),
(14,'N', 20 ,'2016-07-17'),
(15,'O', 15 ,'2016-08-17'),
(16,'P', 17 ,'2016-08-17'),
(17,'Q', 19 ,'2016-09-17'),
(18,'R', 23 ,'2016-09-17'),
(19,'S', 43 ,'2016-10-17'),
(20,'T', 56 ,'2016-10-17'),
(21,'U', 65 ,'2016-11-17'),
(22,'V', 78 ,'2016-11-17'),
(23,'W', 12 ,'2016-12-17'),
(24,'X', 23 ,'2016-12-17'),
(25,'Y', 43 ,'2016-02-17'),
(26,'Z', 67 ,'2016-03-17'),
(27,'AA',35 ,'2015-01-17'),
(28,'AB',23 ,'2015-01-29'),
(29,'AC',43 ,'2015-02-17'),
(30,'AD',35 ,'2015-02-17'),
(31,'AE',45 ,'2015-03-17'),
(32,'AF',23 ,'2015-03-17'),
(33,'AG',43 ,'2015-04-17'),
(34,'AH',19 ,'2015-04-17'),
(35,'AI',21 ,'2015-05-17'),
(36,'AJ',13 ,'2015-05-17'),
(37,'AK',22 ,'2015-06-17'),
(38,'AL',45 ,'2015-06-17'),
(39,'AM',66 ,'2015-07-17'),
(40,'AN',77 ,'2015-07-17'),
(41,'AO',89 ,'2015-08-17'),
(42,'AP',127 ,'2015-08-17'),
(43,'AQ',19 ,'2015-09-17'),
(44,'AR',223 ,'2015-09-17'),
(45,'AS',143 ,'2015-10-17'),
(46,'AT',36 ,'2015-10-17'),
(47,'AU',45 ,'2015-11-17'),
(48,'AV',28 ,'2015-11-17'),
(49,'AW',72 ,'2015-12-17'),
(50,'AX',24 ,'2015-12-17'),
(51,'AY',46 ,'2015-02-17'),
(52,'AZ',62 ,'2015-03-17')
QUERY WITH PARAMS:
DECLARE #Month VARCHAR(2) = '05', #Year VARCHAR(4) = '2015'
SELECT DateName(MONTH, e.DateOfEntry) AS MonthOfEntry,
MONTH(e.DateOfEntry) AS MonthNumber,
SUM(e.Entries) AS TotalEntries
FROM #entry e
WHERE CAST(e.DateOfEntry AS DATE) >= CAST( #Year+#Month+'01' AS DATE)
GROUP BY MONTH(e.DateOfEntry), DateName(MONTH,e.DateOfEntry)
ORDER BY MONTH(e.DateOfEntry) ASC
RESULTS:
add a where clause for the query
WHERE MONTH(e.DateOfEntry) < User.Month AND YEAR(e.DateOfEntry) < User.Year AND MONTH(e.DateOfEntry) > (User.Month-1) AND YEAR(e.DateOfEntry) > (User.Year+1)
Assuming your parameter is an Integer called #FirstMonth, you could get the proper month order using:
Case
WHEN MONTH(e.DateOfEntry) < #FirstMonth then MONTH(e.DateOfEntry) + 12
ELSE MONTH(e.DateOfEntry)
END AS MonthNumber
Of all the answers and suggestions I have come across here, I find this way (suggested by xQbert in the question's comments) to be the simplest one :
SELECT DateName(MONTH, e.DateOfEntry) + ' ' + CONVERT(NVARCHAR(100), YEAR(e.DateOfEntry)) AS MonthOfEntry,
MONTH(e.DateOfEntry) AS MonthNumber,
SUM(e.Entries) AS TotalEntries
FROM Entry e
WHERE e.DateOfEntry BETWEEN #StartDate AND (DATEADD(YEAR, 1, #StartDate))
GROUP BY MONTH(e.DateOfEntry), DateName(MONTH,e.DateOfEntry), YEAR(e.DateOfEntry)
ORDER BY YEAR(e.DateOfEntry) ASC, MONTH(e.DateOfEntry) ASC
A fiddle to demonstrate this : http://rextester.com/CJFFP5640
Initially, I was using the following query :
SELECT sortingList.MonthOfEntry,
sortingList.TotalEntries,
sortingList.MonthNumber
FROM (
SELECT DateName(MONTH, DATEADD(MONTH, MONTH(e.DateOfEntry), 0) - 1) + ' ' + CONVERT(nvarchar(20),YEAR(e.DateOfEntry)) AS MonthOfEntry,
SUM(e.Entries) as TotalEntries,
CASE
WHEN ((MONTH(e.DateOfEntry) - MONTH(#StartDate)) > 0)
THEN (MONTH(e.DateOfEntry) - MONTH(#StartDate)) + 1
WHEN ((MONTH(e.DateOfEntry) - MONTH(#StartDate)) = 0)
THEN 1
ELSE
((12 - MONTH(#StartDate)) + (MONTH(e.DateOfEntry))) + 1
END
AS MonthNumber
FROM Entry e
WHERE e.DateOfEntry >= #StartDate AND e.DateOfEntry < DATEADD(YEAR, 1, #StartDate)
GROUP BY DateName(MONTH, DATEADD(MONTH, MONTH(e.DateOfEntry), 0) - 1), YEAR(e.DateOfEntry), MONTH(e.DateOfEntry) - MONTH(#StartDate), MONTH(e.DateOfEntry)
) sortingList
ORDER BY sortingList.MonthNumber ASC
Here's an Fiddle to demonstrate this : http://rextester.com/LEVD30653
Explanation (non TL;DR)
You can see that it's essentially the same WHERE clause. However, the query at the top uses much simpler logic for sorting and is more fluent and readable.
Do note that the second solution (using the CASE statement) sorts the month numbers as per the user-provided month number i.e. if the user provides December 2015, then the second solution will number Dec 2015 as 1, January 2016 as 2, February 2016 as 3 and so on and so forth. This might be more beneficial in cases where you want to work on top of this data.
As far as my use-case is concerned, this makes more sense. However, as far as the scope of the question is concerned, the query at the top is the best one.

T-SQL Group by day date but i want show query full date

I want to show the date field can not group.
My Query:
SELECT DAY(T1.UI_CreateDate) AS DATEDAY, SUM(1) AS TOTALCOUNT
FROM mydb.dbo.LP_UseImpression T1 WHERE T1.UI_BR_BO_ID = 45
GROUP BY DAY(T1.UI_CreateDate)
Result:
DATEDAY TOTALCOUNT
----------- -----------
15 186
9 1
3 2
26 481
21 297
27 342
18 18
30 14
4 183
25 553
13 8
22 469
16 1
17 28
20 331
28 90
14 33
8 1
But i want to show the full date...
Example result:
DATEDAY TOTALCOUNT
----------- -----------
15/06/2015 186
9/06/2015 1
3/06/2015 2
26/06/2015 481
21/06/2015 297
27/06/2015 342
18/06/2015 18
30/06/2015 14
4/06/2015 183
25/06/2015 553
13/06/2015 8
22/06/2015 469
16/06/2015 1
17/06/2015 28
20/06/2015 331
28/06/2015 90
14/06/2015 33
8/06/2015 1
I want to see the results...
I could not get a kind of results...
How can I do?
Thanx!
How about just casting to date to remove any time component:
SELECT CAST(T1.UI_CreateDate as DATE) AS DATEDAY, COUNT(*) AS TOTALCOUNT
FROM mydb.dbo.LP_UseImpression T1
WHERE T1.UI_BR_BO_ID = 45
GROUP BY CAST(T1.UI_CreateDate as DATE)
ORDER BY DATEDAY;
SUM(1) for calculating the count does work. However, because SQL has the COUNT(*) function, it seems a bit awkward.
So you can group by DAY(T1.UI_CreateDate) or use full date for grouping. But these are different . As both these dates '2015-04-15' and '2015-12-15' result in same DAY value of 15.
Assuming you want to group on DAY rather than date please try the below version of query:
SELECT DISTINCT
T1.UI_CreateDate as DATEDAY,
count(1) over (PARTITION BY DAY(T1.UI_CreateDate) ) AS TOTALCOUNT
FROM mydb.dbo.LP_UseImpression T1 WHERE T1.UI_BR_BO_ID = 45
sql fiddle for demo: http://sqlfiddle.com/#!6/c3337/1

How to verify whether records exist for the last x days (calendar days) in SQL not using the between key word

Want verify whether my table is having the records for the last 6 consecutive days in SQL
SNO FLIGHT_DATE LANDINGS
45 9/1/2013 1
31 10/1/2013 1
32 11/1/2013 1
30 11/24/2013 1
27 11/25/2013 1
28 11/26/2013 1
29 11/26/2013 1
33 11/26/2013 1
26 11/30/2013 1
25 12/1/2013 1
34 12/1/2013 1
24 12/2/2013 1
35 12/3/2013 1
36 12/3/2013 1
44 12/4/2013 1
46 12/6/2013 1
47 12/6/2013 1
Is this what you want?
SELECT
*
FROM
Table1
WHERE
FLIGHT_DATE > dateadd(day,-6,datediff(day,0,getdate()))
AND
FLIGHT_DATE < GETDATE();
SQL FIDDLE