I have this SQL statement that generates the revenue, the total customer and revenue for new business.
I would like to amend my SQL to allow it to show the same result but also run it for the previous 3 years with the same logic.
One option is to use a union and amend every time the current_date with DATEADD function by -1 , -2 etc. But this would be so inefficient.
Is there a better way to amend the code? with a date dimension maybe?
select date_trunc('month',current_date),
COUNT(DISTINCT CASE WHEN (case when RELEVANT_DATE_OUTBOUND> current_date then TOTAL_REVENUE end)>0 THEN CUSTOMER_NAME end) CUSTOMER_ID,
SUM(CASE WHEN (case when date_trunc('month',reporting_date) = date_trunc('month',current_date) then NB_EUR end)>0 THEN nb_eur end) nb_eur
from REVENUE_DATABASE_AGR_VIEW
Please find attached the dataset and result of the SQL and desired outcome.
dataset:
results:
desired outcome:
I would suggest that you create a date_dim table and use that to provide the date values to join with your query as the date parameter.
You could also use a loop, but row based processing is pretty non-performant in Snowflake.
However, if you're only running it once to populate the prior 3 years, and then once per day only to do each new day's data, then you could wrap your existing code in a loop like the following to process all the historical data, and then use your query with just the current_date()-1 and run daily prospectively:
create
or replace table date_dim (loop_date varchar);
declare ctr number(4, 0);
begin ctr: = 0;
while (ctr <= 1095) do -- create date for prior 3 years
insert into
date_dim
select
current_date() - :ctr;
ctr: = ctr + 1;
end while;
end;
Related
I am looking for a way to write an SQL statement that selects data for each month of the year, separately.
In the SQL statement below, I am trying to count the number of instances in the TOTAL_PRECIP_IN and TOTAL_SNOWFALL_IN columns when either column is greater than 0. In my data table, I have information for those two columns ("TOTAL_PRECIP_IN" and "TOTAL_SNOWFALL_IN") for each day of the year (365 total entries).
I want to break up my data by each calendar month, but am not sure of the best way to do this. In the statement below, I am using a UNION statement to break up the months of January and February. If I keep using UNION statements for the remaining months of the year, I can get the answer I am looking for. However, using 11 different UNION statements cannot be the optimal solution.
Can anyone give me a suggestion how I can edit my SQL statement to measure from the first day of the month, to the last day of the month for every month of the year?
select monthname(OBSERVATION_DATE) as "Month", sum(case when TOTAL_PRECIP_IN or TOTAL_SNOWFALL_IN > 0 then 1 else 0 end) AS "Days of Rain" from EMP_BASIC
where OBSERVATION_DATE between '2019-01-01' and '2019-01-31'
and CITY = 'Olympia'
group by "Month"
UNION
select monthname(OBSERVATION_DATE) as "Month", sum(case when TOTAL_PRECIP_IN or TOTAL_SNOWFALL_IN > 0 then 1 else 0 end) from EMP_BASIC
where OBSERVATION_DATE between '2019-02-01' and '2019-02-28'
and CITY = 'Olympia'
group by "Month"```
Your table structure is too unclear to tell you the exact query you will need. But a general easy idea is to build the sum of your value and then group by monthname and/or by month. Sice you wrote you only want sum values greater 0, you can just put this condition in the where clause. So your query will be something like this:
SELECT MONTHNAME(yourdate) AS month,
MONTH(yourdate) AS monthnr,
SUM(yourvalue) AS yoursum
FROM yourtable
WHERE yourvalue > 0
GROUP BY MONTHNAME(yourdate), MONTH(yourdate)
ORDER BY MONTH(yourdate);
I created an example here: db<>fiddle
You might need to modify this general construct for your concrete purpose (maybe take care of different years, of NULL values etc.). And note this is an example for a MYSQL DB because you wrote about MONTHNAME() which is in most cases used in MYSQL databases. If you are using another DB type, maybe you need to do some modifications. To make sure that answers match your DB type, tag it in your question, please.
I'm newbie to Hivesql.
I have a raw table with 6 million records like this:
I want to count the number of IP_address access to each Modem_id everyweek.
The result table I want will be like this:
I did it with left join, and it worked. But since using join will be time-consuming, I want do it with case when statement - but I can't write a correct statement. Do you have any ideas?
This is the join statement I used:
select a.modem_id,
a.Number_of_IP_in_Day_1,
b.Number_of_IP_in_Day_2
from
(select modem_id,
count(distinct ip_address) as Number_of_IP_in_Day_1
from F_ACS_DEVICE_INFORMATION_NEW
where day=1
group by modem_id) a
left join
(select modem_id,
count(distinct param_value) as Number_of_IP_in_Day_2
from F_ACS_DEVICE_INFORMATION_NEW
where day=2
group by modem_id) b
on a.modem_id= b.modem_id;
You can express your logic using just aggregatoin:
select a.modem_id,
count(distinct case when date = 1 then ip_address end) as day_1,
count(distinct case when date = 2 then ip_address end) as day_2
from F_ACS_DEVICE_INFORMATION_NEW a
group by a.modem_id;
You can obviously extend this for more days.
Note: As your question and code are written, this assumes that your base table has data for only one week. Otherwise, I would expect some date filtering. Presumably, that is what the _NEW suffix means on the table name.
Based on your question and further comments, you would like
The number of different IP addresses accessed by each modem
In counts by week (as columns) for 4 weeks
e.g., result would be 5 columns
modem_id
IPs_accessed_week1
IPs_accessed_week2
IPs_accessed_week3
IPs_accessed_week4
My answer here is based on knowledge of SQL - I haven't used Hive but it appears to support the things I use (e.g., CTEs). You may need to tweak the answer a bit.
The first key step is to turn the day_number into a week_number. A straightforward way to do this is FLOOR((day_num-1)/7)+1 so days 1-7 become week 1, days 8-14 become week2, etc.
Note - it is up to you to make sure the day_nums are correct. I would guess you'd actually want info the the last 4 weeks, not the first four weeks of data - and as such you'd probably calculate the day_num as something like SELECT DATEDIFF(day, IP_access_date, CAST(getdate() AS date)) - whatever the equivalent is in Hive.
There are a few ways to do this - I think the clearest is to use a CTE to convert your dataset to what you need e.g.,
convert day_nums to weeknums
get rid of duplicates within the week (your code has COUNT(DISTINCT ...) - I assume this is what you want) - I'm doing this with SELECT DISTINCT (rather than grouping by all fields)
From there, you could PIVOT the data to get it into your table, or just use SUM of CASE statements. I'll use SUM of CASE here as I think it's clearer to understand.
WITH IPs_per_week AS
(SELECT DISTINCT
modem_id,
ip_address,
FLOOR((day-1)/7)+1 AS week_num -- Note I've referred to it as day_num in text for clarity
FROM F_ACS_DEVICE_INFORMATION_NEW
)
SELECT modem_id,
SUM(CASE WHEN week_num = 1 THEN 1 ELSE 0 END) AS IPs_access_week1,
SUM(CASE WHEN week_num = 2 THEN 1 ELSE 0 END) AS IPs_access_week2,
SUM(CASE WHEN week_num = 3 THEN 1 ELSE 0 END) AS IPs_access_week3,
SUM(CASE WHEN week_num = 4 THEN 1 ELSE 0 END) AS IPs_access_week4
FROM IPs_per_week
GROUP BY modem_id;
While fetching count from table by using following query
Select count(*)
from tab
where tdate = '17-05-19' ---> output 0
or
Select count(*)
from tab
where trunc(tdate) = '17-05-19' ---->output 0
If I use:
Select count(*)
from tab
where tdate >sysdate - 1 ---> it returns some count(yesterday+some of the today txn)
But here I want only yesterday txn whenever I fire this query.
But here I want only yesterday txn whenever I fire this query.
You may use this.
Select count (*) from tab where
tdate >= TRUNC(SYSDATE) - 1
AND tdate < TRUNC(SYSDATE)
The advantage of this over using TRUNC on the date column is that it will utilize an index if it exists over tdate
If you tried by using
Select count(*) from tab where trunc(tdate) = date'2019-05-17'
(or, you could use
Select count(*) from tab where to_char(tdate,'dd-mm-yy') = '17-05-19' by formatting through to_char function
or, you could use
Select count(*) from tab where trunc(tdate) = trunc(sysdate)-1 to get only the data for the day before
)
you'd get some results provided you have data for the date 17th May.
So, you need to provide a formatting for your literal as date'2019-05-17'(known as date literal) especially for Oracle DB, it might be used as '2019-05-17' without date part in MySQL as an example.
Btw, trunc function is used to extract the date portion, and remove the time part of a date type column value.
If your table is populated with huge data, therefore performance may matter, then you can even create functional index on trunc(tdate).
Demo
I have a dataset that has product name, order number and the time order was placed.
prod_name,order_no,order_time
a,101,2018-05-01
a,102,2018-06-04
a,103,2018-05-03
b,104,2018-01-21
b,105,2018-01-11
I am trying to build a report that shows time since first order (compared against current time) with an output as below:
prod_name,time_since_first_sale,aging
a,64,Less than 3 months back
b,177,Less than 6 months back
Given below is the SQL I am using:
select DISTINCT b.prod_name,case when((CURRENT_TIMESTAMP - min(a.order_time))) < '90' THEN 'Less than 3 months'
when ((CURRENT_TIMESTAMP - min(order_time))) < '180' THEN 'Less than 6 months'
else 'Other' end as aging
from sales a, prod b where a.id=b.prod_id;
The above SQL when executed returns duplicates, believe it also considers each sale_id in the sales table. How could I modify the above query to get just one record per prod_name. If I however remove the case statement the duplicates are not there. Could any one assist as to what I am doing wrong that pulls in these duplicates.
I am using Amazon Redshift DB.
Thanks..
Never use commas in the FROM clause. Always use proper, explicit, standard JOIN syntax.
Don't use SELECT DISTINCT when you intend GROUP BY.
So your query should look like:
select p.prod_name,
(case when CURRENT_TIMESTAMP - min(s.order_time) < '90'
then 'Less than 3 months'
when CURRENT_TIMESTAMP - min(s.order_time) < '180' then 'Less than 6 months'
else 'Other'
end) as aging
from sales s join
prod p
on s.id = p.prod_id
group by p.prod_name;
Notice that I also added in reasonable table aliases (abbreviations for the table names) and qualified all column references.
So I have created a table that has the following columns from a transaction table with all customer purchase records:
1. Month-Year, 2.Customer ID, 3. Number of Transactions in that month.
I'm trying to create a table that has the output of
1. Month-Year, 2. Number of active customers defined by having at least 1 purchase in the previous year.
The code that I have currently is this but the case when obviously only capturing one date and the where clause isn't dynamic. Would really appreciate your help.
select month_start_date, cust_ID,
(case when month_start_Date between date and add_months(date, -12) then count(cust_ID) else 0 end) as active
from myserver.mytable
where
month_start_Date>add_months(month_start_date,-12)
group by 1,2
EDIT: I'm just trying to put a flag next to a customer if they are active in each month defined as having at least one transaction in the last year thanks!
You might use Teradata's proprietary EXPAND ON synax for creating time series:
SELECT month_start_date, COUNT(*)
FROM
( -- create one row for every month within the next year
-- after a customer's transaction
SELECT DISTINCT
BEGIN(pd) AS month_start_date,
cust_ID
FROM myserver.mytable
EXPAND ON PERIOD(month_start_date, ADD_MONTHS(month_start_date,12)) AS pd
BY ANCHOR MONTH_BEGIN -- every 1st of month
FOR PERIOD (DATE - 500, DATE) -- use this to restrict to a specific date range
) AS dt
GROUP BY month_start_date
ORDER BY month_start_date