Related
I have a table in GBQ in the following format :
UserId Orders Month
XDT 23 1
XDT 0 4
FKR 3 6
GHR 23 4
... ... ...
It shows the number of orders per user and month.
I want to calculate the percentage of users who have orders, I did it as following :
SELECT
HasOrders,
ROUND(COUNT(*) * 100 / CAST( SUM(COUNT(*)) OVER () AS float64), 2) Parts
FROM (
SELECT
*,
CASE WHEN Orders = 0 THEN 0 ELSE 1 END AS HasOrders
FROM `Table` )
GROUP BY
HasOrders
ORDER BY
Parts
It gives me the following result:
HasOrders Parts
0 35
1 65
I need to calculate the percentage of users who have orders, by month, in a way that every month = 100%
Currently to do this I execute the query once per month, which is not practical :
SELECT
HasOrders,
ROUND(COUNT(*) * 100 / CAST( SUM(COUNT(*)) OVER () AS float64), 2) Parts
FROM (
SELECT
*,
CASE WHEN Orders = 0 THEN 0 ELSE 1 END AS HasOrders
FROM `Table` )
WHERE Month = 1
GROUP BY
HasOrders
ORDER BY
Parts
Is there a way execute a query once and have this result ?
HasOrders Parts Month
0 25 1
1 75 1
0 45 2
1 55 2
... ... ...
SELECT
SIGN(Orders),
ROUND(COUNT(*) * 100.000 / SUM(COUNT(*), 2) OVER (PARTITION BY Month)) AS Parts,
Month
FROM T
GROUP BY Month, SIGN(Orders)
ORDER BY Month, SIGN(Orders)
Demo on Postgres:
https://dbfiddle.uk/?rdbms=postgres_10&fiddle=4cd2d1455673469c2dfc060eccea8020
You've stated that it's important for the total to be 100% so you might consider rounding down in the case of no orders and rounding up in the case of has orders for those scenarios where the percentages falls precisely on an odd multiple of 0.5%. Or perhaps rounding toward even or round smallest down would be better options:
WITH DATA AS (
SELECT SIGN(Orders) AS HasOrders, Month,
COUNT(*) * 10000.000 / SUM(COUNT(*)) OVER (PARTITION BY Month) AS PartsPercent
FROM T
GROUP BY Month, SIGN(Orders)
ORDER BY Month, SIGN(Orders)
)
select HasOrders, Month, PartsPercent,
PartsPercent - TRUNCATE(PartsPercent) AS Fraction,
CASE WHEN HasOrders = 0
THEN FLOOR(PartsPercent) ELSE CEILING(PartsPercent)
END AS PartsRound0Down,
CASE WHEN PartsPercent - TRUNCATE(PartsPercent) = 0.5
AND MOD(TRUNCATE(PartsPercent), 2) = 0
THEN FLOOR(PartsPercent) ELSE ROUND(PartsPercent) -- halfway up
END AS PartsRoundTowardEven,
CASE WHEN PartsPercent - TRUNCATE(PartsPercent) = 0.5 AND PartsPercent < 50
THEN FLOOR(PartsPercent) ELSE ROUND(PartsPercent) -- halfway up
END AS PartsSmallestTowardZero
from DATA
It's usually not advisable to test floating-point values for equality and I don't know how BigQuery's float64 will work with the comparison against 0.5. One half is nevertheless representable in binary. See these in a case where the breakout is 101 vs 99. I don't have immediate access to BigQuery so be aware that Postgres's rounding behavior is different:
https://dbfiddle.uk/?rdbms=postgres_10&fiddle=c8237e272427a0d1114c3d8056a01a09
Consider below approach
select hasOrders, round(100 * parts, 2) as parts, month from (
select month,
countif(orders = 0) / count(*) `0`,
countif(orders > 0) / count(*) `1`,
from your_table
group by month
)
unpivot (parts for hasOrders in (`0`, `1`))
with output like below
I have a table with the following columns:
account, validity_date,validity_month,amount.
For each row i want to check if the value in field "amount' exist over the rows range of the next month. if yes, indicator=1, else 0.
account validity_date validity_month amount **required_column**
------- ------------- --------------- ------- ----------------
123 15oct2019 201910 400 0
123 20oct2019 201910 500 1
123 15nov2019 201911 1000 0
123 20nov2019 201911 500 0
123 20nov2019 201911 2000 1
123 15dec2019 201912 400
123 15dec2019 201912 2000
Can anyone help?
Thanks
validity_month/100*12 + validity_month MOD 100 calculates a month number (for comparing across years, Jan to previous Dec) and the inner ROW_NUMBER reduces multiple rows with the same amount per month to a single row (kind of DISTINCT):
SELECT dt.*
,CASE -- next row is from next month
WHEN Lead(nextMonth IGNORE NULLS)
Over (PARTITION BY account, amount
ORDER BY validity_date)
= (validity_month/100*12 + validity_month MOD 100) +1
THEN 1
ELSE 0
END
FROM
(
SELECT t.*
,CASE -- one row per account/month/amount
WHEN Row_Number()
Over (PARTITION BY account, amount, validity_month
ORDER BY validity_date ) = 1
THEN validity_month/100*12 + validity_month MOD 100
END AS nextMonth
FROM tab AS t
) AS dt
Edit:
The previous is for exact matching amounts, for a range match the query is probably very hard to write with OLAP-functions, but easy with a Correlated Subquery:
SELECT t.*
,CASE
WHEN
( -- check if there's a row in the next month matching the current amount +/- 10 percent
SELECT Count(*)
FROM tab AS t2
WHERE t2.account_ = t.account_
AND (t2.validity_month/100*12 + t2.validity_month MOD 100)
= ( t.validity_month/100*12 + t.validity_month MOD 100) +1
AND t2.amount BETWEEN t.amount * 0.9 AND t.amount * 1.1
) > 0
THEN 1
ELSE 0
END
FROM tab AS t
But then performance might be really bad...
Assuming the values are unique within a month and you have a value for each month for each account, you can simplify this to:
select t.*,
(case when lead(seqnum) over (partition by account, amount order by validity_month) = seqnum + 1
then 1 else 0
end)
from (select t.*,
dense_rank() over (partition by account order by validity_month) as seqnum
from t
) t;
Note: This puts 0 for the last month rather than NULL, but that can easily be adjusted.
You can do this without the subquery by using month arithmetic. It is not clear what the data type of validity_month is. If I assume a number:
select t.*,
(case when lead(floor(validity_month / 100) * 12 + (validity_month mod 100)
) over (partition by account, amount order by validity_month) =
(validity_month / 100) * 12 + (validity_month mod 100) - 1
then 1 else 0
end)
from t;
Just to add another way to do this using Standard SQL. This query will return 1 when the condition is met, 0 when it is not, and null when there isn't a next month to evaluate (as implied in your result column).
It is assumed that we're partitioning on the account field. Also includes a 10% range match on the amount field based on the comment made. Note that if you have an id field, you should include it (if two rows have the same account, validity_date, validity_month, amount there will only be one resulting row, due to DISTINCT).
Performance-wise, should be similar to the answer from #dnoeth.
SELECT DISTINCT
t1.account,
t1.validity_date,
t1.validity_month,
t1.amount,
CASE
WHEN t2.amount IS NOT NULL THEN 1
WHEN MAX(t1.validity_month) OVER (PARTITION BY t1.account) > t1.validity_month THEN 0
ELSE NULL
END AS flag
FROM `project.dataset.table` t1
LEFT JOIN `project.dataset.table` t2
ON
t2.account = t1.account AND
DATE_DIFF(
PARSE_DATE("%Y%m", CAST(t2.validity_month AS STRING)),
PARSE_DATE("%Y%m", CAST(t1.validity_month AS STRING)),
MONTH
) = 1 AND
t2.amount BETWEEN t1.amount * 0.9 AND t1.amount * 1.1;
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 5 years ago.
Improve this question
I am trying to create a view for our firm to check the subdollar price.
I create a temp table in the view hoping to get the program running Faster. But it does not work. The query is still super slow... could anyone help me improve my query or do you have ant suggestion what caused the query running slow? Thanks.
This program is to implement:
- If the New Order violates the sub-penny rule
- If the Order Ack violates the sub-penny rule
- When the New Order and Order Ack price are not equal
•Only orders with correct number of decimals/rounding should be in the system – (on Order price)
/*
price < $ 0.01 6 decimal places
price< $1.00 4 decimal places
price>= $1.00 2 decimal places
price>= $100,000 1 decimal places
*/
WITH PRICE (newOrderPrice, NewOrderprice, newTransactionGUID, newMsgText,ackOrderPrice,AckOrderprice,ackTransactionGUID,ackMsgText)
AS (
SELECT
--remove the trailing 0 in price
REVERSE(SUBSTRING(reverse(NewOrderprice), PATINDEX('%[1-9.]%', reverse(NewOrderprice)), len(NewOrderprice) - PATINDEX('%[1-9.]%', reverse(NewOrderprice)) + 1)) AS newOrderPrice
, NewOrderprice
, newTransactionGUID
, newMsgText
, REVERSE(SUBSTRING(reverse(AckOrderprice), PATINDEX('%[1-9.]%', reverse(AckOrderprice)), len(AckOrderprice) - PATINDEX('%[1-9.]%', reverse(AckOrderprice)) + 1)) AS ackOrderPrice
, AckOrderprice
, ackTransactionGUID
, ackMsgText
FROM
(
SELECT --make sure we get the price from transactiontext tag 44 is only numbers
--------------------new Order----------------------------------------------------------
LEFT(SUBSTRING(LscDP.dbo.ParseTransactionText(44,t.MsgText), PATINDEX('%[0-9.]%', LscDP.dbo.ParseTransactionText(44,t.MsgText)), 8000),
PATINDEX('%[^0-9.]%', SUBSTRING(LscDP.dbo.ParseTransactionText(44,t.MsgText), PATINDEX('%[0-9.]%',
LscDP.dbo.ParseTransactionText(44,t.MsgText)), 8000) + 'X') -1)
AS NewOrderprice
, t.TransactionGUID AS newTransactionGUID
, t.MsgText AS newMsgText
-----------------ack order------------------------------------------------------------
,LEFT(SUBSTRING(LscDP.dbo.ParseTransactionText(44,ack.MsgText), PATINDEX('%[0-9.]%', LscDP.dbo.ParseTransactionText(44,ack.MsgText)), 8000),
PATINDEX('%[^0-9.]%', SUBSTRING(LscDP.dbo.ParseTransactionText(44,ack.MsgText), PATINDEX('%[0-9.]%',
LscDP.dbo.ParseTransactionText(44,ack.MsgText)), 8000) + 'X') -1)
AS AckOrderprice
, ack.TransactionGUID AS ackTransactionGUID
, ack.MsgText AS ackMsgText
FROM LscDP..DailyTransactionText t
INNER JOIN LscDP..DailyTransactionText ack
ON t.Sender = ack.Sender
AND t.Branch = ack.Branch
AND t.BranchSeq = ack.BranchSeq
and ack.MsgType = '1'
and t.MsgType = '0'
INNER JOIN LscDP..DailyOrders o
ON t.TransactionGUID = o.TransactionGUID
OR ack.TransactionGUID = o.TransactionGUID
LEFT JOIN AccountsEX..OutQueues oq
ON o.Route = oq.OutQueue
WHERE
t.Symbol not like '%-%' -- exclude foreign symbols
AND LscDP.dbo.ParseTransactionText(167,t.MsgText) <> 'OPT' --Exclude options
AND LscDP.dbo.ParseTransactionText(167,ack.MsgText) <> 'OPT' --Exclude options
AND ISNULL(oq.QueueType, '') NOT IN ('DC','CL')
AND LscDP.dbo.ParseTransactionText(44,t.MsgText) <> ''
AND LscDP.dbo.ParseTransactionText(44,ack.MsgText) <> ''
AND LscDP.dbo.ParseTransactionText(44,t.MsgText) like '%.%' --Only need verify the subdollar.
AND LscDP.dbo.ParseTransactionText(44,ack.MsgText) like '%.%' --Only need verify the subdollar.
) X
)
SELECT
CASE WHEN
----------------------NEW-------------------------------------
( --Price < 0.01 , 6 decimal max
CAST(newOrderPrice AS FLOAT) < 0.01
AND (LEN(newOrderPrice) - PATINDEX('%.%',newOrderPrice)) <= 6
)
OR
( --Price < 1 , 4 decimal max
CAST(newOrderPrice AS FLOAT) < 1
AND (LEN(newOrderPrice) - PATINDEX('%.%',newOrderPrice)) <= 4
)
OR
( --Price >= 1 , 2 decimal max
CAST(newOrderPrice AS FLOAT) >= 1
AND (LEN(newOrderPrice) - PATINDEX('%.%',newOrderPrice)) <= 2
)
OR
( --Price >= 100000 , 1 decimal max
CAST(newOrderPrice AS FLOAT) >= 100000
AND (LEN(newOrderPrice) - PATINDEX('%.%',newOrderPrice)) <= 1
)
----------------------ACK-------------------------------------
OR
( --Price < 0.01 , 6 decimal max
CAST(ackOrderPrice AS FLOAT) < 0.01
AND (LEN(ackOrderPrice) - PATINDEX('%.%',ackOrderPrice)) <= 6
)
OR
( --Price < 1 , 4 decimal max
CAST(ackOrderPrice AS FLOAT) < 1
AND (LEN(ackOrderPrice) - PATINDEX('%.%',ackOrderPrice)) <= 4
)
OR
( --Price >= 1 , 2 decimal max
CAST(ackOrderPrice AS FLOAT) >= 1
AND (LEN(ackOrderPrice) - PATINDEX('%.%',ackOrderPrice)) <= 2
)
OR
( --Price >= 100000 , 1 decimal max
CAST(ackOrderPrice AS FLOAT) >= 100000
AND (LEN(ackOrderPrice) - PATINDEX('%.%',ackOrderPrice)) <= 1
)
THEN 'Qualified'
WHEN ISNULL(ackOrderPrice,'') <> ISNULL(newOrderPrice,'')
THEN 'NewAckPriceMismatch'
ELSE 'Violation'
END AS PriceVerify
,*
FROM PRICE
Found your problem:
AND price like '%.%'
By using a wildcard to start with you are forcing a table scan, which will tank performance. There are several ways you could get the same result without making something not a SARG. Patindex would be my reccomendation:
https://learn.microsoft.com/en-us/sql/t-sql/functions/patindex-transact-sql
In general the Like condition with % is slowing down the queries and the parsetransactionText also sound not that fast
AND price like '%.%'
AND t.Symbol not like '%-%' -- exclude foreign symbols
AND LscDP.dbo.ParseTransactionText(167,MsgText) <> 'OPT' --Exclude options
Plus please manage the trailing zero as in following example
How to display two digits after decimal point in SQL Server
another big mistake is to filter later from the temp table result, move up the filters
WHERE Price <> ''
AND price like '%.%' --Only need verify the subdollar.
in order to reduce the size of the temptable
plus, all the "postprocessing" if possible needs to be moved down on the temp table in order to be performed only to a smaller set of data
LEFT(SUBSTRING(LscDP.dbo.ParseTransactionText(44,MsgText), PATINDEX('%[0-9.]%', LscDP.dbo.ParseTransactionText(44,MsgText)), 8000),
PATINDEX('%[^0-9.]%', SUBSTRING(LscDP.dbo.ParseTransactionText(44,MsgText), PATINDEX('%[0-9.]%',
LscDP.dbo.ParseTransactionText(44,MsgText)), 8000) + 'X') -1)
AS price
in general rearranging data as you're doing in your select and where conditions should be avoited in sql, better cast and convert data BEFORE inserting in the table in order to light up queries
I need to calculate revenue based on how many items a user has.
So for example, the first 10 items are free, up to 100 items are 0.50, up to 200 are 0.25 and up to 500 are 0.15 for example.
I have no idea where to start with this, can I get some direction please?
EG. If a user has 365 items, this would be (10 * 0) + (90 * 0.5) + (100 * 0.25) + (165 * 0.15)
Ideally I'd be doing this in python or something, but the dashboarding tool doesn't have that capability...
EDIT:
I should have mentioned that the number of items isn't actually the number they have, it's the limit they have chosen. The limit is saved as a single number in a subscription event. So for each user I will have an integer representing their max items eg. 365
First number items using window function row_number,
then use a case expression to assign a proper value for each item.
Simple example: http://sqlfiddle.com/#!17/32e4a/9
SELECT user_id,
SUM(
CASE
WHEN rn <= 10 THEN 0
WHEN rn <= 100 THEN 0.5
WHEN rn <= 200 THEN 0.25
WHEN rn <= 500 THEN 0.15
ELSE 0.05
END
) As revenue
FROM (
SELECT *,
row_number() OVER (partition by user_id order by item_no ) As rn
FROM mytable
) x
GROUP BY user_id
I should have mentioned that the number of items isn't actually the
number they have, it's the limit they have chosen. The limit is saved
as a single number in a subscription event. So for each user I will
have an integer representing their max items eg. 365
In this case the below query probably fits your needs:
Demo: http://sqlfiddle.com/#!17/e7a6a/2
SELECT *,
(SELECT SUM(
CASE
WHEN rn <= 10 THEN 0
WHEN rn <= 100 THEN 0.5
WHEN rn <= 200 THEN 0.25
WHEN rn <= 500 THEN 0.15
ELSE 0.05
END
)
FROM generate_series(1,t.user_limit) rn
)
FROM mytab t;
I'm using SQL report builder and wish to calculate the % within turnaround times
my table looks like
Name count within tat
jeff 1 1
jeff 1 0
jeff 1 1
jeff 1 0
i would like it to look like this.
Name count within tat
jeff 4 2 (50%)
The code im using to calculate within tat is
case
when (convert(Decimal(10,2),
(cast(datediff(minute,
(CAST(RequestDate AS DATETIME) + CAST(RequestTime AS DATETIME)),
REQUEST_TESTLINES.AuthDateTime)as float)/60/24))) >
EXP_TAT then '1' else '0' end as [withintat]
How can I sum this column ?
you looking for something like that?
select name , sum(count) total, sum(within_tat)*100 /sum(count) as percent
from Table1
Group by name
LOOK DEMO SQLFIDDLE
EDIT.
OR if you want it exactly as yyou want try this
select name , sum(count) as total, CONCAT(sum(within_tat),' ' ,'(',sum(within_tat)*100 /sum(count), '%',')' ) as percent
from Table1
Group by name
CHECK DEMO HERE
You could wrap it in another SELECT.
SELECT SUM(count), SUM(withintat)
FROM (/*Your original query*/)
yes, you can use case statment inside sum()
but it would need to return a number..
a change in your "within tat" to something like
select case when (convert(Decimal(10,2),
(cast(datediff(minute,
(CAST(RequestDate AS DATETIME) + CAST(RequestTime AS DATETIME)),
REQUEST_TESTLINES.AuthDateTime)as float)/60/24))) >
EXP_TAT
then 1
else 0 end as [withintat]
....
but, if you need the sum and the percentual.
you will need to use this value two times.
and I am sure you dont want to keep replicate code.
so use your actual query as a sub query to sum it may be a good idea...
if you realy dont want to use it as a subquery
you should use a outer apply to gather the value of withintat, doing something like this
select Name
,count(*) as [count]
,sum(OA.[withintat]) as [withintat]
,sum(OA.[withintat])*100/count(*) as [percent]
,cast(sum(OA.[withintat]) as varchar(max))+' ('+cast(sum(OA.[withintat])*100/count(*) as varchar(max))+' %)' as [withintat_and_percent]
from your_actual_query
outer apply(select case when (convert(Decimal(10,2),
(cast(datediff(minute,
(CAST(RequestDate AS DATETIME) + CAST(RequestTime AS DATETIME)),
REQUEST_TESTLINES.AuthDateTime)as float)/60/24))) > EXP_TAT
then 1
else 0 end as [withintat]
)OA
where ....
I would use an IF in this case (pun aside). I tried to reduce the complexity of your comparison, but without seeing some actual data, it's a best guess.
select
name,
count(name) as count,
concat(
sum(if(datediff(minute,
(cast(RequestDate AS DATETIME) +
cast(RequestTime AS DATETIME)),
REQUEST_TESTLINES.AuthDateTime) / 60 / 24 > EXP_TAT, 1, 0 )),
' (',
format(
(sum(if(datediff(minute,
(cast(RequestDate AS DATETIME) +
cast(RequestTime AS DATETIME)),
REQUEST_TESTLINES.AuthDateTime) / 60 / 24 > EXP_TAT, 1, 0 )
)/count(name))*100,1),
'%)'
) as 'within tat'