Replace Union All to join to improve performance - sql

I have a working query which takes 20 mins to return data. I want to optimize it .
I have table
Incentives:
Transaction_ID | Incentive_On_A | Incentive_On_B | Incentive_On_C
--------------+-----------------+-----------------+---------------
1 | 0 | 0 | 10
2 | 30 | 0 | 0
3 | 0 | 20 | 0
4 | 40 | 0 | 0
Required Output:
Transaction_ID| Product_Category | Incentive_Amt
---------- + -----------------+--------------
1 | A | 30
2 | B | 20
3 | C | 10
4 | A | 40
I am using this query :
select Transaction_ID, 'A' as Product_Category,
Incentive_On_A from Incentives
Union all
select Transaction_ID, 'B' as Product_Category,
Incentive_On_B from Incentives
Union all
select Transaction_ID, 'C' as Product_Category,
Incentive_On_C from Incentives
Is there any way I can optimize this query by removing union all with join?
Thanks alot for the help.
Edited**
1.Added one more row in both the tables.
Note:- Basically we are just doing a transpose of data - converting columns- 'Incentive_on_A','Incentive_on_B','Incentive_on_C' to a column - 'Category' having the values of the above 3 columns.

You don't need a JOIN here, you just need to unpivot your data:
SELECT transaction_id, REGEXP_SUBSTR(incentive_col, '[^_]*$') AS product_category
, incentive_amt
FROM (
SELECT transaction_id, incentive_a, incentive_b, incentive_c
FROM incentives
) UNPIVOT (
incentive_amt
FOR incentive_col IN (incentive_a, incentive_b, incentive_c )
) WHERE incentive_amt > 0;
Whether or not this will actually improve your performance, I could not say. My guess is that with the UNION ALL version of your query you're actually doing a full table scan 3 times.

To start with: this is a bad datamodel. If each record can only have one value, then just store one value, exactly as shown in your desired output.
As is, you can just add all values and use CASE WHEN to see which value is greater than zero:
select
transaction_id,
case when incentive_on_a > 0 then 'A'
when incentive_on_b > 0 then 'B'
when incentive_on_c > 0 then 'C'
end as product_category,
incentive_on_a + incentive_on_b + incentive_on_c as incentive_amt
from incentives
order by transaction_id;
(However, I still fail to see how such simple query like the one you are showing can run twenty minutes.)

Related

Is there a version of 'CONTAINS' function in SQLITE other than 'LIKE'?

I'm trying to find totals for each number in the range of 1 to 7. But the data contains different combinations of these numbers. For e.g. 1; 2; 3,7; 1,2,3 and so on. I want to find the total number of times each number pops up. What I essentially want is a code for SQLite that's goes like:
select <fields>, count(*)
from tablexyz
where <field> contains '2' (and '3','4',... individually)
When I input "where like '2%'" and such, it only gives me all series that start with 2 but negates series that starts with 1 but contains 2.
Any help would be appreciated!
I want to find the total number of times each number pops up
Your sample code and the solution you say you want don't exactly align. The closest I can think of is
with t (txt) as -- a sample record from your table
(select '1; 2; 3,7; 1,2,3'),
t2 (num) as -- a lookup table we can create for range of numbers 1-7
(select 1 union all
select 2 union all
select 3 union all
select 4 union all
select 5 union all
select 6 union all
select 7)
select t2.num, length(t.txt) - length(replace(t.txt,t2.num,'')) as num_occurence
from t2
left join t on t.txt like '%' || t2.num || '%'
Outputs
+-----+---------------+
| num | num_occurence |
+-----+---------------+
| 1 | 2 |
| 2 | 2 |
| 3 | 2 |
| 4 | NULL |
| 5 | NULL |
| 6 | NULL |
| 7 | 1 |
+-----+---------------+
Demo
Using the solution below, you can build a "table" of the numbers 1 to 7, then join it to your source data table to count if the number occurs in that row, then sum it together.
Query
WITH
sample_data (nums)
AS
(SELECT '1,2,3,4,5,6'
UNION ALL
SELECT '3,4,5,6'
UNION ALL
SELECT '1,2,7,6'
UNION ALL
SELECT '6' ),
search_nums (search_num)
AS
(VALUES(1)
UNION ALL
SELECT search_num+1 FROM search_nums WHERE search_num<7)
select search_num, sum(count_of_num) from (
SELECT s.nums,
n.search_num,
case
instr(s.nums, n.search_num)
when 0 then 0
else 1
end as count_of_num
FROM sample_data s, search_nums n
) group by search_num;
Result
search_num sum(count_of_num)
1 2
2 2
3 2
4 2
5 2
6 4
7 1

query for column that are within a variable + or 1 of another column

I have a table that has 2 columns, and I am trying to determine a way to select the records where the two columns are CLOSE to one another. Maybe based on standard deviation if i can think about how to do that. But for now, this is what my table looks like:
ID| PCT | RETURN
1 | 20 | 1.20
2 | 15 | 0.90
3 | 0 | 3.00
The values in the pct field is a percent number (for example 20%). The value in the return field is a not fully calculated % number (so its supposed to be 20% above what the initial value was). The query I am working with so far is this:
select * from TABLE1 where ((pct = ((return - 1)* 100)));
What I'd like to end up with are the rows where both are within a set value of each other. For example If they are within 5 points of each other, then the row would be returned and the output would be:
ID| PCT | RETURN
1 | 20 | 1.20
2 | 15 | 0.90
In the above, ID 1 should work out to be PCT = 20 and Return = 20, and ID 2, is PCT = 15 and RETURN = 10. Because it was within 5 points of each other, it was returned.
ID 3 was not returned because 0 and 200 are way above the 5 point threshold.
Is there any way to set a variable that would return a +- 5 when comparing the two values from the above attributes? Thanks.
RexTester Example:
Use Lead() over (Order by PCT) to look ahead and LAG() to look back to the next row do the math and evaluate results...
WITH CTE (ID, PCT , RETURN) as (
SELECT 1 , 20 , 1.20 FROM DUAL UNION ALL
SELECT 2 , 15 , 0.90 FROM DUAL UNION ALL
SELECT 3 , 0 , 3.00 FROM DUAL),
CTE2 as (SELECT A.*, LEAD(PCT) Over (ORDER BY PCT) LEADPCT, LAG(PCT) Over (order by PCT) LAGPCT
FROM CTE A)
SELECT * FROM CTE2
WHERE LEADPCT-PCT <=5 OR PCT-LAGPCT <=5
Order by ID
Giving us:
+----+----+-----+--------+---------+--------+
| | ID | PCT | RETURN | LEADPCT | LAGPCT |
+----+----+-----+--------+---------+--------+
| 1 | 1 | 20 | 1,20 | NULL | 15 |
| 2 | 2 | 15 | 0,90 | 20 | 0 |
+----+----+-----+--------+---------+--------+
or use the return value instead of PCT... just depends on what you're after. But maybe I don't fully understand the question..

SQL Server 2012 - Find a steadily rising value of a column

I have a table like below:
ID | Name | Ratio | Miles
____________________________________
1 | ABC | 45 | 21
1 | ABC | 46 | 24
1 | ABC | 46 | 25
2 | PQR | 41 | 19
2 | PQR | 39 | 17
3 | XYZ | 27 | 13
3 | XYZ | 26 | 11
4 | DEF | 40 | 18
4 | DEF | 40 | 18
4 | DEF | 42 | 20
I want to write a query that will find an ID whose Miles value has been steadily rising.
For instance,
Miles values of Name 'ABC' and 'DEF' are steadily rising.
It's fine if the Miles value drops by up to 5% and rises again.
It should also include this Name.
I tried self join on this table but it gives me Cartesian product.
Can anyone help me with this?
I am using SQL server 2012.
Thanks in advance!
SQL tables represent unordered sets. Let me assume that you have a column that specifies the ordering. Then, you can use lag() and some logic:
select id, name
from (select t.*,
lag(miles) over (partition by id order by orderingcol) as prev_miles
from t
) t
group by id, name
having min(case when prev_miles is null or miles >= prev_miles * 0.95 then 1 else 0 end) = 1;
The having clause is simply determining if all the rows meet your specific condition.
try this:
Note: 5% case is not handled here
create table #tmp(ID INT,Name VARCHAR(50),Ratio INT,Miles INT)
INSERT INTO #tmp
SELECT 1,'ABC',45,21
union all
SELECT 1,'ABC',46,24
union all
SELECT 1,'ABC',46,25
union all
SELECT 2,'PQR',41,19
union all
SELECT 2,'PQR',39,17
union all
SELECT 3,'XYZ',27,13
union all
SELECT 3,'XYZ',26,11
union all
SELECT 4,'DEF',40,18
union all
SELECT 4,'DEF',40,18
union all
SELECT 4,'DEF',42,21
Select *,CASE WHEN Miles<=LEAD(Miles,1,Miles) OVER(partition by ID Order by ID) THEN 1
--NEED ADD 5%condition Here
ELSE 0 END AS nextMiles
into #tmp2
from #tmp
;with cte
AS(
select * , ROW_NUMBER() OVER (partition by ID,nextMiles order by ID) rn from #tmp2
)
SELECT DISTINCT ID,Name FROM cte WHERE rn>1
Drop table #tmp
Drop table #tmp2

Aggregate rows between two rows with certain value

I'm trying to formulate a query to aggregate rows that are between rows with a specific value: in this example I want to collapse and sum time of all rows that have an ID other than 1, but still show rows with ID 1.
This is my table:
ID | Time
----+-----------
1 | 60
2 | 10
3 | 15
1 | 30
4 | 100
1 | 20
This is the result I'm looking for:
ID | Time
--------+-----------
1 | 60
Other | 25
1 | 30
Other | 100
1 | 20
I have attempted to SUM and add a condition with CASE, or but so far my solutions only get me to sum ALL rows and I lose the intervals, so I get this:
ID | Time
------------+-----------
Other | 125
1 | 110
Any help or suggestions in the right direction would be greatly appreciated, thanks!
You need to define the groupings. SQLite is not great for this sort of manipulation, but you can do it by summing the "1" values up to each value.
In SQLite, we can use the rowid column for the ordering:
select (case when id = 1 then '1' else 'other' end) as which,
sum(time)
from (select t.*,
(select count(*) from t t2 where t2.rowid <= t.rowid and t2.id = 1) as grp
from t
) t
group by (case when id = 1 then '1' else 'other' end), grp
order by grp, which;

SQL query update by grouping

I'm dealing with some legacy data in an Oracle table and have the following
--------------------------------------------
| RefNo | ID |
--------------------------------------------
| FOO/BAR/BAZ/AAAAAAAAAA | 1 |
| FOO/BAR/BAZ/BBBBBBBBBB | 1 |
| FOO/BAR/BAZ/CCCCCCCCCC | 1 |
| FOO/BAR/BAZ/DDDDDDDDDD | 1 |
--------------------------------------------
For each of the /FOO/BAR/BAZ/% records I want to make the ID a Unique incrementing number.
Is there a method to do this in SQL?
Thanks in advance
EDIT
Sorry for not being specific. I have several groups of records /FOO/BAR/BAZ/, /FOO/ZZZ/YYY/. The same transformation needs to occur for each of these other (example) groups. The recnum can't be used I want ID to start from 1, incrementing, for each group of records I have to change.
Sorry for making a mess of my first post. Output should be
--------------------------------------------
| RefNo | ID |
--------------------------------------------
| FOO/BAR/BAZ/AAAAAAAAAA | 1 |
| FOO/BAR/BAZ/BBBBBBBBBB | 2 |
| FOO/BAR/BAZ/CCCCCCCCCC | 3 |
| FOO/BAR/BAZ/DDDDDDDDDD | 4 |
| FOO/ZZZ/YYY/AAAAAAAAAA | 1 |
| FOO/ZZZ/YYY/BBBBBBBBBB | 2 |
--------------------------------------------
Let's try something like this(Oracle version 10g and higher):
SQL> with t1 as(
2 select 'FOO/BAR/BAZ/AAAAAAAAAA' as RefNo, 1 as ID from dual union all
3 select 'FOO/BAR/BAZ/BBBBBBBBBB', 1 from dual union all
4 select 'FOO/BAR/BAZ/CCCCCCCCCC', 1 from dual union all
5 select 'FOO/BAR/BAZ/DDDDDDDDDD', 1 from dual union all
6 select 'FOO/ZZZ/YYY/AAAAAAAAAA', 1 from dual union all
7 select 'FOO/ZZZ/YYY/BBBBBBBBBB', 1 from dual union all
8 select 'FOO/ZZZ/YYY/CCCCCCCCCC', 1 from dual union all
9 select 'FOO/ZZZ/YYY/DDDDDDDDDD', 1 from dual
10 )
11 select row_number() over(partition by ComPart order by DifPart) as id
12 , RefNo
13 From (select regexp_substr(RefNo, '[[:alpha:]]+$') as DifPart
14 , regexp_substr(RefNo, '([[:alpha:]]+/)+') as ComPart
15 , RefNo
16 , Id
17 from t1
18 ) q
19 ;
ID REFNO
---------- -----------------------
1 FOO/BAR/BAZ/AAAAAAAAAA
2 FOO/BAR/BAZ/BBBBBBBBBB
3 FOO/BAR/BAZ/CCCCCCCCCC
4 FOO/BAR/BAZ/DDDDDDDDDD
1 FOO/ZZZ/YYY/AAAAAAAAAA
2 FOO/ZZZ/YYY/BBBBBBBBBB
3 FOO/ZZZ/YYY/CCCCCCCCCC
4 FOO/ZZZ/YYY/DDDDDDDDDD
I think that actual updating the ID column wouldn't be a good idea. Every time you add new groups of data you would have to run the update statement again. The better way would be creating a view and you will see desired output every time you query it.
rownum can be used as an incrementing ID?
UPDATE legacy_table
SET id = ROWNUM;
This will assign unique values to all records in the table. This link contains documentation about Oracle Pseudocolumn.
You can run the following:
update <table_name> set id = rownum where descr like 'FOO/BAR/BAZ/%'
This is pretty rough and I'm not sure if your RefNo is a single value column or you just made it like that for simplicity.
select
sub.RefNo
row_number() over (order by sub.RefNo) + (select max(id) from TABLE),
from (
select FOO+'/'+BAR+'/'+BAZ+'/'+OTHER as RefNo
from TABLE
group by FOO+'/'+BAR+'/'+BAZ+'/'+OTHER
) sub