We have a table which have millions of entry. The table have two columns, now there is correlation between X and Y when X is beyond a value, Y tends to be B (However it is not always true, its a trend not a certainty).
Here i want to find the threshold value for X, i.e(X1) such that at least 99% of the value which are less than X1 are B.
It can be done using code easily. But is there a SQL query which can do the computation.
For the below dataset expected is 6 because below 6 more than 99% is 'B' and there is no bigger value of X for which more than 99% is 'B'. However if I change it to precision of 90% then it will become 12 because if X<12 more than 90% of the values are 'B' and there is no bigger value of X for which it holds true
So we need to find the biggest value X1 such that at least 99% of the value lesser than X1 are 'B'.
X Y
------
2 B
3 B
3 B
4 B
5 B
5 B
5 B
6 G
7 B
7 B
7 B
8 B
8 B
8 B
12 G
12 G
12 G
12 G
12 G
12 G
12 G
12 G
13 G
13 G
13 B
13 G
13 G
13 G
13 G
13 G
14 B
14 G
14 G
Ok, I think this accomplishes what you want to do, but it will not work for the data volume you are mentioning. I'm posting it anyway in case it can help someone else provide an answer.
This may be one of those cases where the most efficient way is to use a cursor with sorted data.
Oracle has some builting functions for correlation analysis but I've never worked with it so I don't know how they work.
select max(x)
from (select x
,y
,num_less
,num_b
,num_b / nullif(num_less,0) as percent_b
from (select x
,y
,(select count(*) from table b where b.x<a.x) as num_less
,(select count(*) from table b where b.x<a.x and b.y = 'B') as num_b
from table a
)
where num_b / nullif(num_less,0) >= 0.99
);
The inner select does the following:
For every value of X
Count the nr of values < X
Count the nr of 'B'
The next SELECT computes the ratio of B's and filter only the rows where the ratio is above the threshold. The outer just picks the max(x) from those remaining rows.
Edit:
The non-scalable part in the above query is the semi-cartesian self-joins.
This is mostly inspired from the previous answer, which had some flaws.
select max(next_x) from
(
select
count(case when y='B' then 1 end) over (order by x) correct,
count(case when y='G' then 1 end) over (order by x) wrong,
lead(x) over (order by x) next_x
from table_name
)
where correct/(correct + wrong) > 0.99
Sample data:
create table table_name(x number, y varchar2(1));
insert into table_name
select 2, 'B' from dual union all
select 3, 'B' from dual union all
select 3, 'B' from dual union all
select 4, 'B' from dual union all
select 5, 'B' from dual union all
select 5, 'B' from dual union all
select 5, 'B' from dual union all
select 6, 'G' from dual union all
select 7, 'B' from dual union all
select 7, 'B' from dual union all
select 7, 'B' from dual union all
select 8, 'B' from dual union all
select 8, 'B' from dual union all
select 8, 'B' from dual union all
select 12, 'G' from dual union all
select 12, 'G' from dual union all
select 12, 'G' from dual union all
select 12, 'G' from dual union all
select 12, 'G' from dual union all
select 12, 'G' from dual union all
select 12, 'G' from dual union all
select 12, 'G' from dual union all
select 13, 'G' from dual union all
select 13, 'G' from dual union all
select 13, 'B' from dual union all
select 13, 'G' from dual union all
select 13, 'G' from dual union all
select 13, 'G' from dual union all
select 13, 'G' from dual union all
select 13, 'G' from dual union all
select 14, 'B' from dual union all
select 14, 'G' from dual union all
select 14, 'G' from dual;
Give a try with this and share the results:
Assuming table name as table_name and columns as x and y
with TAB AS (
select (count(x) over (PARTITION BY Y order by x rows between unbounded preceding and current row))/
(COUNT(case when y='B' then 1 end) OVER (PARTITION BY Y)) * 100 CC, x, y
from table_name)
select x,y from (SELECT min(cc) over (partition by y) min_cc, x, cc, y
FROM TAB
where cc >= 99)
where min_cc = cc
Related
I have input like :-
Table 1
Table 2
A
4
B
5
C
6
1
X
2
Y
3
Z
And Output muse be
Output
A
B
C
X
Y
Z
One method uses a union followed by a filter:
SELECT val
FROM
(
SELECT col1 AS val FROM yourTable
UNION ALL
SELECT col2 FROM yourTable
) t
WHERE REGEXP_LIKE(val, '^[A-Z]+$')
ORDER BY val;
If data really looks as you put it, a simple option is to use greatest function.
Sample data:
SQL> with test (col1, col2) as
2 (select 'A', '4' from dual union all
3 select 'B', '5' from dual union all
4 select 'C', '6' from dual union all
5 select '1', 'X' from dual union all
6 select '2', 'Y' from dual union all
7 select '3', 'Z' from dual
8 )
Query:
9 select greatest(col1, col2) result
10 from test;
RESULT
----------
A
B
C
X
Y
Z
6 rows selected.
SQL>
I am looking for ideas on how to group numbers into low and high ranges in Oracle SQL. I looking to to avoid cursors...any ideas welcome
Example input
ID
LOW
HIGH
A
0
2
A
2
3
A
3
5
A
9
11
A
11
13
A
13
15
B
0
1
B
1
4
B
7
9
B
11
12
B
12
17
B
17
18
Which would result in the following grouping into ranges
ID
LOW
HIGH
A
0
5
A
9
15
B
0
4
B
7
9
B
11
18
This is a Gaps & Islands problem. You can use the traditional solution.
For example:
select max(id) as id, min(low) as low, max(high) as high
from (
select x.*, sum(i) over(order by id, low) as g
from (
select t.*,
case when low = lag(high) over(partition by id order by low)
and id = lag(id) over(partition by id order by low)
then 0 else 1 end as i
from t
) x
) y
group by g
Result:
ID LOW HIGH
--- ---- ----
A 0 5
A 9 15
B 0 4
B 7 9
B 11 18
See running example at db<>fiddle.
From Oracle 12, you should use MATCH_RECOGNIZE for row-by-row pattern matching:
SELECT *
FROM table_name
MATCH_RECOGNIZE(
PARTITION BY id
ORDER BY low, high
MEASURES
FIRST(low) AS low,
MAX(high) AS high
PATTERN (overlapping* last_row)
DEFINE
overlapping AS NEXT(low) <= MAX(high)
)
Which, for the sample data:
CREATE TABLE table_name (id, low, high) AS
SELECT 'A', 0, 2 FROM DUAL UNION ALL
SELECT 'A', 2, 3 FROM DUAL UNION ALL
SELECT 'A', 3, 5 FROM DUAL UNION ALL
SELECT 'A', 9, 11 FROM DUAL UNION ALL
SELECT 'A', 11, 13 FROM DUAL UNION ALL
SELECT 'A', 13, 15 FROM DUAL UNION ALL
SELECT 'B', 0, 1 FROM DUAL UNION ALL
SELECT 'B', 1, 4 FROM DUAL UNION ALL
SELECT 'B', 7, 9 FROM DUAL UNION ALL
SELECT 'B', 11, 12 FROM DUAL UNION ALL
SELECT 'B', 12, 17 FROM DUAL UNION ALL
SELECT 'B', 17, 18 FROM DUAL UNION ALL
SELECT 'C', 0, 10 FROM DUAL UNION ALL
SELECT 'C', 1, 3 FROM DUAL UNION ALL
SELECT 'C', 5, 8 FROM DUAL UNION ALL
SELECT 'C', 9, 15 FROM DUAL UNION ALL
SELECT 'C', 10, 14 FROM DUAL UNION ALL
SELECT 'C', 11, 13 FROM DUAL;
Outputs:
ID
LOW
HIGH
A
0
5
A
9
15
B
0
4
B
7
9
B
11
18
C
0
15
fiddle
I have a table in AS400 as below
Type Values Status
A 1 Y
A 2 N
A 3 Y
A 4 Y
A 5 N
B 2 Y
B 7 N
C 3 Y
C 5 N
C 4 Y
C 6 Y
C 7 Y
C 1 Y
D 3 Y
D 5 Y
E 7 N
E 4 N
E 3 Y
E 6 N
E 7 Y
E 8 N
What I need is Top 2 of each type that have a status Y. I.e. the result should be something like A 1 , A 3, B 2 , C3, C4, D3, D5, E3, E7.
The query I have used is this
SELECT type,
REFERENCES
FROM (
SELECT type,
REFERENCES,
STATUS,
rank() OVER (PARTITION BY Type ORDER BY REFERENCES DESC) AS Rank
FROM Tables
) a
WHERE rank <= 2
AND Type IN (A,B,C,D,E)
AND STATUS = Y;
The issue here is it doesn't filter out the status beforehand. It picks up the top 2, then filters out with Y. So the result looks something like A1 instead of A1 and A3, because it has first picked A1 and A2 , and then filtered out A2 .
Where do I insert the Status=y to get a more accurate result.
I am a novice in SQL so if theres a better way to write the above query as well, I am fine with that.
This outta do it. Use the with clause to feed the filtered result into a new query which then you can rank.
with testtable (type, value, status) as (
select 'A', 1, 'Y' from dual union all
select 'A', 2, 'N' from dual union all
select 'A', 3, 'Y' from dual union all
select 'A', 4, 'Y' from dual union all
select 'A', 5, 'N' from dual union all
select 'B', 2, 'Y' from dual union all
select 'B', 7, 'N' from dual union all
select 'C', 3, 'Y' from dual union all
select 'C', 5, 'N' from dual union all
select 'C', 4, 'Y' from dual union all
select 'C', 6, 'Y' from dual union all
select 'C', 7, 'Y' from dual union all
select 'C', 1, 'Y' from dual union all
select 'D', 3, 'Y' from dual union all
select 'D', 5, 'Y' from dual union all
select 'E', 7, 'N' from dual union all
select 'E', 4, 'N' from dual union all
select 'E', 3, 'Y' from dual union all
select 'E', 6, 'N' from dual union all
select 'E', 7, 'Y' from dual union all
select 'E', 8, 'N' from dual
)
, ys as (
select
*
from testtable
where STATUS = 'Y'
)
, yrank as (
select
type,
value,
status,
rank() over(partition by type order by value) Y_RANK
from ys
)
select
*
from yrank
where Y_RANK <= 2
Suppose having the data set:
with
data_table(title, x) as (
select 'a', 1 from dual union all
select 'a', 3 from dual union all
select 'a', 4 from dual union all
select 'a', 5 from dual union all
select 'b', 1 from dual union all
select 'b', 2 from dual union all
select 'b', 3 from dual union all
select 'b', 6 from dual
)
select * from data_table;
TITLE | X
-----------
a 1
a 3
a 4
a 5
b 1
b 2
b 3
b 6
Wee see that points related to a and b are different.
How to align values in X column so both groups have the same points, filling the gaps with NULL?
Expected result is:
TITLE | X
-----------
a 1
a NULL
a 3
a 4
a 5
a NULL
b 1
b 2
b 3
b NULL
b NULL
b 6
Straightforward solution I got is
with
data_table(title, x) as (
select 'a', 1 from dual union all
select 'a', 3 from dual union all
select 'a', 4 from dual union all
select 'a', 5 from dual union all
select 'b', 1 from dual union all
select 'b', 2 from dual union all
select 'b', 3 from dual union all
select 'b', 6 from dual
),
all_points(x) AS (
select distinct x from data_table
),
all_titles(title) AS (
select distinct title from data_table
),
aligned_data(title, x) as (
select t.title, p.x from all_points p cross join all_titles t
)
select ad.title, dt.x
from aligned_data ad
left join data_table dt on dt.title = ad.title and dt.x = ad.x
order by ad.title, ad.x;
As wee see cross join in aligned_data definition is bottleneck and can kill performance on valuable data sets.
I wonder if this task could be solved more elegantly. Maybe a trick with window functions can be proposed.
How do we use group by only to consider a certain value of a column
eg
if the column has values like , and I only want to group the records with the merge_ind = 'Y' or null, if it is say N the record should be treated as separate value
Merge1 Merge2
A Y
A Y
A Y
B Y
B Y
B Y
C N
C N
C N
D N
D N
E null
E null
F null
F null
null null
the o/p should be
count Merge1 merge2
3 A Y
3 B Y
1 C N
1 C N
1 C N
1 D N
1 D N
2 E null
1 F null
1 null null
I implemented it using a union but am not very happy with the performance.
Thanks
Ali
you can do something like this:
SQL> with data as (select 'A' Merge1, 'Y' Merge2 from dual union all
2 select 'A', 'Y' from dual union all
3 select 'A', 'Y' from dual union all
4 select 'B', 'Y' from dual union all
5 select 'B', 'Y' from dual union all
6 select 'B', 'Y' from dual union all
7 select 'C', 'N' from dual union all
8 select 'C', 'N' from dual union all
9 select 'C', 'N' from dual union all
10 select 'D', 'N' from dual union all
11 select 'D', 'N' from dual union all
12 select 'E', null from dual union all
13 select 'E', null from dual union all
14 select 'F', null from dual union all
15 select 'F', null from dual union all
16 select null, null from dual)
17 select merge1, max(merge2), count(*)
18 from (select merge1, merge2,
19 case when merge2 = 'Y' or merge2 is null then merge2 else to_char(rownum) end grp
20 from data)
21 group by merge1, grp
22 order by merge1;
M M COUNT(*)
- - ----------
A Y 3
B Y 3
C N 1
C N 1
C N 1
D N 1
D N 1
E 2
F 2
1
test fiddle: http://sqlfiddle.com/#!4/b85cc/1
Try:
select Merge1, Merge2, count(*)
from table1
group by Merge1, Merge2, case when Merge2 = 'N' then to_char(rownum) else Merge2 end
order by Merge1
Here is a sqlfiddle demo
After some considerable mucking around, I have a query that does the job although I could swear this question was originally tagged mysql, and unfortunately this is a mysql only answer:
select count, merge1, merge2
from (
select count(*) count, merge1, merge2,
if(merge2 = 'Y' or merge2 is null, 0, n)
from (
select merge1, merge2,
(#n := if(#n is null, 1, #n + 1)) n
from t
) x
group by 2, 3, 4
) y
Values not Y are treated as separate values with their own group.
It works by assigning a unique number to each row, them selectively grouping by that too when the value is not Y, thus creating a separate group for each non-Y row.
Here's an sqlfiddle with this query running your dara.