I'm trying to find totals for each number in the range of 1 to 7. But the data contains different combinations of these numbers. For e.g. 1; 2; 3,7; 1,2,3 and so on. I want to find the total number of times each number pops up. What I essentially want is a code for SQLite that's goes like:
select <fields>, count(*)
from tablexyz
where <field> contains '2' (and '3','4',... individually)
When I input "where like '2%'" and such, it only gives me all series that start with 2 but negates series that starts with 1 but contains 2.
Any help would be appreciated!
I want to find the total number of times each number pops up
Your sample code and the solution you say you want don't exactly align. The closest I can think of is
with t (txt) as -- a sample record from your table
(select '1; 2; 3,7; 1,2,3'),
t2 (num) as -- a lookup table we can create for range of numbers 1-7
(select 1 union all
select 2 union all
select 3 union all
select 4 union all
select 5 union all
select 6 union all
select 7)
select t2.num, length(t.txt) - length(replace(t.txt,t2.num,'')) as num_occurence
from t2
left join t on t.txt like '%' || t2.num || '%'
Outputs
+-----+---------------+
| num | num_occurence |
+-----+---------------+
| 1 | 2 |
| 2 | 2 |
| 3 | 2 |
| 4 | NULL |
| 5 | NULL |
| 6 | NULL |
| 7 | 1 |
+-----+---------------+
Demo
Using the solution below, you can build a "table" of the numbers 1 to 7, then join it to your source data table to count if the number occurs in that row, then sum it together.
Query
WITH
sample_data (nums)
AS
(SELECT '1,2,3,4,5,6'
UNION ALL
SELECT '3,4,5,6'
UNION ALL
SELECT '1,2,7,6'
UNION ALL
SELECT '6' ),
search_nums (search_num)
AS
(VALUES(1)
UNION ALL
SELECT search_num+1 FROM search_nums WHERE search_num<7)
select search_num, sum(count_of_num) from (
SELECT s.nums,
n.search_num,
case
instr(s.nums, n.search_num)
when 0 then 0
else 1
end as count_of_num
FROM sample_data s, search_nums n
) group by search_num;
Result
search_num sum(count_of_num)
1 2
2 2
3 2
4 2
5 2
6 4
7 1
Related
EDIT: added a link to Fiddle for a more comprehensive sample (actual dataset)
I wonder if the below is possible in SQL, in BigQuery in particular, and in one SELECT statement.
Consider following input:
Key | Value
-----|-------
a | 2
a | 3
b | 2
b | 3
b | 5
c | 2
c | 5
c | 7
Logic: select the lowest value "available" for each key. Available meaning not yet assigned/used. See below.
Key | Value | Rule
-----|-------|--------------------------------------------
a | 2 | keep
a | 3 | ignore because key "a" has a value already
b | 2 | ignore because value "2" was already used
b | 3 | keep
b | 5 | ignore because key "b" has a value already
c | 2 | ignore because value "2" was already used
c | 5 | keep
c | 7 | ignore because key "c" has a value already
Hence expected outcome:
Key | Value
-----|-------
a | 2
b | 3
c | 5
Here the SQL to create the dummy table:
with t as ( select
'a' key, 2 value UNION ALL select 'a', 3
UNION ALL select 'b', 2 UNION ALL select 'b', 3 UNION ALL select 'b', 5
UNION ALL select 'c', 2 UNION ALL select 'c', 5 UNION ALL select 'c', 7
)
select * from t
EDIT: here another dataset
Not sure what combination of FULL JOIN, DISTINCT, ARRAY or WINDOW functions I can use.
Any guidance is appreciated.
EDIT: This is an incorrect answer that worked with the original example dataset, but has issues (as seen with comprehensive sample). I'm leaving it here for now to maintain comment history.
I don't have a specific BigQuery answer, but here is one SQL solution using a Common Table Expression and recursion.
WITH MyCTE AS
(
/* ANCHOR SUBQUERY */
SELECT MyKey, MyValue
FROM MyTable t
WHERE t.MyKey = (SELECT MIN(MyKey) FROM MyTable)
UNION ALL
/* RECURSIVE SUBQUERY */
SELECT t.MyKey, t.MyValue
FROM MyTable t
INNER JOIN MyCTE c
ON c.MyKey < t.MyKey
AND c.MyValue < t.MyValue
)
SELECT MyKey, MIN(MyValue)
FROM MyCTE
GROUP BY MyKey
;
Results:
Key | Value
-----|-------
a | 2
b | 3
c | 5
SQL Fiddle
I have a working query which takes 20 mins to return data. I want to optimize it .
I have table
Incentives:
Transaction_ID | Incentive_On_A | Incentive_On_B | Incentive_On_C
--------------+-----------------+-----------------+---------------
1 | 0 | 0 | 10
2 | 30 | 0 | 0
3 | 0 | 20 | 0
4 | 40 | 0 | 0
Required Output:
Transaction_ID| Product_Category | Incentive_Amt
---------- + -----------------+--------------
1 | A | 30
2 | B | 20
3 | C | 10
4 | A | 40
I am using this query :
select Transaction_ID, 'A' as Product_Category,
Incentive_On_A from Incentives
Union all
select Transaction_ID, 'B' as Product_Category,
Incentive_On_B from Incentives
Union all
select Transaction_ID, 'C' as Product_Category,
Incentive_On_C from Incentives
Is there any way I can optimize this query by removing union all with join?
Thanks alot for the help.
Edited**
1.Added one more row in both the tables.
Note:- Basically we are just doing a transpose of data - converting columns- 'Incentive_on_A','Incentive_on_B','Incentive_on_C' to a column - 'Category' having the values of the above 3 columns.
You don't need a JOIN here, you just need to unpivot your data:
SELECT transaction_id, REGEXP_SUBSTR(incentive_col, '[^_]*$') AS product_category
, incentive_amt
FROM (
SELECT transaction_id, incentive_a, incentive_b, incentive_c
FROM incentives
) UNPIVOT (
incentive_amt
FOR incentive_col IN (incentive_a, incentive_b, incentive_c )
) WHERE incentive_amt > 0;
Whether or not this will actually improve your performance, I could not say. My guess is that with the UNION ALL version of your query you're actually doing a full table scan 3 times.
To start with: this is a bad datamodel. If each record can only have one value, then just store one value, exactly as shown in your desired output.
As is, you can just add all values and use CASE WHEN to see which value is greater than zero:
select
transaction_id,
case when incentive_on_a > 0 then 'A'
when incentive_on_b > 0 then 'B'
when incentive_on_c > 0 then 'C'
end as product_category,
incentive_on_a + incentive_on_b + incentive_on_c as incentive_amt
from incentives
order by transaction_id;
(However, I still fail to see how such simple query like the one you are showing can run twenty minutes.)
Hi,I have a column as below
+--------+--------+
| day | amount|
+--------+---------
| 2 | 2 |
| 1 | 3 |
| 1 | 4 |
| 2 | 2 |
| 3 | 3 |
| 4 | 3 |
+--------+--------+
now I want something like this sum day 1- day2 as row one , sum day1-3 as row 2, and so on.
+--------+--------+
| day | amount|
+--------+---------
| 1-2 | 11 |
| 1-3 | 14 |
| 1-4 | 17 |
+--------+--------+
Could you offer any one help ,thanks!
with data as(
select 2 day, 2 amount from dual union all
select 1 day, 3 amount from dual union all
select 1 day, 4 amount from dual union all
select 2 day, 2 amount from dual union all
select 3 day, 3 amount from dual union all
select 4 day, 3 amount from dual)
select distinct day, sum(amount) over (order by day range unbounded preceding) cume_amount
from data
order by 1;
DAY CUME_AMOUNT
---------- -----------
1 7
2 11
3 14
4 17
if you are using oracle you can do something like the above
Assuming the day range in left column always starts from "1-", What you need is a query doing cumulative sum on the grouped table(dayWiseSum below). Since it needs to be accessed twice I'd put it into a temporary table.
CREATE TEMPORARY TABLE dayWiseSum AS
(SELECT day,SUM(amount) AS amount FROM table1 GROUP BY day ORDER BY day);
SELECT CONCAT("1-",t1.day) as day, SUM(t2.amount) AS amount
FROM dayWiseSum t1 INNER JOIN dayWiseSum
t2 ON t1.day > t2.day
--change to >= if you want to include "1-1"
GROUP BY t1.day, t1.amount ORDER BY t1.day
DROP TABLE dayWiseSum;
Here's a fiddle to test with:
http://sqlfiddle.com/#!9/c1656/1/0
Note: Since sqlfiddle isn't allowing CREATE statements, I've replaced dayWiseSum with it's query there. Also, I've used "Text to DDL" option to paste the exact text of the table from your question to generate the create table query :)
I have a Power table that stores building circuit details. A circuit can be 1 phase or 3 phase but is always represented as 1 row in the circuit table.
I want to insert the details of the circuits into a join table which joins panels to circuits
My current circuit table has the following details
CircuitID | Voltage | Phase | PanelID | Cct |
1 | 120 | 1 | 1 | 1 |
2 | 208 | 3 | 1 | 3 |
3 | 208 | 2 | 1 | 8 |
Is it possible to create a select where by when it sees a 3 phase row it selects 3 rows (or 2 select 2 rows) and increments the Cct column by 1 each time or do I have to create a loop?
CircuitID | PanelID | Cct |
1 | 1 | 1 |
2 | 1 | 3 |
2 | 1 | 4 |
2 | 1 | 5 |
3 | 1 | 8 |
3 | 1 | 9 |
Here is one way to do it
First generate numbers using tally table(best possible way). Here is one excellent article about generating number without loops. Generate a set or sequence without loops
Then join the numbers table with yourtable where phase value of each record should be greater than sequence number in number's table
;WITH e1(n) AS
(
SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL
SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL
SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1
), -- 10
e2(n) AS (SELECT 1 FROM e1 CROSS JOIN e1 AS b), -- 10*10
e3(n) AS (SELECT 1 FROM e1 CROSS JOIN e2), -- 10*100
numbers as ( SELECT n = ROW_NUMBER() OVER (ORDER BY n) FROM e3 )
SELECT CircuitID,
PanelID,
Cct = Cct + ( n - 1 )
FROM Yourtable a
JOIN numbers b
ON a.Phase >= b.n
You can do this with a one recursive cte.
WITH cte AS
(
SELECT [CircuitID], [Voltage], [Phase], [PanelID], [Cct], [Cct] AS [Ref]
FROM [Power]
UNION ALL
SELECT [CircuitID], [Voltage], [Phase], [PanelID], [Cct] + 1, [Ref]
FROM cte
WHERE [Cct] + 1 < [Phase] + [Ref]
)
SELECT [CircuitID], [PanelID], [Cct]
FROM cte
ORDER BY [CircuitID]
Simplest way,
Select y.* from (
Select 1 CircuitID,120 Voltage,1 Phase,1 PanelID, 1 Cct
union
Select 2,208,3,1,3
union
Select 3,208,2,1,8)y,
(Select 1 x
union
Select 2 x
union
Select 3 x)x
Where x.x <= y.Phase
Directly copy paste this and try, it will run 100%. After that, just replace my 'y' table with your real table.
I'm dealing with some legacy data in an Oracle table and have the following
--------------------------------------------
| RefNo | ID |
--------------------------------------------
| FOO/BAR/BAZ/AAAAAAAAAA | 1 |
| FOO/BAR/BAZ/BBBBBBBBBB | 1 |
| FOO/BAR/BAZ/CCCCCCCCCC | 1 |
| FOO/BAR/BAZ/DDDDDDDDDD | 1 |
--------------------------------------------
For each of the /FOO/BAR/BAZ/% records I want to make the ID a Unique incrementing number.
Is there a method to do this in SQL?
Thanks in advance
EDIT
Sorry for not being specific. I have several groups of records /FOO/BAR/BAZ/, /FOO/ZZZ/YYY/. The same transformation needs to occur for each of these other (example) groups. The recnum can't be used I want ID to start from 1, incrementing, for each group of records I have to change.
Sorry for making a mess of my first post. Output should be
--------------------------------------------
| RefNo | ID |
--------------------------------------------
| FOO/BAR/BAZ/AAAAAAAAAA | 1 |
| FOO/BAR/BAZ/BBBBBBBBBB | 2 |
| FOO/BAR/BAZ/CCCCCCCCCC | 3 |
| FOO/BAR/BAZ/DDDDDDDDDD | 4 |
| FOO/ZZZ/YYY/AAAAAAAAAA | 1 |
| FOO/ZZZ/YYY/BBBBBBBBBB | 2 |
--------------------------------------------
Let's try something like this(Oracle version 10g and higher):
SQL> with t1 as(
2 select 'FOO/BAR/BAZ/AAAAAAAAAA' as RefNo, 1 as ID from dual union all
3 select 'FOO/BAR/BAZ/BBBBBBBBBB', 1 from dual union all
4 select 'FOO/BAR/BAZ/CCCCCCCCCC', 1 from dual union all
5 select 'FOO/BAR/BAZ/DDDDDDDDDD', 1 from dual union all
6 select 'FOO/ZZZ/YYY/AAAAAAAAAA', 1 from dual union all
7 select 'FOO/ZZZ/YYY/BBBBBBBBBB', 1 from dual union all
8 select 'FOO/ZZZ/YYY/CCCCCCCCCC', 1 from dual union all
9 select 'FOO/ZZZ/YYY/DDDDDDDDDD', 1 from dual
10 )
11 select row_number() over(partition by ComPart order by DifPart) as id
12 , RefNo
13 From (select regexp_substr(RefNo, '[[:alpha:]]+$') as DifPart
14 , regexp_substr(RefNo, '([[:alpha:]]+/)+') as ComPart
15 , RefNo
16 , Id
17 from t1
18 ) q
19 ;
ID REFNO
---------- -----------------------
1 FOO/BAR/BAZ/AAAAAAAAAA
2 FOO/BAR/BAZ/BBBBBBBBBB
3 FOO/BAR/BAZ/CCCCCCCCCC
4 FOO/BAR/BAZ/DDDDDDDDDD
1 FOO/ZZZ/YYY/AAAAAAAAAA
2 FOO/ZZZ/YYY/BBBBBBBBBB
3 FOO/ZZZ/YYY/CCCCCCCCCC
4 FOO/ZZZ/YYY/DDDDDDDDDD
I think that actual updating the ID column wouldn't be a good idea. Every time you add new groups of data you would have to run the update statement again. The better way would be creating a view and you will see desired output every time you query it.
rownum can be used as an incrementing ID?
UPDATE legacy_table
SET id = ROWNUM;
This will assign unique values to all records in the table. This link contains documentation about Oracle Pseudocolumn.
You can run the following:
update <table_name> set id = rownum where descr like 'FOO/BAR/BAZ/%'
This is pretty rough and I'm not sure if your RefNo is a single value column or you just made it like that for simplicity.
select
sub.RefNo
row_number() over (order by sub.RefNo) + (select max(id) from TABLE),
from (
select FOO+'/'+BAR+'/'+BAZ+'/'+OTHER as RefNo
from TABLE
group by FOO+'/'+BAR+'/'+BAZ+'/'+OTHER
) sub