Now I include some parallelism to my app ProcessXX, I'm not sure the data can be process in the right order. So Im working in a query to return the lower and upperbound to pass to ProcessZZ.
My table avl_pool has avl_id and has_link and some other fields and a steady flow of data, when new data arrive they start with has_link=null, when ProcessX finish with the rows has_link have the link value xxxx is some number.
Now on the next step I have to process only those rows with links, but I cant skip rows, because order is very important.
In this case I need ProcessZZ(23561211, 23561219)
rn | avl_id | has_link
1 | 23561211 | xxxx -- start
2 | 23561212 | xxxx
3 | 23561213 | xxxx
4 | 23561214 | xxxx
5 | 23561215 | xxxx
6 | 23561216 | xxxx
7 | 23561217 | xxxx
8 | 23561218 | xxxx
9 | 23561219 | xxxx -- end
10 | 23561220 | null
11 | 23561221 | xxxx
12 | 23561222 | xxxx
13 | 23561223 | xxxx
Currently I have:
-- starting avl_id need to be send to ProcessZZ
SELECT MIN(avl_id) as min_avl_id
FROM avl_db.avl_pool
WHERE NOT has_link IS NULL
-- first avl_id still on hands of ProcessXX ( but can be null )
SELECT MIN(avl_id) as max_avl_id -- here need add a LAG
FROM avl_db.avl_pool
WHERE has_link IS NULL
AND avl_id > (SELECT MIN(avl_id)
FROM avl_db.avl_pool
WHERE NOT has_link IS NULL)
-- In case everyone has_link already the upper limit is the last one on the table.
SELECT MAX(avl_id) as max_avl_id
FROM avl_db.avl_pool
I can put everthing in muliple CTE and return both result, but I think this can be handle like some island, but not sure how.
So the query should looks like
SELECT min_avl_id, min_avl_id
FROM cte
min_avl_id | min_avl_id
23561211 | 23561219
If I understand correctly, you want to assign a sequential number to each block. This number is demarcated by the NULL values in has_link.
If this is the problem, then a cumulative sum solves the problem:
select p.*,
sum(case when has_link is null then 1 else 0 end) over (order by rn) as grp
from avl_db.avl_pool p;
This actually includes the NULL values in the output. The simplest method is probably then a subquery:
select p.*
from (select p.*,
sum(case when has_link is null then 1 else 0 end) over (order by rn) as grp
from avl_db.avl_pool p
) p
where has_link is not null;
Related
I am trying to write a SQL query to get the start date for employees in a store. As seen in the first screenshot, employee number 5041 had the number A0EH but as the number got updated, it updated the start date for the employee as well. This effects the metric of total duration in the store.
I am trying to get to the output below but haven't been able to figure out how to get this view.
This is the code I was trying but I am not getting the correct output.
select
esd.employee_number,
(case when esd.old_employee_number is null then es.employee_number else es.old_employee_number end) as old_employee_number,
esd.entity_id,
esd.original_start_date
from earliest_start_date as esd
left join earliest_start_date as es
on (es.employee_number = esd.old_employee_number)
How do I solve this on SQL?
Redshift reportedly supports recursion via WITH clause. Here's an example:
MariaDB 10.5 has similar support. Test case is here:
Fully working test case (via MariaDB 10.5) (Updated)
Link to Amazon Redshift detail for WITH clause and window functions:
Amazon Redshift - WITH clause
Amazon redshift - Window functions
WITH RECURSIVE cte (employee_number, original_no, entity_id, original_start_date, n) AS (
SELECT employee_number, employee_number, entity_id, original_start_date, 1 FROM earliest_start_date WHERE old_employee_number IS NULL UNION ALL
SELECT new_tbl.employee_number, cte.original_no, cte.entity_id, cte.original_start_date, n+1
FROM earliest_start_date new_tbl
JOIN cte
ON cte.employee_number = new_tbl.old_employee_number
)
, xrows AS (
SELECT *, ROW_NUMBER() OVER (PARTITION BY entity_id ORDER BY n DESC) AS rn
FROM cte
)
SELECT * FROM xrows WHERE rn = 1
;
Result:
+-----------------+-------------+-----------+---------------------+------+----+
| employee_number | original_no | entity_id | original_start_date | n | rn |
+-----------------+-------------+-----------+---------------------+------+----+
| XXXX | XXXX | 88 | 2021-09-02 | 1 | 1 |
| 5041 | A0EH | 96 | 2021-09-05 | 2 | 1 |
+-----------------+-------------+-----------+---------------------+------+----+
2 rows in set
Raw test data:
SELECT * FROM earliest_start_date;
+-----------------+---------------------+-----------+---------------------+
| employee_number | old_employee_number | entity_id | original_start_date |
+-----------------+---------------------+-----------+---------------------+
| 5041 | A0EH | 96 | 2021-09-10 |
| A0EH | NULL | 96 | 2021-09-05 |
| XXXX | NULL | 88 | 2021-09-02 |
+-----------------+---------------------+-----------+---------------------+
Note that the logic makes assumption about uniqueness of the employee_number and, in the current form, can't handle cases where the employee_number is reused by the same employee or used again with a different employee without adjusting prior data. There may not be enough detail in the current structure to handle those cases.
Our data table looks like this:
Machine Name
Lot Number
Qty
Load TxnDate
Unload TxnDate
M123
ABC
500
10/1/2020
10/2/2020
M741
DEF
325
10/1/2020
M123
ZZZ
100
10/5/2020
10/7/2020
M951
AAA
550
10/5/2020
10/9/2020
M123
BBB
550
10/7/2020
I need to create an SQL query that shows the currently loaded Lot number - Machines with no Unload TxnDate - and the last loaded Lot number based on the unload TxnDate.
So in the example, when I run a query for M123, the result will show:
Machine Name
Lot Number
Qty
Load TxnDate
Unload TxnDate
M123
ZZZ
100
10/5/2020
10/7/2020
M123
BBB
550
10/7/2020
As you can see although Machine Name has 3 records, the results only show the currently loaded and the last loaded. Is there anyway to replicate this? The Machine Name is dynamic, so my user can enter the Machine Name and see the results the machine based on the missing Unload TxnDate and the last Unload Txn Date
You seem to want the last two rows. That would be something like this:
select t.*
from t
where machine_name = 'M123'
order by load_txn_date desc
fetch first 2 rows only;
Note: not all databases support the first first clause. Some spell it limit, or select top, or even something else.
If you want two rows per machine, one option uses window functions:
select *
from (
select t.*,
row_number() over(
partition by machine_name, (case when unload_txn_date is null then 0 else 1 end)
order by coalesce(unload_txn_date, load_txn_date) desc
) rn
from mytable t
) t
where rn = 1
The idea is to separate rows between those that have an unload date, and those that do not. We can then bring the top record per group.
For your sample data, this returns:
Machine_Name | Lot_Number | Qty | Load_Txn_Date | Unload_Txn_Date | rn
:----------- | :--------- | --: | :------------ | :-------------- | -:
M123 | BBB | 550 | 2020-10-07 | null | 1
M123 | ZZZ | 100 | 2020-10-05 | 2020-10-07 | 1
M741 | DEF | 325 | 2020-10-01 | null | 1
M951 | AAA | 550 | 2020-10-05 | 2020-10-09 | 1
You might use the following query, presuming that you're on a database having Window(or Analytic) Function
WITH t AS
(
SELECT COALESCE(Unload_Txn_Date -
LAG(Load_Txn_Date) OVER
(PARTITION BY Machine_Name ORDER BY Load_Txn_Date DESC),0) AS lg,
MAX(CASE WHEN Unload_Txn_Date IS NULL THEN Load_Txn_Date END) OVER
(PARTITION BY Machine_Name) AS mx,
t.*
FROM tab t
), t2 AS
(
SELECT DENSE_RANK() OVER (ORDER BY mx DESC NULLS LAST) AS dr, t.*
FROM t
WHERE mx IS NOT NULL
)
SELECT Machine_Name,Lot_Number,Qty,Load_Txn_Date,Unload_Txn_Date
FROM t2
WHERE dr = 1 AND lg = 0
ORDER BY Load_Txn_Date
where if previous row's Unload_Txn_Date is equal to the current Load_Txn_Date, then it's accepted that there's no interruption will occur for the job, while determining the last Unload Txn Dates with no unload date values per each machine. And then, the result set returns through filtering by the values derived from the window functions within the penultimate query.
Demo
How do you do to retrieve only the max value of a group with only consecutive values?
I have a telephone database with only unique values and I want to get only the highest number of each telephone number group TelNr and I am struggling.
id | TeNr | Position
1 | 100 | SLMO2.1.3
2 | 101 | SLMO2.3.4
3 | 103 | SLMO2.4.1
4 | 104 | SLMO2.3.2
5 | 200 | SLMO2.5.1
6 | 201 | SLMO2.5.2
7 | 204 | SLMO2.5.5
8 | 300 | SLMO2.3.5
9 | 301 | SLMO2.6.2
10 | 401 | SLMO2.4.8
Result should be:
TelNr
101
104
201
204
301
401
I have tried almost every tip I could find so far and whether I get all TelNr or no number at all which is useless in my case.
Any brilliant idea to run this with SQLITE?
So you're searching for gaps and want to get the first value of those gaps.
This is probably the best way to get them, try to check for a row with the current TeNr plus 1 and if there's none you found it:
select t1.TeNr, t1.TeNr + 1 as unused_TeNr
from tab as t1
left join Tab as t2
on t2.TeNr = t1.TeNr + 1
where t2.TeNr is null
Edit:
To get the range of missing values you need to use some old-style SQL as SQLite doesn't seem to support ROW_NUMBER, etc.
select
TeNr + 1 as RangeStart,
nextTeNr - 1 as RangeEnd,
nextTeNr - TeNr - 1 as cnt
from
(
select TeNr,
( select min(TeNr) from tab as t2
where t2.TeNr > t1.TeNr ) as nextTeNr
from tab as t1
) as dt
where nextTeNr > TeNr + 1
It's probably not very efficient, but might be ok if the number of rows is small and/or there's a index on TeNr.
Getting each value in the gap as a row in your result set is very hard, if your version of SQLite supports recursive queries:
with recursive cte (TeNr, missing, maxTeNr) as
(
select
min(TeNr) as TeNr, -- start of range of existing numbers
0 as missing, -- 0 = TeNr exists, 1 = TeNr is missing
max(TeNr) as maxTeNr -- end of range of existing numbers
from tab
union all
select
cte.TeNr + 1, -- next TeNr, if it doesn't exists tab.TeNr will be NULL
case when tab.TeNr is not null then 0 else 1 end,
maxTeNr
from cte left join tab
on tab.TeNr = cte.TeNr + 1
where cte.TeNr + 1 < maxTeNr
)
select TeNr
from cte
where missing = 1
Depending on your data this might return a huge amount of rows.
You might also use the result of the previous RangeStart/RangeEnd query as input to this recursion.
In a SQL Server DB, I have a table of values that I am interested in ranking.
When I perform a RANK() OVER (ORDER BY VALUE DESC) as RANK, I get the following results (in a hypothetical table):
RANK | USER_ID | VALUE
------------------------
1 | 33 | 30000
2 | 10 | 20000
3 | 45 | 10000
4 | 12 | 5000
5 | 43 | 2000
6 | 32 | NULL
6 | 13 | NULL
6 | 19 | NULL
6 | 28 | NULL
The problem is, I do not want the rows which have NULL for a VALUE to get a rank - I need some way to set the rank for these to NULL. So far, searching the web has brought me no answers on how I might be able to do this.
Thanks for any help you can provide.
You can try a CASE statement:
SELECT
CASE WHEN Value IS NULL THEN NULL
ELSE RANK() OVER (ORDER BY VALUE DESC)
END AS RANK,
USER_ID,
VALUE
FROM yourtable
The CASE statement provided earlier would count the NULL records in the rank if the SORT BY was ascending rather than descending. This would start the ranking at 5 rather than 1 - probably not what is desired.
To ensure that the nulls do not get counted in the rank, you can force them to the bottom by adding an initial sort criteria on whether the value IS NULL or not, like so:
SELECT
CASE WHEN Value IS NULL THEN NULL
ELSE RANK() OVER
(ORDER BY CASE WHEN Value IS NULL THEN 1 ELSE 0 END, VALUE DESC)
END AS RANK,
USER_ID,
VALUE
FROM yourtable
*** credit to Hugo Kornelis: https://social.msdn.microsoft.com/Forums/sqlserver/en-US/deb8a0aa-aaab-442b-a667-11220333a4e0/rank-without-counting-null-values?forum=transactsql
I need to create a view that automatically adds virtual row number in the result. the graph here is totally random all that I want to achieve is the last column to be created dynamically.
> +--------+------------+-----+
> | id | variety | num |
> +--------+------------+-----+
> | 234 | fuji | 1 |
> | 4356 | gala | 2 |
> | 343245 | limbertwig | 3 |
> | 224 | bing | 4 |
> | 4545 | chelan | 5 |
> | 3455 | navel | 6 |
> | 4534345| valencia | 7 |
> | 3451 | bartlett | 8 |
> | 3452 | bradford | 9 |
> +--------+------------+-----+
Query:
SELECT id,
variety,
SOMEFUNCTIONTHATWOULDGENERATETHIS() AS num
FROM mytable
Use:
SELECT t.id,
t.variety,
(SELECT COUNT(*) FROM TABLE WHERE id < t.id) +1 AS NUM
FROM TABLE t
It's not an ideal manner of doing this, because the query for the num value will execute for every row returned. A better idea would be to create a NUMBERS table, with a single column containing a number starting at one that increments to an outrageously large number, and then join & reference the NUMBERS table in a manner similar to the variable example that follows.
MySQL Ranking, or Lack Thereof
You can define a variable in order to get psuedo row number functionality, because MySQL doesn't have any ranking functions:
SELECT t.id,
t.variety,
#rownum := #rownum + 1 AS num
FROM TABLE t,
(SELECT #rownum := 0) r
The SELECT #rownum := 0 defines the variable, and sets it to zero.
The r is a subquery/table alias, because you'll get an error in MySQL if you don't define an alias for a subquery, even if you don't use it.
Can't Use A Variable in a MySQL View
If you do, you'll get the 1351 error, because you can't use a variable in a view due to design. The bug/feature behavior is documented here.
Oracle has a rowid pseudo-column. In MySQL, you might have to go ugly:
SELECT id,
variety,
1 + (SELECT COUNT(*) FROM tbl WHERE t.id < id) as num
FROM tbl
This query is off the top of my head and untested, so take it with a grain of salt. Also, it assumes that you want to number the rows according to some sort criteria (id in this case), rather than the arbitrary numbering shown in the question.