I need to delete all the rows with a max value of duty_perd_id where the rotn_prng_nbr and empl_nbr are the same (not the same to each other, but the max where of all of the rows where those two remain constant). From the table below it should delete rows 3,7 and 9.
rotn_prng_nbr
empl_nbr
duty_perd_id
B93
12
1
B93
12
2
B93
12
3
B21
12
1
B21
12
2
B21
12
3
B21
12
4
B21
18
1
B21
18
2
using SAS EG. Right now all have is below:
Option 1:
create table middle_legs as
select t.*
from actual_flt_leg as t
where t.duty_perd_id < (select max(t2.duty_perd_id)
from actual_flt_leg as t2
where t2.rotn_prng_nbr = t.rotn_prng_nbr and
t2.empl_nbr = t.empl_nbr
);
this works exactly as intended, but is incredibly slow. The other thought that I had but couldnt quite finish was as follows.
Option 2:
create table last_duty_day as
Select * from actual_flt_leg
inner join (
select actual_flt_leg.Rotn_Prng_Nbr,actual_flt_leg.empl_nbr, max(duty_perd_id) as last_duty
from actual_flt_leg
group by actual_flt_leg.Rotn_Prng_Nbr, actual_flt_leg.empl_nbr
) maxtable on
actual_flt_leg.Rotn_Prng_Nbr = maxtable.Rotn_Prng_Nbr
and actual_flt_leg.empl_Nbr = maxtable.empl_Nbr
and actual_flt_leg.duty_perd_id = maxtable.last_duty;
option 2 finds all the highest duty_perd_id for the given pair, and I was wondering if there was any "reverse join" that could only show the rows from the original table that do not match this new table i created in option 2.
If there is a way to make option 1 faster, finish option 2, or anything else i cant think of id appreciate it. Thanks!
You are almost there. You just want <:
Select *
from actual_flt_leg inner join
(select actual_flt_leg.Rotn_Prng_Nbr,actual_flt_leg.empl_nbr, max(duty_perd_id) as last_duty
from actual_flt_leg
group by actual_flt_leg.Rotn_Prng_Nbr, actual_flt_leg.empl_nbr
) maxtable
on actual_flt_leg.Rotn_Prng_Nbr = maxtable.Rotn_Prng_Nbr and
actual_flt_leg.empl_Nbr = maxtable.empl_Nbr and
actual_flt_leg.duty_perd_id < maxtable.last_duty;
In SAS SQL, this is pretty easy:
data have;
input rotn_prng_nbr $ empl_nbr duty_perd_id;
datalines;
B93 12 1
B93 12 2
B93 12 3
B21 12 1
B21 12 2
B21 12 3
B21 12 4
B21 18 1
B21 18 2
;;;;
run;
proc sql;
select *
from have
group by rotn_prng_nbr, empl_nbr
having duty_perd_id lt max(duty_perd_id);
quit;
This isn't legit SQL in any other system that I've ever seen, but it works in SAS. You can group by a set of variables while still using select for all of the variables including ones not on group by; SAS just does two queries and merges them behind the scenes for you.
NOTE: The query requires remerging summary statistics back with the original data.
As far as I understand the actual under the hood result is exactly identical to the more "compatible" version Gordon suggests; just a matter of whether you prefer typing less or more compatible SQL code.
Related
My data is as follows:
ID
1
2
3
3
4
5
6
6
I want to create a column that indicates the uniqueness of a value in the ID column as such:
ID COUNT
1 1
2 1
3 1
3 0
4 1
5 1
6 1
6 0
I'd like to do this without creating a temporary table, via a subquery or something. Any assistance would be much appreciated.
One option would be to use by functionality in the data step:
data have;
input ID;
datalines;
1
2
3
3
4
5
6
6
;run;
data want;
set have;
by ID;
if first.ID then count = 1;
else count = 0;
run;
That type of logic is not really amenable to SQL since the order of observations is not really insured. In a more modern version of SQL you could use windowing functions (like ROW_NUMBER() with PARTITION BY) to impose an record count.
If you really wanted to try to do it just in PROC SQL you might need to resort to using the undocumented MONOTONIC() function. But even then to defeat the optimizer eliminating the duplicate rows you might need to make a temporary table with the row counter first.
data have;
input ID ##;
datalines;
1 2 3 3 4 5 6 6
;
proc sql ;
create table _temp_ as select id,monotonic() as row from have;
create table want as
select a.id
, b.row=min(b.row) as FLAG
from have a,_temp_ b
where a.id=b.id
group by a.id
order by 1,2
;
quit;
I need to accomplish the following :
I have a table with multiple column (c1, c2, c3, c4 ... cn).
I want a query that would return multiple rows in the following fashion (r1 r2 .. rx are the rows in the original table) :
r1c1 r1c2 r1c3
r1c4 r1c5 r1c6
...
r1cn-2 r1cn-1 r1cn
r2c1 r2c2 r2c3
r2c4 r2c5 r2c6
...
r2cn-2 r2cn-1 r2cn
...
rxc1 rxc2 rxc3
rxc4 rxc5 rxc6
...
rxcn-2 rxn-1 rxcn
I know I can use unions and repeat basically the same query n times, but I need to use that query in a web based reporting system that I have no control over and the query is to big for the maximum number of characters allowed in queries.
Any suggestions ?
Thank you !
EDIT : FYI I'm building a report in a report tool I can't change using a database I can't change. So using custom functions/procedures is not a solution. It has to be a PL-SQL query.
To be more specific, i need to have multiple rows from the original row, lets say row 1 is
a b c d e f h i j
and row 2 is
1 2 3 4 5 6 7 8 9
then I would get the following table with 3 columns :
a b c
d e f
h i j
1 2 3
4 5 6
7 8 9
So number 1, if you have a set number of columns in the original table that is divisible by 3, you can just do that many UNION ALL.
Your only other options are pivot or a pointer, neither of which is going to be any better.
The simple case when n is divisible by 3 just use rownum and union:
WITH T1 AS
(
SELECT rownum as rn,1 as tNum, c1 as s1,c2 as s2,c3 as s3 FROM T
UNION ALL
SELECT rownum as rn,2 as tNum, c4 as s1,c5 as s2,c6 as s3 FROM T
UNION ALL
SELECT rownum as rn,3 as tNum, c7 as s1,c8 as s2,c9 as s3 FROM T
)
SELECT s1,s2,s3 FROM T1
ORDER BY rn,tNum
SQLFiddle demo
I have a table with 3 columns (sorted by the first two):
letter
number (sorted for each letter)
difference between current number and previous number of the same letter
I'd like to calculate (with vanlla SQL) a fourth new column RESULT to group these data when the third column (difference of number between contiguos record; i.e #2 --> 4 = 5-1) is greater than 30 marking all the records of this interval with letter-number of the first record (i.e A1 for #1,#2,#3).
Since the difference between contiguos numbers makes sense just for records with the same letter, for the first record of a new letter, the value of differnce is 31 (meaning that it's a new group; i.e. #6).
Here is what I'd like to get as result:
# Letter Number Difference RESULT (new column)
1 A 1 1 A1
2 A 5 4 A1
3 A 7 2 A1
4 A 40 33 A40 (*)
5 A 43 3 A40
6 B 1 31 B1 (*)
7 B 25 24 B1
8 B 27 2 B1
9 B 70 43 B70 (*)
10 B 75 5 B70
Now I can only find the "breaking values" (*) with this query where they get a value of 1:
select letter
,number
,cast(difference/30 as int) break
from table
where cast(difference/30 as int) = 1
Even though I'm able to find these breaking values I can't finish my task.
Can anyone help me finding a way to obtain the column RESULT?
Thanks in advance
FF
As I understand you need to construct the last result column. You can use concat to do that:
SELECT letter
,number
,concat(letter, cast(difference/30 as int)) result
FROM table
HAVING result = 'A1'
after some exercise and a little help from a friend of mine, I've found a possible solution to my sql prolblem.
The only requirment for the solution is that my first record must have a value of 31 in Difference field (since I need "breaks" when Difference > 30 than the previous record).
Here is the query to get the column RESULT I needed:
select alls.letter
,alls.number
,ints.letter||ints.number as result
from competition.lag alls
,(select letter
,number
,difference
,result
from (select letter
,number
,difference
,case when difference>30 then 1 else 2 end as result
from competition.lag
) temp
where result = 1
) ints
where ints.letter=alls.letter
and alls.number>=ints.number
and alls.number-30<=ints.number
I have a source data set like this (simplified to be more clear):
Key F1 F2
1 X 4
2 X 5
3 Y 6
4 X 9
5 X 7
6 X 8
7 Y 9
8 X 6
9 X 5
10 Y 3
The data is sorted by the Key field. Now, I want to compute an aggregate of the F2 field over partitions that are defined by the F1 field: A partition starts at the first X value and ends with the first subsequent Y value.
So, for example, I might want wo compute the MIN() over the partitions defined as described above. Then the result set would look like this:
rownum MIN(F2)
1 4
2 7
3 3
I have tried a number of resources (incl. our own intranet community and of course stackoverflow) but found nothing for my case. Usually partitioning only works with a field that can be used to identify the partitions. Here, the partitions are defined by a change in a field's content with respect to a given order.
Although I am aware that I may have to resort to writing a procedural solution I would prefer to solve this in pure SQL.
Any ideas how such a partitioning could be achieved with a SQL select statement?
Thanks and regards
Kai.
A little bit shorter solution: http://sqlfiddle.com/#!12/7390d/24
Query:
select min(f2)
from t t1
group by (select max(key)
from t t2
where t2.f1='Y' and
t1.key > t2.key)
Result:
| MIN |
-------
| 4 |
| 7 |
| 3 |
The idea is to find the key of preceding 'Y' for each row and group by it. Should work with any SQL engine.
You didn't specify engine or dialect or version so I assumed SQL Server 2012.
Example that you can run to see the solution: http://sqlfiddle.com/#!6/f5d38/21
You solve it by creating correct partitions in your set. Code looks like this.
WITH groupLimits as
(
SELECT
[Key] AS groupend
,COALESCE(LAG([Key]) OVER (order by [Key]),0)+1 AS groupstart
FROM sourceData
WHERE F1 = 'Y'
)
SELECT
MIN(sourceData.F2)
FROM groupLimits
INNER JOIN sourceData
ON sourceData.[Key] BETWEEN groupLimits.groupstart and groupLimits.groupend
GROUP BY groupLimits.groupstart
ORDER BY groupLimits.groupstart
Here is a sample of my data (n>3000) that ties two numbers together:
id a b
1 7028344 7181310
2 7030342 7030344
3 7030354 7030353
4 7030343 7030345
5 7030344 7030342
6 7030364 7008059
7 7030659 7066051
8 7030345 7030343
9 7031815 7045692
10 7032644 7102337
Now, the problem is that id=2 is a duplicate of id=5 and id=4 is a duplicate of id=8. So, when I tried to write if-then statements to map column a to column b, basically the numbers just get swapped. There are many cases like this in my full data.
So, my question is to identify the duplicate(s) and somehow delete one of the duplicates (either id=2 or id=5). And I preferably want to do this in Excel but I could work with SQL Server or SAS, too.
Thank you in advance. Please comment if my question is not clear.
What I want:
id a b
1 7028344 7181310
2 7030342 7030344
3 7030354 7030353
4 7030343 7030345
6 7030364 7008059
7 7030659 7066051
9 7031815 7045692
10 7032644 7102337
All sorts of ways to do this.
In SAS or SQL, this is simple (for SQL Server, the SQL portion should be identical or nearly so):
data have;
input id a b;
datalines;
1 7028344 7181310
2 7030342 7030344
3 7030354 7030353
4 7030343 7030345
5 7030344 7030342
6 7030364 7008059
7 7030659 7066051
8 7030345 7030343
9 7031815 7045692
10 7032644 7102337
;;;;
run;
proc sql undopolicy=none;
delete from have H where exists (
select 1 from have V where V.id < H.id
and (V.a=H.a and V.b=H.b) or (V.a=H.b and V.b=H.a)
);
quit;
The excel solution would require creating an additional column I believe with the concatenation of the two strings, in order (any order will do) and then a lookup to see if that is the first row with that value or not. I don't think you can do it without creating an additional column (or using VBA, which if you can use that will have a fairly simple solution as well).
Edit:
Actually, the excel solution IS possible without creating a new column (well, you need to put this formula somewhere, but without ANOTHER additional column).
=IF(OR(AND(COUNTIF(B$1:B1,B2),COUNTIF(C$1:C1,C2)),AND(COUNTIF(B$1:B1,C2),COUNTIF(C$1:C1,B2))),"DUPLICATE","")
Assuming ID is in A, B and C contain the values (and there is no header row). That formula goes in the second row (ie, B2/C2 values) and then is extended to further rows (so row 36 will have the arrays be B1:B35 and C1:C35 etc.). That puts DUPLICATE in the rows which are duplicates of something above and blank in rows that are unique.
I haven't tested this out but here is some food for thought, you could join the table against itself and get the ID's that have duplicates
SELECT
id, a, b
FROM
[myTable]
INNER JOIN ( SELECT id, a, b FROM [myTable] ) tbl2
ON [myTable].a = [tbl2].b
OR [myTable].b = tbl2.a