I am trying to set up a report that queries for data within five minutes of each other. This way, I can recognize that one of my machines is down. This is what I have so far:
SELECT A_M_DEVICE.M_Device_ID,
A_M_DEVICE.Machine_Name,
A_M_DEVICE.IP_Address,
A_M_DEVICE.Device_Type,
A_M_DEVICE.M_Device_Status_ID,
A_M_DEVICE.Status_Date,
S_M_DEVICE_STATUS.M_Device_Status_Desc,
A_M_DEVICE.Domain_Name FROM MConsole.dbo.A_M_DEVICE A_M_DEVICE
INNER JOIN
MConsole.dbo.S_M_DEVICE_STATUS S_M_DEVICE_STATUS
ON (A_M_DEVICE.M_Device_Status_ID =
S_M_DEVICE_STATUS.M_Device_Status_ID) WHERE (A_M_DEVICE.Machine_Name IN
('DeVA',
'DevB',
))
AND (A_M_DEVICE.Status_Date = DateDiff (Second) <= 300)
Image not allowed since I am a newbie. Else I would have posted one.
Alright - looks like I have enough reputations for a screenshot!
Any help, as always, will be highly appreciated.
Thank you in advance.
Here's an approximation - check for "< 5" vs. "> 5" etc.
SELECT M_Device_ID,
Machine_Name,
IP_Address,
Device_Type,
M_Device_Status_ID,
M_Device_Status_Desc,
Domain_Name
FROM
( SELECT M_Device_ID,
Machine_Name,
IP_Address,
Device_Type,
M_Device_Status_ID,
M_Device_Status_Desc,
Domain_Name,
MAX(Status_Date)
FROM A_M_DEVICE
GROUP BY
M_Device_ID,
Machine_Name,
IP_Address,
Device_Type,
M_Device_Status_ID,
M_Device_Status_Desc,
Domain_Name
) AS a
JOIN A_M_DEVICE AS b
ON
a.Machine_Name = b.Machine_Name
AND a.IP_Address = b.IP_Address
AND a.Device_Type = b.Device_Type
AND a.M_Device_Status_ID = b.M_Device_Status_ID
AND a.M_Device_Status_Desc = b.M_Device_Status_Desc
AND a.Domain_Name = b.Domain_Name
WHERE
DATEDIFF(minute, a.Status_Date, b.Status_Date) < 5
Select the rows that have a status date that doesn't match the MAX status date:
SELECT A_M_DEVICE.M_Device_ID,
A_M_DEVICE.Machine_Name,
A_M_DEVICE.IP_Address,
A_M_DEVICE.Device_Type,
A_M_DEVICE.M_Device_Status_ID,
A_M_DEVICE.Status_Date,
S_M_DEVICE_STATUS.M_Device_Status_Desc,
A_M_DEVICE.Domain_Name FROM MConsole.dbo.A_M_DEVICE A_M_DEVICE
INNER JOIN
MConsole.dbo.S_M_DEVICE_STATUS S_M_DEVICE_STATUS
ON (A_M_DEVICE.M_Device_Status_ID =
S_M_DEVICE_STATUS.M_Device_Status_ID) WHERE (A_M_DEVICE.Machine_Name IN
('DeVA',
'DevB',
))
AND (A_M_DEVICE.Status_Date NOT IN (SELECT MAX(A_M_DEVICE.Status_Date) FROM A_M_DEVICE)
The date will be identical for all rows that represent machines that are operational - so eliminating those that match the MAX date leave records representing machines that haven't had a status update.
Related
I'd like to know how I could improve the performance of the below query, since it is taking way too long to run, after all, it returns millions of rows... I'm a dummy when it comes to SQL...
SELECT CIAM.EXTERNAL_ID,
(SELECT NEW_CHARGES / 100
FROM BI_OWNER.CMF_BALANCE
WHERE ( ACCOUNT_NO, BILL_REF_NO ) = (SELECT ACCOUNT_NO,
MAX(BILL_REF_NO)
FROM BI_OWNER.CMF_BALANCE
WHERE
ACCOUNT_NO = CIAM.ACCOUNT_NO
GROUP BY ACCOUNT_NO))
"AMOUNT LAST BILL",
(SELECT 'ACTIVE DISCOUNT'
|| ' '
|| CCK.AVAIL_PERIODS
|| '/'
|| CC.TOTAL_PERIODS
FROM BI_OWNER.CUSTOMER_CONTRACT_KEY CCK,
BI_OWNER.CUSTOMER_CONTRACT CC
WHERE CC.PARENT_ACCOUNT_NO = CIAM.ACCOUNT_NO
AND CC.END_DT IS NULL
AND EXISTS (SELECT 1
FROM CONTRACT_TYPES
WHERE CONTRACT_TYPE = CC.CONTRACT_TYPE
AND PLAN_ID_DISCOUNT IS NOT NULL
AND DURATION_UNITS = -3)
AND ROWNUM = 1
AND CCK.TRACKING_ID = CC.TRACKING_ID
AND CCK.TRACKING_ID_SERV = CC.TRACKING_ID_SERV) "DISCOUNT",
(SELECT CC.TOTAL_PERIODS
FROM BI_OWNER.CUSTOMER_CONTRACT_KEY CCK,
BI_OWNER.CUSTOMER_CONTRACT CC
WHERE CC.PARENT_ACCOUNT_NO = CIAM.ACCOUNT_NO
AND CC.END_DT IS NULL
AND EXISTS (SELECT 1
FROM CONTRACT_TYPES
WHERE CONTRACT_TYPE = CC.CONTRACT_TYPE
AND PLAN_ID_DISCOUNT IS NOT NULL
AND DURATION_UNITS = -3)
AND ROWNUM = 1
AND CCK.TRACKING_ID = CC.TRACKING_ID
AND CCK.TRACKING_ID_SERV = CC.TRACKING_ID_SERV) "CYCLE"
,
(SELECT SUM(BALANCE_DUE)
FROM BI_OWNER.CMF_BALANCE
WHERE ACCOUNT_NO = CIAM.ACCOUNT_NO
AND PPDD_DATE < TRUNC(SYSDATE))
"DEBT"
FROM BI_OWNER.CUSTOMER_ID_ACCT_MAP CIAM
WHERE EXTERNAL_ID_TYPE = 1
AND EXISTS (SELECT 1
FROM BI_OWNER.CMF
WHERE ACCOUNT_NO = CIAM.ACCOUNT_NO
AND PREV_CUTOFF_DATE > SYSDATE - 30)
I would recommend identifying the SQL id for the query then using the SQL Monitor Report as it will tell you exactly what the execution plan is and where the SQL is spending most of it's time.
A simple way to get the SQL Monitor Report from SQL*Plus follows:
spool c:\temp\SQL_Monitor_rpt.html
SET LONG 1000000
SET LONGCHUNKSIZE 1000000
SET LINESIZE 1000
SET PAGESIZE 0
SET TRIM ON
SET TRIMSPOOL ON
SET ECHO OFF
SET FEEDBACK OFF
alter session set "_with_subquery" = optimizer;
SELECT DBMS_SQLTUNE.report_sql_monitor(
sql_id => '&SQLID' ,
type => 'HTML',
report_level => 'ALL') AS report
FROM dual;
spool off
Basically, you need to know your table sizes and how to get the large tables to have data access via an index (e.g. index on columns found in the where clause).
Here is an initial stab and may provide significant improvement. Many of your queries were correlated subqueries being executed for every record. Instead, I tried to build pre-query aggregates per account number in the select from/join section. Query first, then I'll explain logic after.
SELECT
CIAM.EXTERNAL_ID,
CMF_BALANCE.New_Charges / 100.0 "AMOUNT LAST BILL",
CCKs.Discount,
CCKs.Cycle,
AcntLast30.SumBalance "DEBT"
FROM
(SELECT
CMF.Account_No,
max( Bal.Bill_Ref_No ) MaxBillRef,
sum( case when Bal.PPDD_Date < TRUNC(SYSDATE )
then Bal.Balance_Due else 0 end ) SumBalance
from
BI_OWNER.CMF
JOIN BI_OWNER.CMF_BALANCE BAL
on CMF.Account_No = Bal.Account_No
where
CMF.PREV_CUTOFF_DATE > SYSDATE - 30
group by
CMF.Account_No ) AcntLast30
JOIN BI_OWNER.CUSTOMER_ID_ACCT_MAP CIAM
on AcntLast30.Account_No = CIAM.Account_No
AND CIAM.EXTERNAL_ID_TYPE = 1
JOIN BI_OWNER.CMF_BALANCE
on AcntLast30.Account_No = CMFBalance.Account_No
AND AcntLast30.MaxBillRef = CMFBalance.Bill_Ref_No
JOIN
(select
CC.Parent_Account_No,
CC.TOTAL_PERIODS "CYCLE",
'ACTIVE DISCOUNT' || ' ' || CCK.AVAIL_PERIODS || '/' || CC.TOTAL_PERIODS "DISCOUNT"
FROM
BI_OWNER.CUSTOMER_CONTRACT CC
JOIN BI_OWNER.CUSTOMER_CONTRACT_KEY CCK
ON CC.TRACKING_ID = CCK.TRACKING_ID
AND CC.TRACKING_ID_SERV = CCK.TRACKING_ID_SERV
AND ROWNUM = 1
JOIN ( select distinct Contract_Type
FROM CONTRACT_TYPES
WHERE PLAN_ID_DISCOUNT IS NOT NULL
AND DURATION_UNITS = -3) CT
on CC.Contract_Type = CT.Contract_Type
WHERE
CC.END_DT IS NULL ) CCKs
on AcntLast30.Account_No = CCKs.Parent_Account_No
The initial "FROM" clause, I have a subquery because you appear to be only interested in accounts within the last 30 days. So, while I'm there, I am joining to your CMF_Balance table and getting the maximum Bill_Ref_No per account AND the sum of the balance when the PPDD_Date is less than the TRUNC(sysdate) which is your "DEBT" result column. So now we have the finite list of accounts you are interested in with the account, max bill on file and the balance due summed up.
(SELECT
CMF.Account_No,
max( Bal.Bill_Ref_No ) MaxBillRef,
sum( case when Bal.PPDD_Date < TRUNC(SYSDATE )
then Bal.Balance_Due else 0 end ) SumBalance
from
BI_OWNER.CMF
JOIN BI_OWNER.CMF_BALANCE BAL
on CMF.Account_No = Bal.Account_No
where
CMF.PREV_CUTOFF_DATE > SYSDATE - 30
group by
CMF.Account_No ) AcntLast30
Next, simple join to the CIAM table to only get accounts for External_ID_Type = 1. This too could have been merged into the query above for "AcntLast30" alias result.
JOIN BI_OWNER.CUSTOMER_ID_ACCT_MAP CIAM
on AcntLast30.Account_No = CIAM.Account_No
AND CIAM.EXTERNAL_ID_TYPE = 1
Now, since the "AcntLast30" query has the account and max bill reference we then join back to the CMF_Balance on the account and bill reference # once thus giving us the CMF_BALANCE.New_Charges / 100.0 "AMOUNT LAST BILL"
JOIN BI_OWNER.CMF_BALANCE
on AcntLast30.Account_No = CMFBalance.Account_No
AND AcntLast30.MaxBillRef = CMFBalance.Bill_Ref_No
Finally the subquery alias result "CCKs". Since the Discount and Cycle use the same query/subquery/exists, I just ran it once that qualified on the discounts types and pulled the Account_No for the JOIN condition. Now we have the Discount and Cycle values per account.
If you are returning so many rows, I believe the performance gained by grabbing these pre-query aggregates once up-front and joining to by the account will be much faster than that of each time individually subquerying at every row.
There was a reference to ROWNUM without any table/alias reference so I am not sure the impact of that one within the query.
Final note. For things like the discount that may not be applicable, you may need to change it to a LEFT JOIN, of which those values would show as NULL. But without knowing the extent of data, Cartesian products of 1:many entries in given tables, I think this will work well for you. For the most part it looked like everything was resulting in only one record qualified per account where higher importance on join (such as the max bill reference).
The code below joins two tables and I need to extract only the latest date per account, though it holds multiple accounts and history records. I wanted to use the MAX function, but not sure how to incorporate it for this case. I am using My SQL server.
Appreciate any help !
select
PROP.FileName,PROP.InsName, PROP.Status,
PROP.FileTime, PROP.SubmissionNo, PROP.PolNo,
PROP.EffDate,PROP.ExpDate, PROP.Region,
PROP.Underwriter, PROP_DATA.Data , PROP_DATA.Label
from
Property.dbo.PROP
inner join
Property.dbo.PROP_DATA on Property.dbo.PROP.FileID = Actuarial.dbo.PROP_DATA.FileID
where
(PROP_DATA.Label in ('Occupancy' , 'OccupancyTIV'))
and (PROP.EffDate >= '42278' and PROP.EffDate <= '42643')
and (PROP.Status = 'Bound')
and (Prop.FileTime = Max(Prop.FileTime))
order by
PROP.EffDate DESC
Assuming your DBMS supports windowing functions and the with clause, a max windowing function would work:
with all_data as (
select
PROP.FileName,PROP.InsName, PROP.Status,
PROP.FileTime, PROP.SubmissionNo, PROP.PolNo,
PROP.EffDate,PROP.ExpDate, PROP.Region,
PROP.Underwriter, PROP_DATA.Data , PROP_DATA.Label,
max (PROP.EffDate) over (partition by PROP.PolNo) as max_date
from Actuarial.dbo.PROP
inner join Actuarial.dbo.PROP_DATA
on Actuarial.dbo.PROP.FileID = Actuarial.dbo.PROP_DATA.FileID
where (PROP_DATA.Label in ('Occupancy' , 'OccupancyTIV'))
and (PROP.EffDate >= '42278' and PROP.EffDate <= '42643')
and (PROP.Status = 'Bound')
and (Prop.FileTime = Max(Prop.FileTime))
)
select
FileName, InsName, Status, FileTime, SubmissionNo,
PolNo, EffDate, ExpDate, Region, UnderWriter, Data, Label
from all_data
where EffDate = max_date
ORDER BY EffDate DESC
This also presupposes than any given account would not have two records on the same EffDate. If that's the case, and there is no other objective means to determine the latest account, you could also use row_numer to pick a somewhat arbitrary record in the case of a tie.
Using straight SQL, you can use a self-join in a subquery in your where clause to eliminate values smaller than the max, or smaller than the top n largest, and so on. Just set the number in <= 1 to the number of top values you want per group.
Something like the following might do the trick, for example:
select
p.FileName
, p.InsName
, p.Status
, p.FileTime
, p.SubmissionNo
, p.PolNo
, p.EffDate
, p.ExpDate
, p.Region
, p.Underwriter
, pd.Data
, pd.Label
from Actuarial.dbo.PROP p
inner join Actuarial.dbo.PROP_DATA pd
on p.FileID = pd.FileID
where (
select count(*)
from Actuarial.dbo.PROP p2
where p2.FileID = p.FileID
and p2.EffDate <= p.EffDate
) <= 1
and (
pd.Label in ('Occupancy' , 'OccupancyTIV')
and p.Status = 'Bound'
)
ORDER BY p.EffDate DESC
Have a look at this stackoverflow question for a full working example.
Not tested
with temp1 as
(
select foo
from bar
whre xy = MAX(xy)
)
select PROP.FileName,PROP.InsName, PROP.Status,
PROP.FileTime, PROP.SubmissionNo, PROP.PolNo,
PROP.EffDate,PROP.ExpDate, PROP.Region,
PROP.Underwriter, PROP_DATA.Data , PROP_DATA.Label
from Actuarial.dbo.PROP
inner join temp1 t
on Actuarial.dbo.PROP.FileID = t.dbo.PROP_DATA.FileID
ORDER BY PROP.EffDate DESC
I'm struggling here trying to write a script that finds where an order was returned multiple times by the same associate (count greater than 1). I'm guessing my syntax with the subquery is incorrect. When I run the script, I get a message back that the "SELECT failed.. [3669] More than one value was returned by the subquery."
I'm not tied to the subquery, and have tried using just the group by and having statements, but I get an error regarding a non-aggregate value. What's the best way to proceed here and how do I fix this?
Thank you in advance - code below:
SEL s.saletran
, s.saletran_dt SALE_DATE
, r.saletran_id RET_TRAN
, r.saletran_dt RET_DATE
, ra.user_id RET_ASSOC
FROM salestrans s
JOIN salestrans_refund r
ON r.orig_saletran_id = s.saletran_id
AND r.orig_saletran_dt = s.saletran_dt
AND r.orig_loc_id = s.loc_id
AND r.saletran_dt between s.saletran_dt and s.saletran_dt + 30
JOIN saletran rt
ON rt.saletran_id = r.saletran_id
AND rt.saletran_dt = r.saletran_dt
AND rt.loc_id = r.loc_id
JOIN assoc ra --Return Associate
ON ra.assoc_prty_id = rt.sls_assoc_prty_id
WHERE
(SELECT count(*)
FROM saletran_refund
GROUP BY ORIG_SLTRN_ID
) > 1
AND s.saletran_dt between '2015-01-01' and current_date - 1
Based on what you've got so far, I think you want to use this instead:
where r.ORIG_SLTRN_ID in
(select
ORIG_SLTRN_ID
from
saletran_refund
group by ORIG_SLTRN_ID
having count (*) > 1)
That will give you the ORIG_SLTRN_IDs that have more than one row.
you don't give enough for a full answer but this is a start
group by s.saletran
, s.saletran_dt SALE_DATE
, r.saletran_id RET_TRAN
, r.saletran_dt RET_DATE
, ra.user_id RET_ASSOC
having count(distinct(ORIG_SLTRN_ID)) > 0
this does return more the an one row
run it
SELECT count(*)
FROM saletran_refund
GROUP BY ORIG_SLTRN_ID
I am trying to distinguish the physical servers uptime from virtual ones by looking at OS. I am able to pull out the result of the VMWare OS, however, I'd like to group the physical servers as one row.
Here is the code I have so far:
SELECT TOP (100) PERCENT Avg(dbo.tblserveruptime.uptime) AS Uptime,
Count(*) AS Total
FROM dbo.server
INNER JOIN dbo.tblserveruptime
ON dbo.server.name = dbo.tblserveruptime.name
WHERE ( dbo.server.status = N'production' )
AND ( dbo.server.server_env = N'prod' )
AND ( dbo.server.os_type <> N'vmware' )
GROUP BY dbo.tblserveruptime.month,
dbo.tblserveruptime.year
HAVING ( dbo.tblserveruptime.month = 4 )
AND ( dbo.tblserveruptime.year = 2013 )
UNION
SELECT TOP (100) PERCENT Avg(dbo.tblserveruptime.uptime) AS Uptime,
Count(*) AS Total
FROM dbo.server
INNER JOIN dbo.tblserveruptime
ON dbo.server.name = dbo.tblserveruptime.name
WHERE ( dbo.server.status = N'production' )
AND ( dbo.server.server_env = N'prod' )
AND ( dbo.server.os_type = N'vmware' )
GROUP BY dbo.tblserveruptime.month,
dbo.tblserveruptime.year
HAVING ( dbo.tblserveruptime.month = 4 )
AND ( dbo.tblserveruptime.year = 2013 )
Whatever field it is that gives you the physical server name, add that field to your GROUP BY statement and put it first. It will then group by Server, Month and Year (as you currently have it). My guess is you might want to swap Year and Month as it makes more sense to do it that way than by Month and then Year.
Okay, thanks to all who tried to help. I tried putting images of my before and after results, but I couldn't since I don't have 10 reputation points. Anyways, I think I got it. Here it is:
SELECT AVG(dbo.tblServerUptime.Uptime) AS Uptime,
CASE WHEN OS_TYPE = 'VMWare' THEN 'Virtual' ELSE 'Physical' END AS [Physical vs VM]
FROM dbo.Server INNER JOIN
dbo.tblServerUptime ON dbo.Server.NAME = dbo.tblServerUptime.NAME
GROUP BY CASE WHEN OS_TYPE = 'VMWare' THEN 'Virtual' ELSE 'Physical' END
I have a mysql database that stores some timestamps. Let's assume that all there is in the table is the ID and the timestamp. The timestamps might be duplicated.
I want to find the average time difference between consecutive rows that are not duplicates (timewise). Is there a way to do it in SQL?
If your table is t, and your timestamp column is ts, and you want the answer in seconds:
SELECT TIMESTAMPDIFF(SECOND, MIN(ts), MAX(ts) )
/
(COUNT(DISTINCT(ts)) -1)
FROM t
This will be miles quicker for large tables as it has no n-squared JOIN
This uses a cute mathematical trick which helps with this problem. Ignore the problem of duplicates for the moment. The average time difference between consecutive rows is the difference between the first timestamp and the last timestamp, divided by the number of rows -1.
Proof: The average distance between consecutive rows is the sum of the distance between consective rows, divided by the number of consecutive rows. But the sum of the difference between consecutive rows is just the distance between the first row and last row (assuming they are sorted by timestamp). And the number of consecutive rows is the total number of rows -1.
Then we just condition the timestamps to be distinct.
Are the ID's contiguous ?
You could do something like,
SELECT
a.ID
, b.ID
, a.Timestamp
, b.Timestamp
, b.timestamp - a.timestamp as Difference
FROM
MyTable a
JOIN MyTable b
ON a.ID = b.ID + 1 AND a.Timestamp <> b.Timestamp
That'll give you a list of time differences on each consecutive row pair...
Then you could wrap that up in an AVG grouping...
Here's one way:
select avg(timestampdiff(MINUTE,prev.datecol,cur.datecol))
from table cur
inner join table prev
on cur.id = prev.id + 1
and cur.datecol <> prev.datecol
The timestampdiff function allows you to choose between days, months, seconds, and so on.
If the id's are not consecutive, you can select the previous row by adding a rule that there are no other rows in between:
select avg(timestampdiff(MINUTE,prev.datecol,cur.datecol))
from table cur
inner join table prev
on prev.datecol < cur.datecol
and not exists (
select *
from table inbetween
where prev.datecol < inbetween.datecol
and inbetween.datecol < cur.datecol)
)
OLD POST but ....
Easies way is to use the Lag function and TIMESTAMPDIFF
SELECT
id,
TIMESTAMPDIFF('MINUTES', PREVIOUS_TIMESTAMP, TIMESTAMP) AS TIME_DIFF_IN_MINUTES
FROM (
SELECT
id,
TIMESTAMP,
LAG(TIMESTAMP, 1) OVER (ORDER BY TIMESTAMP) AS PREVIOUS_TIMESTAMP
FROM TABLE_NAME
)
Adapted for SQL Server from this discussion.
Essential columns used are:
cmis_load_date: A date/time stamp associated with each record.
extract_file: The full path to a file from which the record was loaded.
Comments:
There can be many records in each file. Records have to be grouped by the files loaded on the extract_file column. Intervals of days may pass between one file and the next being loaded. There is no reliable sequential value in any column, so the grouped rows are sorted by the minimum load date in each file group, and the ROW_NUMBER() function then serves as an ad hoc sequential value.
SELECT
AVG(DATEDIFF(day, t2.MinCMISLoadDate, t1.MinCMISLoadDate)) as ElapsedAvg
FROM
(
SELECT
ROW_NUMBER() OVER (ORDER BY MIN(cmis_load_date)) as RowNumber,
MIN(cmis_load_date) as MinCMISLoadDate,
CASE WHEN NOT CHARINDEX('\', extract_file) > 0 THEN '' ELSE RIGHT(extract_file, CHARINDEX('\', REVERSE(extract_file)) - 1) END as ExtractFile
FROM
TrafTabRecordsHistory
WHERE
court_id = 17
and
cmis_load_date >= '2019-09-01'
GROUP BY
CASE WHEN NOT CHARINDEX('\', extract_file) > 0 THEN '' ELSE RIGHT(extract_file, CHARINDEX('\', REVERSE(extract_file)) - 1) END
) t1
LEFT JOIN
(
SELECT
ROW_NUMBER() OVER (ORDER BY MIN(cmis_load_date)) as RowNumber,
MIN(cmis_load_date) as MinCMISLoadDate,
CASE WHEN NOT CHARINDEX('\', extract_file) > 0 THEN '' ELSE RIGHT(extract_file, CHARINDEX('\', REVERSE(extract_file)) - 1) END as ExtractFile
FROM
TrafTabRecordsHistory
WHERE
court_id = 17
and
cmis_load_date >= '2019-09-01'
GROUP BY
CASE WHEN NOT CHARINDEX('\', extract_file) > 0 THEN '' ELSE RIGHT(extract_file, CHARINDEX('\', REVERSE(extract_file)) - 1) END
) t2 on t2.RowNumber + 1 = t1.RowNumber