I have a table with million records. The following is an example of one group of data:
select id,
id_depend,
Item,
values as 'Current Values'
from mytable
where id in (685690, 691282, 691297)
order by 1
The first id (685690) correspond to a first movement, the second one (691282) cancel the first movement and the third is the correction of the first movement. The id_depend field relates the movements with the original.
I need to show the same data adding a new column with the values for the last movement related. I mean, sometimes the first movement (other ids) is rigth and ther is no corrections after this movement (e.g.: id 691371).
This can help if I understand it correctly:
SELECT m.id,m.id_depend,m.Item,m.[values] [Current Values]
,CASE WHEN m.id_depend = 0 AND NOT EXISTS(SELECT 1 FROM mytable cor WHERE cor.id_depend = m.id)
THEN m.[values]
ELSE COALESCE((SELECT SUM(mt.[values]) FROM mytable mt WHERE mt.Item = m.Item AND mt.id < m.id)+m.[values],0)
END [Values Required]
FROM mytable m
There is also my query to play with:
CREATE TABLE #mytable (id BIGINT, id_depend BIGINT, Item VARCHAR(50), [values] DECIMAL(23,10))
INSERT INTO #mytable (id,id_depend,Item,[values])VALUES(685690,0,'1',216),(685690,0,'2',108)
,(691282,685690,'1',-216),(691282,685690,'2',-108)
,(691297,685690,'1',324),(691297,685690,'2',162)
,(691371,0,'1',100),(691371,0,'2',200),(691371,0,'3',300)
SELECT m.id,m.id_depend,m.Item,m.[values] [Current Values]
FROM #mytable m
SELECT m.id,m.id_depend,m.Item,m.[values] [Current Values]
,CASE WHEN m.id_depend = 0 AND NOT EXISTS(SELECT 1 FROM #mytable cor WHERE cor.id_depend = m.id)
THEN m.[values]
ELSE COALESCE((SELECT SUM(mt.[values]) FROM #mytable mt WHERE mt.Item = m.Item AND mt.id < m.id)+m.[values],0)
END [Values Required]
FROM #mytable m
DROP TABLE #mytable
Please let me know if you have any questions.
Related
I am building a data warehouse currently that processes data (for the sake of this question, let's just say one table) from a table that is updated every 15 minutes. My process stores a snapshot of the table and then compares the refreshed version with the snapshot and then stores the difference - or delta - in a separate staging table that will then be processed at the end of the day. At the end of the day I want a row describing the name of the column that has changed with a timestamp, to be then used when create snapshots at any point in time. It is worth noting that by the end of the day, there can be multiple rows for each unique identifier created i.e. a row for every change someone might have actioned during the day. So, I am stuck on the last part. I found this clever link Return column Names of Changed values with XML but the problem is this is very inefficient when processing thousands of rows. I would be grateful to anyone who has
any ideas on a more appropriate solution (excluding change Data Capture)?
Thank you.
if OBJECT_ID('tempdb..#TempHistory') is Not Null
drop table #TempHistory
-- I just made up a few column here, there are LOADS of them in the real query but for Governance of course...
SELECT
distinct
a.ClaimId,
a.FirstName,
a.Surname,
a.Incident,
a.Total,
a.extractDate -- this field is create by ETL process
into #temphistory
FROM [Data Mart Test].[Staging].[Claim] a JOIN [Data Mart Test].[dbo].[Claim] b
ON a.ClaimId = b.ClaimId
WHERE
ISNULL(a.ClaimId,0) <> ISNULL(b.ClaimId,0) OR
ISNULL(a.FirstName,'') <> ISNULL(b.FirstName,'') OR
ISNULL(a.Surname,'') <> ISNULL(b.Surname,'') OR
ISNULL(a.Incident,'') <> ISNULL(b.Incident,'') OR
ISNULL(a.Total,0.0) <> ISNULL(b.Total,0.0) OR
if OBJECT_ID('tempdb..#TempHistoryA') is Not Null
drop table #TempHistoryA
select
*
, 1 as Version
into #TempHistoryA
FROM [Data Mart Test].[dbo].[Claim] where ClaimID in (select distinct claimid FROM #TempHistory)
if OBJECT_ID('tempdb..#TempHistoryB') is Not Null
drop table #TempHistoryB
SELECT *
,(RANK() OVER(PARTITION BY [ClaimId] ORDER BY [ExtractDate])) + 1 as Version
into #TempHistoryB
FROM [Data Mart Test].[Staging].[Claim]where ClaimID in (select distinct claimid FROM #TempHistory)
if OBJECT_ID('tempdb..#TempChanges') is Not Null
drop table #TempChanges
DECLARE #x xml
SET #x = (
SELECT
t2.ClaimID AS [#key]
, t2.Version AS [#version]
, ( SELECT t1.* FOR XML PATH('t1'), TYPE ) AS [*]
, ( SELECT t2.* FOR XML PATH('t2'), TYPE ) AS [*]
FROM #TempHistoryA AS t1
INNER JOIN #TempHistoryB AS t2
ON t1.ClaimID = t2.ClaimID
AND t1.Version = t2.Version - 1
FOR XML PATH('row'), ROOT('root')
);
WITH Nodes AS (
SELECT
C.value('../../#key', 'int') AS [Key]
, C.value('../../#version', 'int') AS Version_ID
, C.value('local-name(..)', 'varchar(255)')AS Version_Alias
, C.value('local-name(.)', 'varchar(max)') AS Field
, C.value('.', 'varchar(max)') AS Val
FROM #x.nodes('/root/row/*/*') AS T(C)
)
SELECT
[Key] as ClaimID,
x.ExtractDate,
Field
, Max(CASE Version_Alias WHEN 't1' THEN Val END) AS [Initial Value]
, Max(CASE Version_Alias WHEN 't2' THEN Val END) AS [New Value]
Into #TempChanges
FROM [Nodes]v
inner join [#TempHistoryB]x on x.ClaimId = v.[Key]
and x.Version = v.Version_ID
where Field not in ( 'ExtractDate','Version')
GROUP BY
[Key],
x.ExtractDate,
Field
HAVING Max(CASE Version_Alias WHEN 't1' THEN Val END) <> Max(CASE Version_Alias WHEN 't2' THEN Val END)
--Find records in [Data Mart Test].dbo.Claim that are not in [Data Mart Test].Staging.Claim
SELECT
*
FROM [Data Mart Test].dbo.Claim
WHERE ClaimId NOT IN (SELECT b.ClaimId FROM [Data Mart Test].Staging.Claim b)
delete from [Data Mart Test].[dbo].[Claim] where ClaimID in (select distinct claimid FROM #TempHistory)
delete from [#TempHistoryB]
where not exists
(
select
*
from
(select
claimid,
max(Version) as LastVersion
FROM #TempHistoryB b
group by ClaimId
) b
where b.ClaimId = claimid
and b.LastVersion = Version
)
insert into [Data Mart Test].[dbo].[Claim]
select
[ClaimId]
--a whole lot of other columns
from #TempHistoryB
Looking to try to return the dataset below in SQL Server. I have the 2 columns TrailerPosition & Divider and looking to also return Zone. Zone would be calculated as starting with Zone 1 and then would change to zone 2 on the record after divider = 1. And then to 3 after the next record where Divider = 1. The screenshot below looks like the column I'm trying to return.
any ideas how this can be done in SQL Server?
Test data for the below:
declare #t table (TrailerPosition nvarchar(5),Divider bit);
insert into #t values ('01L',0),('01R',0),('02L',0),('02R',1),('03L',1),('03R',0),('04L',0),('04R',0),('05L',1),('05R',1),('06L',0),('06R',0),('07L',0),('07R',0),('08L',0),('08R',0),('09L',0),('09R',0),('10L',0),('10R',0),('11L',0),('11R',0),('12L',0),('12R',0),('13L',0),('13R',0),('14L',0),('14R',0),('15L',0),('15R',0);
If 2012+, the window functions would be a nice fit here
Select TrailerPosition
,Divider
,Zone = 1+sum(Flag) over (Order By TrailerPosition)
From (
Select *
,Flag = case when Lag(Divider,1) over (Order By TrailerPosition) =1 and Divider=0 then 1 else 0 end
From YourTable
) A
Returns
So, the Zone = 1 + The number of previous rows with the divider value of 0 and the previous row having a divider value of 1.
UPDATED
SELECT TrailerPosition, Divider,
(SELECT COUNT(*)
FROM #MyTable T1
WHERE (T1.TrailerPosition <= t0.TrailerPosition)
AND (T1.Divider = 0)
AND (SELECT Divider
FROM #MyTable t2
WHERE T2.TrailerPosition =
(SELECT MAX(T3.TrailerPosition)
FROM #MyTable T3
WHERE T3.TrailerPosition < T1.TrailerPosition)) = 1) + 1
AS Zone
FROM #MyTable t0
I have Sql Table data and i need to filter only the Consecutive dates blocks as i highlighted on image below..
.
and i need to add custom rates for each row on that selected blocks(this rate can display with separate column on out put).If there is more than 6 rows captured then $200 apply for each column of that block.if it is less than 6 ,it will be $125.The out put should be like this
And it should group by EmpID.
i need to get the out put using MSSQL. Can any one help me
this is what i have done through the sql view
ALTER view [dbo].[vw_Test2] AS
SELECT
tbl2.ID as Tbl2ID,
tbl1.[EmpID],
tbl1.[ExpInDateTime] as Tbl1ExpDate,
tbl2.[ExpInDateTime] as Tbl2ExpDate,
case when(CONVERT(date,tbl1.[ActInDateTime]) = CONVERT(date, DATEADD(DAY,1,tbl2.[ExpInDateTime]))) then
1
else 0
end as Token
from [dbo].[vw_Test] tbl1 join [dbo].[vw_Test] tbl2
on tbl1.ID=(tbl2.ID+1)
GO
only thing is i have to do this using SQL views
Please try this as a view:
ALTER VIEW [dbo].[vw_Test2] AS
WITH PreResult AS (
SELECT p.Id,p.EmpID,p.[DateTime],CASE WHEN LEAD(p.diff)OVER(ORDER BY p.Id) > 1 OR LEAD(p.EmpID)OVER(ORDER BY p.Id)<>p.EmpID THEN 1 ELSE 0 END StartNewGroup
FROM (
SELECT t.Id,t.EmpID,t.[DateTime], COALESCE(DATEDIFF(day,LAG(t.[DateTime])OVER(PARTITION BY t.EmpID ORDER BY t.Id),t.[DateTime]),1) [diff]
FROM [dbo].[vw_Test] t
) p
)
SELECT r.Id,r.EmpID,r.[DateTime]
,CASE WHEN COUNT(*)OVER(PARTITION BY r.NewGroup ORDER BY r.NewGroup) >= 6 THEN 250 ELSE 125 END [Rate]
FROM (
SELECT b.Id,b.EmpID,b.[DateTime],1+COALESCE((SELECT SUM(a.StartNewGroup) FROM PreResult a WHERE a.Id<b.Id),0) NewGroup
FROM PreResult b
) r
GO
There is also query to play with:
CREATE TABLE #Test (Id BIGINT IDENTITY(1,1),EmpID BIGINT, [DateTime] DATETIME)
INSERT INTO #Test (EmpID,[DateTime]) VALUES (5,'20150106'),(5,'20150107'),(5,'20150109'),
(5,'20150110'),(5,'20150126'),(5,'20150127'),
(5,'20150128'),(5,'20150129'),(5,'20150130'),
(5,'20150131'),(10,'20121203'),(10,'20121204'),
(10,'20121205'),(10,'20121206'),
(10,'20121207'),(10,'20121208'),(10,'20121209')
;WITH PreResult AS (
SELECT p.Id,p.EmpID,p.[DateTime],CASE WHEN LEAD(p.diff)OVER(ORDER BY p.Id) > 1 OR LEAD(p.EmpID)OVER(ORDER BY p.Id)<>p.EmpID THEN 1 ELSE 0 END StartNewGroup
FROM (
SELECT t.Id,t.EmpID,t.[DateTime], COALESCE(DATEDIFF(day,LAG(t.[DateTime])OVER(PARTITION BY t.EmpID ORDER BY t.Id),t.[DateTime]),1) [diff]
FROM #Test t
) p
)
SELECT r.Id,r.EmpID,r.[DateTime]
,CASE WHEN COUNT(*)OVER(PARTITION BY r.NewGroup ORDER BY r.NewGroup) >= 6 THEN 250 ELSE 125 END [Rate]
FROM (
SELECT b.Id,b.EmpID,b.[DateTime],1+COALESCE((SELECT SUM(a.StartNewGroup) FROM PreResult a WHERE a.Id<b.Id),0) NewGroup
FROM PreResult b
) r
DROP TABLE #Test
Please let me know if you have any questions.
i have following table "vehicle_data" :
ID ALERT_TYPE VALUE
58 2 1
58 1 1
104 1 1
104 2 1
Here alert_type = 2 is for GPS value and alert_type=1 is for engine_value .
so if alert_type=2 and its value is =1 then it means its value is correct.
when alert_type=2 and its value is =0 then it means its value is wrong.
same for alert_type=1
so now here i want the following output:
ID gps engine_value
58 1 1
104 1 1
how can i perform this query??
You can do it like this.
SELECT ID
,CASE WHEN [ALERT_TYPE]=2 and [value ]=1 THEN 1 ELSE 0 END as gps
,CASE WHEN [ALERT_TYPE]=1 and [value ]=1 THEN 1 ELSE 0 END as engine
FROM vehicle_data
SELECT ID, alert_type=2 AS gps, alert_type=1 AS [engine] FROM vehicle_data WHERE value=1;
EDITED to account for your explanation of VALUE.
Schema
CREATE TABLE table3 (id int, ALERT_TYPE int)
INSERT table3 VALUES (58, 1), (58, 2), (104, 1), (104, 2)
Query
SELECT *
FROM (
SELECT ID
,ROW_NUMBER() OVER (
PARTITION BY id ORDER BY id
) AS row_num
,gps = CASE
WHEN ALERT_TYPE = 2
THEN 1
ELSE 0
END
,engine = CASE
WHEN ALERT_TYPE = 1
THEN 1
ELSE 0
END
FROM table3
) a
WHERE a.row_num = 1
Output
ID gps engine
58 1 0
104 0 1
One possible way using subqueries :
select
Ids.ID
, gps.VALUE 'gps'
, engine_value.VALUE 'engine_value'
from (select distinct ID from vehicle_data) Ids
left join
(select ID, VALUE from vehicle_data where ALERT_TYPE = 2) gps
on gps.ID = Ids.ID
left join
(select ID, VALUE from vehicle_data where ALERT_TYPE = 1) engine_value
on engine_value.ID = Ids.ID
[SQL Fiddle demo]
I hope this should work for you,
Select ID,sum(gps) as gps ,sum(engine) as engine from
(SELECT ID
,CASE WHEN [ALERT_TYPE]=2 and [value ]=1 THEN 1 ELSE 0 END as gps
,CASE WHEN [ALERT_TYPE]=1 and [value ]=1 THEN 1 ELSE 0 END as engine
FROM vehicle_data
)a
group by id
select x.id,x.alert_type as GPS,x.value as engine_value from (
select ID,alert_type,value,ROW_NUMBER() over (partition by id order by alert_type ) as Rnk from mytable
)x
where Rnk=1
Please check this query in SQL :
create table mytable (id int, alert_type int, value int);
insert into mytable (id, alert_type, value)
values (58, 2, 1),
(58, 1, 1),
(104, 1, 1),
(104, 2, 1);
SELECT distinct ID
,(select count (id) from mytable mt where mt.id=mytable.id and mt.[ALERT_TYPE]=2 and mt.[value ]=1) as gps
,(select count (id) from mytable mt where mt.id=mytable.id and mt.[ALERT_TYPE]=1 and mt.[value ]=1) as engine
FROM mytable
BASED ON YOUR QUESTION I BELIEVE YOU WANT THE DATA IN COLUMN AND TO SUIT YOUR REQUIREMENT I HAVE MADE A SQL FIDDLE WORKING - CODE IS ALSO MENTIONED BELOW -
HERE YOU GO WITH THE WORKING FIDDLE -
WORKING DEMO
SQL CODE FOR REFERNECE -
CREATE TABLE ALERTS (ID INT, ALERT_TYPE INT, VALUE INT)
INSERT INTO ALERTS VALUES (58,2,1)
INSERT INTO ALERTS VALUES (58,1,0)
INSERT INTO ALERTS VALUES (104,1,1)
INSERT INTO ALERTS VALUES (104,2,0)
CREATE TABLE ALERTSVALUE (ID INT, gps INT,engine INT)
INSERT INTO ALERTSVALUE VALUES (58,1,0)
INSERT INTO ALERTSVALUE VALUES (104,0,1)
SELECT A.ID,
CASE A.ALERT_TYPE WHEN 2 THEN 1 ELSE 0 END AS GPS,
CASE A.ALERT_TYPE WHEN 1 THEN 1 ELSE 0 END AS ENGINE_VALUE,
A.VALUE FROM ALERTS A WHERE A.VALUE = 1
EDIT BASED ON COMMENT - TO MERGE THE ROWS FOR BOTH GPS AND ENGINE_VALUE:
SELECT X.ID,X.ALERT_TYPE as GPS,X.VALUE as ENGINE_VALUE
FROM (
SELECT ID,ALERT_TYPE ,VALUE ,ROW_NUMBER() OVER (PARTITION BY ID ORDER BY alert_type ) AS [Rank] FROM ALERTS
)X
WHERE [Rank]=1
SQL FIDDLE DEMO
I have three address line columns, aline1, aline2, aline3 for a street
address. As staged from inconsistent data, any or all of them can be
blank. I want to move the first non-blank to addrline1, 2nd non-blank
to addrline2, and clear line 3 if there aren't three non blank lines,
else leave it. ("First" means aline1 is first unless it's blank,
aline2 is first if aline1 is blank, aline3 is first if aline1 and 2
are both blank)
The rows in this staging table do not have a key and there could be
duplicate rows. I could add a key.
Not counting a big case statement that enumerates the possible
combination of blank and non blank and moves the fields around, how
can I update the table? (This same problem comes up with a lot more
than 3 lines, so that's why I don't want to use a case statement)
I'm using Microsoft SQL Server 2008
Another alternative. It uses the undocumented %%physloc%% function to work without a key. You would be much better off adding a key to the table.
CREATE TABLE #t
(
aline1 VARCHAR(100),
aline2 VARCHAR(100),
aline3 VARCHAR(100)
)
INSERT INTO #t VALUES(NULL, NULL, 'a1')
INSERT INTO #t VALUES('a2', NULL, 'b2')
;WITH cte
AS (SELECT *,
MAX(CASE WHEN RN=1 THEN value END) OVER (PARTITION BY %%physloc%%) AS new_aline1,
MAX(CASE WHEN RN=2 THEN value END) OVER (PARTITION BY %%physloc%%) AS new_aline2,
MAX(CASE WHEN RN=3 THEN value END) OVER (PARTITION BY %%physloc%%) AS new_aline3
FROM #t
OUTER APPLY (SELECT ROW_NUMBER() OVER (ORDER BY CASE WHEN value IS NULL THEN 1 ELSE 0 END, idx) AS
RN, idx, value
FROM (VALUES(1,aline1),
(2,aline2),
(3,aline3)) t (idx, value)) d)
UPDATE cte
SET aline1 = new_aline1,
aline2 = new_aline2,
aline3 = new_aline3
SELECT *
FROM #t
DROP TABLE #t
Here's an alternative
Sample table for discussion, don't worry about the nonsensical data, they just need to be null or not
create table taddress (id int,a varchar(10),b varchar(10),c varchar(10));
insert taddress
select 1,1,2,3 union all
select 2,1, null, 3 union all
select 3,null, 1, 2 union all
select 4,null,null,2 union all
select 5,1, null, null union all
select 6,null, 4, null
The query, which really just normalizes the data
;with tmp as (
select *, rn=ROW_NUMBER() over (partition by t.id order by sort)
from taddress t
outer apply
(
select 1, t.a where t.a is not null union all
select 2, t.b where t.b is not null union all
select 3, t.c where t.c is not null
--- EXPAND HERE
) u(sort, line)
)
select t0.id, t1.line, t2.line, t3.line
from taddress t0
left join tmp t1 on t1.id = t0.id and t1.rn=1
left join tmp t2 on t2.id = t0.id and t2.rn=2
left join tmp t3 on t3.id = t0.id and t3.rn=3
--- AND HERE
order by t0.id
EDIT - for the update back into table
;with tmp as (
select *, rn=ROW_NUMBER() over (partition by t.id order by sort)
from taddress t
outer apply
(
select 1, t.a where t.a is not null union all
select 2, t.b where t.b is not null union all
select 3, t.c where t.c is not null
--- EXPAND HERE
) u(sort, line)
)
UPDATE taddress
set a = t1.line,
b = t2.line,
c = t3.line
from taddress t0
left join tmp t1 on t1.id = t0.id and t1.rn=1
left join tmp t2 on t2.id = t0.id and t2.rn=2
left join tmp t3 on t3.id = t0.id and t3.rn=3
Update - Changed statement to an Update statement. Removed Case statement solution
With this solution, you will need a unique key in the staging table.
With Inputs As
(
Select PK, 1 As LineNum, aline1 As Value
From StagingTable
Where aline1 Is Not Null
Union All
Select PK, 2, aline2
From StagingTable
Where aline2 Is Not Null
Union All
Select PK, 3, aline3
From StagingTable
Where aline3 Is Not Null
)
, ResequencedInputs As
(
Select PK, Value
, Row_Number() Over( Order By LineNum ) As LineNum
From Inputs
)
, NewValues As
(
Select S.PK
, Min( Case When R.LineNum = 1 Then R.addrline1 End ) As addrline1
, Min( Case When R.LineNum = 2 Then R.addrline1 End ) As addrline2
, Min( Case When R.LineNum = 3 Then R.addrline1 End ) As addrline3
From StagingTable As S
Left Join ResequencedInputs As R
On R.PK = S.PK
Group By S.PK
)
Update OtherTable
Set addrline1 = T2.addrline1
, addrline2 = T2.addrline2
, addrline3 = T2.addrline3
From OtherTable As T
Left Join NewValues As T2
On T2.PK = T.PK
R. A. Cyberkiwi, Thomas, and Martin, thanks very much - these were very generous responses by each of you. All of these answers were the type of spoonfeeding I was looking for. I'd say they all rely on a key-like device and work by dividing addresses into lines, some of which are empty and some of which aren't, excluding the empties. In the case of lines of addresses, in my opinion this is semantically a gimmick to make the problem fit what SQL does well, and it's not a natural way to conceptualize the problem. Address lines are not "really" separate rows in a table that just got denormalized for a report. But that's debatable and whether you agree or not, I (a rank beginner) think each of your alternatives are idiomatic solutions worth elaborating on and studying.
I also get lots of similar cases where there really is normalization to be done - e.g., collatDesc1, collatCode1, collatLastAppraisal1, ... collatLastAppraisal5, with more complex criteria about what in excludeand how to order than with addresses, and I think techniques from your answers will be helpful.
%%phsloc%% is fun - since I'm able to create a key in this case I won't use it (as Martin advises). There was other stuff in Martin's stuff I wasn't familiar with too, and I'm still tossing them all around.
FWIW, here's the trigger I tried out, I don't know that I'll actually use it for the problem at hand. I think this qualifies a "bubble sort", with the swapping expressed in a peculiar way.
create trigger fixit on lines
instead of insert as
declare #maybeblank1 as varchar(max)
declare #maybeblank2 as varchar(max)
declare #maybeblank3 as varchar(max)
set #maybeBlank1 = (select line1 from inserted)
set #maybeBlank2 = (select line2 from inserted)
set #maybeBlank3 = (select line3 from inserted)
declare #counter int
set #counter = 0
while #counter < 3
begin
set #counter = #counter + 1
if #maybeBlank2 = ''
begin
set #maybeBlank2 =#maybeblank3
set #maybeBlank3 = ''
end
if #maybeBlank1 = ''
begin
set #maybeBlank1 = #maybeBlank2
set #maybeBlank2 = ''
end
end
select * into #kludge from inserted
update #kludge
set line1 = #maybeBlank1,
line2 = #maybeBlank2,
line3 = #maybeBlank3
insert into lines
select * from #kludge
You could make an insert and update trigger that check if the fields are empty and then move them.