Change an iterative query to a relational set-based query - sql

SQL Fiddle
I'm trying without success to change an iterative/cursor query (that is working fine) to a relational set query to achieve a better performance.
What I have:
table1
| ID | NAME |
|----|------|
| 1 | A |
| 2 | B |
| 3 | C |
Using a function, I want to insert my data into another table. The following function is a simplified example:
Function
CREATE FUNCTION fn_myExampleFunction
(
#input nvarchar(50)
)
RETURNS #ret_table TABLE
(
output nvarchar(50)
)
AS
BEGIN
IF #input = 'A'
INSERT INTO #ret_table VALUES ('Alice')
ELSE IF #input = 'B'
INSERT INTO #ret_table VALUES ('Bob')
ELSE
INSERT INTO #ret_table VALUES ('Foo'), ('Bar')
RETURN
END;
My expected result is to insert data in table2 like the following:
table2
| ID | NAME |
|----|-------|
| 1 | Alice |
| 2 | Bob |
| 3 | Foo |
| 3 | Bar |
To achieve this, I've tried some CTEs (Common Table Expression) and relational queries, but none worked as desired. The only working solution that I've got so far was an iterative and not performatic solution.
My current working solution:
BEGIN
DECLARE
#ID int,
#i int = 0,
#max int = (SELECT COUNT(name) FROM table1)
WHILE ( #i < #max ) -- In this example, it will iterate 3 times
BEGIN
SET #i += 1
-- Select table1.ID where row_number() = #i
SET #ID =
(SELECT
id
FROM
(SELECT
id,
ROW_NUMBER() OVER (ORDER BY id) as rn
FROM
table1) rows
WHERE
rows.rn = #i
)
-- Insert into table2 one or more rows related with table1.ID
INSERT INTO table2
(id, name)
SELECT
#ID,
fn_result.output
FROM
fn_myExampleFunction (
(SELECT name FROM table1 WHERE id = #ID)
) fn_result
END
END
The objective is to achieve the same without iterating through the IDs.

if the question is about how to apply a function in a set oriented way, then cross apply (or outer apply) is your friend:
insert into table2 (
id, name
) select
t1.id,
t2.output
from
table1 t1
cross apply
fn_myExampleFunction(t1.name) t2
Example SQLFiddle
If the non-simplified version of your function is amenable to rewriting, the other solutions will likely be faster.

A query like this will do what you want:
insert into table2(id, name)
select id, (case when name = 'A' then 'Alice'
when name = 'B' then 'Bob'
when name = 'C' then 'Foo'
end)
from table1
union all
select id, 'Bar'
from table1
where name = 'C';

Why wouldn't you store this data as a table? It's relational. Coding it in a function or stored procedure seems less than ideal.
In any case, I hope the following gives you ideas about how to improve your code. I realize that you said your function is more complicated than your example, but you can still use this idea even inside of the function as necessary.
INSERT dbo.table2 (ID, Name)
SELECT
T1.ID,
N.FullName
FROM
dbo.table1 T1
INNER JOIN (VALUES -- A "derived table" made up of only constants
('A', 'Alice'),
('B', 'Bob'),
('C', 'Foo'),
('C', 'Bar')
) N (ShortName, FullName)
ON T1.Name = N.ShortName
;
But of course, that could just be rendered INNER JOIN dbo.NameTranslation N if it were in a real table (and then updating it would be so much easier!).
If your function absolutely can't be rewritten to be relational (it must take a single name at a time) then you would use CROSS APPLY:
INSERT dbo.table2 (ID, Name)
SELECT
T1.ID,
N.OutputName
FROM
dbo.table1 T1
CROSS APPLY dbo.YourFunction(T1.Name) F
;
However, this will not perform very well for large rowsets. Rewriting the function to be the type that RETURNS TABLE is a step in the right direction (instead of RETURNS #variable TABLE (definition)).

Related

SQL Select Where Opposite Match Does Not Exist

Trying to compare between two columns and check if there are no records that exist with the reversal between those two columns. Other Words looking for instances where 1-> 3 exists but 3->1 does not exist. If 1->2 and 2->1 exists we will still consider 1 to be part of the results.
Table = Betweens
start_id | end_id
1 | 2
2 | 1
1 | 3
1 would be added since it is a start to an end with no opposite present of 3,1. Though it did not get added until the 3rd entry since 1 and 2 had an opposite.
So, eventually it will just return names where the reversal does not exist.
I then want to join another table where the number from the previous problem has its name installed on it.
Table = Names
id | name
1 | Mars
2 | Earth
3 | Jupiter
So results will just be the names of those that don't have an opposite.
You can use a not exists condition:
select t1.start_id, t1.end_id
from the_table t1
where not exists (select *
from the_table t2
where t2.end_id = t1.start_id
and t2.start_id = t1.end_id);
I'm not sure about your data volume, so with your ask, below query will supply desired result for you in Sql Server.
create table TableBetweens
(start_id INT,
end_id INT
)
INSERT INTO TableBetweens VALUES(1,2)
INSERT INTO TableBetweens VALUES(2,1)
INSERT INTO TableBetweens VALUES(1,3)
create table TableNames
(id INT,
NAME VARCHAR(50)
)
INSERT INTO TableNames VALUES(1,'Mars')
INSERT INTO TableNames VALUES(2,'Earth')
INSERT INTO TableNames VALUES(3,'Jupiter')
SELECT *
FROM TableNames c
WHERE c.id IN (
SELECT nameid1.nameid
FROM (SELECT a.start_id, a.end_id
FROM TableBetweens a
LEFT JOIN TableBetweens b
ON CONCAT(a.start_id,a.end_id) = CONCAT(b.end_id,b.start_id)
WHERE b.end_id IS NULL
AND b.start_id IS NULL) filterData
UNPIVOT
(
nameid
FOR id IN (filterData.start_id,filterData.end_id)
) AS nameid1
)

Oracle -- Update the exact column referenced in the ON clause

I think this requirement is rarely encountered so I couldn't search for similar questions.
I have a table that needs to update the ID. For example ID 123 in table1 is actually supposed to be 456. I have a separate reference table built that stores the mapping (e.g. old 123 maps to new id 456).
I used the below query but apparently it returned error 38104, columns referenced in the ON clause cannot be updated.
MERGE INTO table1
USING ref_table ON (table1.ID = ref_table.ID_Old)
WHEN MATCHED THEN UPDATE SET table.ID = ref_table.ID_New;
Is there other way to achieve my purpose?
Thanks and much appreciated for your answer!
Use the ROWID pseudocolumn:
SQL Fiddle
Oracle 11g R2 Schema Setup:
CREATE TABLE TABLE1( ID ) AS
SELECT 1 FROM DUAL UNION ALL
SELECT 2 FROM DUAL UNION ALL
SELECT 3 FROM DUAL;
CREATE TABLE REF_TABLE( ID_OLD, ID_NEW ) AS
SELECT 1, 4 FROM DUAL UNION ALL
SELECT 2, 5 FROM DUAL;
MERGE INTO TABLE1 dst
USING ( SELECT t.ROWID AS rid,
r.id_new
FROM TABLE1 t
INNER JOIN REF_TABLE r
ON ( t.id = r.id_old ) ) src
ON ( dst.ROWID = src.RID )
WHEN MATCHED THEN
UPDATE SET id = src.id_new;
Query 1:
SELECT * FROM table1
Results:
| ID |
|----|
| 4 |
| 5 |
| 3 |
You can't update a column used in the ON clause in a MERGE. But if you don't need to make other changes that MERGE allows like WHEN NOT MATCHED or deleting, etc. you can just use a UPDATE to achieve this.
You mentioned this is an ID that needs an update. Here's an example using a scalar subquery. As it is an ID, this presumes UNIQUE ID_OLD values in REF_TABLE. I wasn't sure if Every row needs an update or only a sub-set, so set the update here to only update rows that have a value in REF_TABLE.
CREATE TABLE TABLE1(
ID NUMBER
);
CREATE TABLE REF_TABLE(
ID_OLD NUMBER,
ID_NEW NUMBER
);
INSERT INTO TABLE1 VALUES (1);
INSERT INTO TABLE1 VALUES (2);
INSERT INTO TABLE1 VALUES (100);
INSERT INTO REF_TABLE VALUES (1,10);
INSERT INTO REF_TABLE VALUES (2,20);
Initial State:
SELECT * FROM TABLE1;
ID
1
2
100
Then make the UPDATE
UPDATE TABLE1
SET TABLE1.ID = (SELECT REF_TABLE.ID_NEW
FROM REF_TABLE
WHERE REF_TABLE.ID_OLD = ID)
WHERE TABLE1.ID IN (SELECT REF_TABLE.ID_OLD
FROM REF_TABLE);
2 rows updated.
And check the change:
SELECT * FROM TABLE1;
ID
10
20
100

How to update in order and query the updated fields when updating in SQL in a single statement

I need to calculate Dividend Factors in the DB and the basic calculation needed in a general way is row2 field2 = (row2's field1) * (row1's field2) where the field2 is the value I need to both update and query at the same time i.e. when I calculate it for one row, I need the calculated value of the previous row for this row.
Now I have a temp table with has all the values and now I need to calculate the final values, but when I tried this:
UPDATE
#temp
SET
field2 = IsNull(
(SELECT d2.field2 * d.field1 FROM #temp AS d2 WHERE d2.rowNr = d.rowNr - 1)
,d.field1
)
FROM
#temp as d
;
It always saw that the field2 was always NULL and went with the default action, with it should do only for the first row.
Now currently there are only two methods I know for doing this:
Loop through the #temp with a cursor
Use a while statement and loop through the table that way (I opted for this one, because I thought there is no point in using a cursor for a small table of 10-20 rows max)
But I still would like to get this into a single statement, but I have no idea how to do this. I am using MS SQL 2008 R2.
EDIT:
This is the actual data I am working with: (Note, that all field2 values are NULL prior to the calculation and the data type is money)
field1 field2(expected values)
------ ----------------------
1,033 1,033
1,0363 1,0705
1,0558 1,1302
1,0157 1,1479
1,0188 1,1695
1,026 1,1999
1,0286 1,2342
1,0323 1,2741
1,0319 1,3147
Okay if I'm understanding this, you want to find field2 which is based on previous rows of field2 that were just calculated so you need either some form of loop or recursion. Try this recursive solution out:
Setting Up Tables
IF OBJECT_ID('tempdb..#temp') IS NOT NULL
DROP TABLE #temp;
DECLARE #yourTable TABLE (ID INT,field1 INT, field2 INT);
INSERT INTO #yourTable(ID,field1,field2)
VALUES (1111,11,11),(2222,22,22),(3333,33,33);
SELECT ROW_NUMBER() OVER (ORDER BY ID) rowNr,
ID,
field1,
field2 INTO #temp
FROM #yourTable;
Calculating values
WITH cte_recursion
AS
(
SELECT TOP 1
rowNR,
ID,
field1,
field2,
field1 AS dividend_factor
FROM #temp A
ORDER BY rowNr
UNION ALL
SELECT B.rowNr,
B.ID,
B.field1,
B.field2,
B.field1 * A.dividend_factor
FROM cte_recursion A
INNER JOIN #temp B
ON A.rowNr = B.rowNr - 1
)
Actual Update
UPDATE #yourTable
SET field2 = B.dividend_factor
FROM #yourTable A
INNER JOIN cte_recursion B
ON A.ID = B.ID
OPTION (MAXRECURSION 0)
SELECT *
FROM #yourTable
Results:
ID field1 field2
----------- ----------- -----------
1111 11 11
2222 22 242
3333 33 7986
Personally I wouldn't use the update because you have to constantly make sure the data is update to date. I'd much rather use the CTE I used to calculate the values and put it in a view so that you know the values are ALWAYS up to date and you don't have to worry about running it. Either that or having a dividend_factor column in your actual table that will be NULL unless the value is updated. Just my two cents
UPDATE d1
SET d1.field2 = IsNull(d2.field2 * d1.field1, d1.field1)
FROM #temp AS d1
left outer join #temp AS d2
on d2.rowNr = d1.rowNr - 1
magic
select d1.field1, EXP(SUM(LOG(d2.field1)))
from #temp AS d1
join #temp AS d2
on d2.rowNr <= d1.rowNr
group by d1.field1
op claims wrong answer
test for youself
drop table #temp;
create table #temp (ID int, val money);
insert into #temp (ID, val) values
(1, 1.033)
, (2, 1.0363)
, (3, 1.0558)
, (4, 1.0157)
, (5, 1.0188)
, (6, 1.026)
, (7, 1.0286)
, (8, 1.0323)
, (9, 1.0319);
SELECT TOP 10 [t1].[ID], EXP(SUM(LOG([t2].[val])))
from #temp AS t1
join #temp AS t2
on t2.[ID] <= t1.[ID]
group by t1.[ID]
order by t1.[ID]

Ungrouping effect?

I have a dynamic set of data X of the form:
----------------------------------
x.id | x.allocated | x.unallocated
----------------------------------
foo | 2 | 0
bar | 1 | 2
----------------------------------
And I need to get to a result of Y (order is unimportant):
----------------------------------
y.id | y.state
----------------------------------
foo | allocated
foo | allocated
bar | allocated
bar | unallocated
bar | unallocated
----------------------------------
I have a UTF based solution, but I'm looking for hyper-efficiency so I'm idly wondering if there's a statement based, non-procedural way to get this kind of "ungroup by" effect?
It feels like an unpivot, but my brain can't get there right now.
If you have a numbers table in your database, you could use that to help get your results. In my database, I have a table named Numbers with a Num column.
Declare #Temp Table(id VarChar(10), Allocated Int, UnAllocated Int)
Insert Into #Temp Values('foo', 2, 0)
Insert Into #Temp Values('bar',1, 2)
Select T.id,'Allocated'
From #Temp T
Inner Join Numbers
On T.Allocated >= Numbers.Num
Union All
Select T.id,'Unallocated'
From #Temp T
Inner Join Numbers
On T.unAllocated >= Numbers.Num
Using Sql Server 2005, UNPIVOT, and CTE you can try something like
DECLARE #Table TABLE(
id VARCHAR(20),
allocated INT,
unallocated INT
)
INSERT INTO #Table SELECT 'foo', 2, 0
INSERT INTO #Table SELECT 'bar', 1, 2
;WITH vals AS (
SELECT *
FROM
(
SELECT id,
allocated,
unallocated
FROM #Table
) p
UNPIVOT (Cnt FOR Action IN (allocated, unallocated)) unpvt
WHERE Cnt > 0
)
, Recurs AS (
SELECT id,
Action,
Cnt - 1 Cnt
FROM vals
UNION ALL
SELECT id,
Action,
Cnt - 1 Cnt
FROM Recurs
WHERE Cnt > 0
)
SELECT id,
Action
FROM Recurs
ORDER BY id, action
This answer is just to ping back to G Mastros and doesn't need any upvotes. I thought he would appreciate a performance boost to his already superior query.
SELECT
T.id,
CASE X.Which WHEN 1 THEN 'Allocated' ELSE 'Unallocated' END
FROM
#Temp T
INNER JOIN Numbers N
On N.Num <= CASE X.Which WHEN 1 THEN T.Allocated ELSE T.Unallocated END
CROSS JOIN (SELECT 1 UNION ALL SELECT 2) X (Which)

Adding Row Numbers To a SELECT Query Result in SQL Server Without use Row_Number() function

i need Add Row Numbers To a SELECT Query without using Row_Number() function.
and without using user defined functions or stored procedures.
Select (obtain the row number) as [Row], field1, field2, fieldn from aTable
UPDATE
i am using SAP B1 DIAPI, to make a query , this system does not allow the use of rownumber() function in the select statement.
Bye.
I'm not sure if this will work for your particular situation or not, but can you execute this query with a stored procedure? If so, you can:
A) Create a temp table with all your normal result columns, plus a Row column as an auto-incremented identity.
B) Select-Insert your original query, sans the row column (SQL will fill this in automatically for you)
C) Select * on the temp table for your result set.
Not the most elegant solution, but will accomplish the row numbering you are wanting.
This query will give you the row_number,
SELECT
(SELECT COUNT(*) FROM #table t2 WHERE t2.field <= t1.field) AS row_number,
field,
otherField
FROM #table t1
but there are some restrictions when you want to use it. You have to have one column in your table (in the example it is field) which is unique and numeric and you can use it as a reference. For example:
DECLARE #table TABLE
(
field INT,
otherField VARCHAR(10)
)
INSERT INTO #table(field,otherField) VALUES (1,'a')
INSERT INTO #table(field,otherField) VALUES (4,'b')
INSERT INTO #table(field,otherField) VALUES (6,'c')
INSERT INTO #table(field,otherField) VALUES (7,'d')
SELECT * FROM #table
returns
field | otherField
------------------
1 | a
4 | b
6 | c
7 | d
and
SELECT
(SELECT COUNT(*) FROM #table t2 WHERE t2.field <= t1.field) AS row_number,
field,
otherField
FROM #table t1
returns
row_number | field | otherField
-------------------------------
1 | 1 | a
2 | 4 | b
3 | 6 | c
4 | 7 | d
This is the solution without functions and stored procedures, but as I said there are the restrictions. But anyway, maybe it is enough for you.
RRUZ, you might be able to hide the use of a function by wrapping your query in a View. It would be transparent to the caller. I don't see any other options, besides the ones already mentioned.