I have a function that each minute queries Google BigQuery, and I want to use BigQuery results to update rows in another relational database.
I was trying to do something like this:
BEGIN TRANSACTION
UPDATE MyTable SET MyField = BigQueryResult_Row1_MyField WHERE a_id = BigQueryResult_Row1_a_id
UPDATE MyTable SET MyField = BigQueryResult_Row2_MyField WHERE a_id = BigQueryResult_Row2_a_id
UPDATE MyTable SET MyField = BigQueryResult_Row3_MyField WHERE a_id = BigQueryResult_Row3_a_id
.
.
.
UPDATE MyTable SET MyField = BigQueryResult_RowN_MyField WHERE a_id = BigQueryResult_RowN_a_id
COMMIT TRANSACTION
That is an UPDATE statement for each bigquery row. I can't do a single UPDATE statement combined with SELECT statement because data come from BigQuery, not from another database table.
Trying to execute this transaction, I get timeout error, so I want to ask: is this a proper way to do hundred updates at a time? There could even be thousand updates at a time in some cases. How can I do that in a better way?
Renat's answer is fine. But a more colloquial way of writing uses values():
UPDATE t
SET MyField = v.MyField
FROM MyTable t JOIN
(VALUES ('BigQueryResult_Row1_MyField', 'BigQueryResult_Row1_a_id'),
('BigQueryResult_Row2_MyField', 'BigQueryResult_Row2_a_id'),
('BigQueryResult_Row3_MyField', 'BigQueryResult_Row3_a_id'),
('BigQueryResult_RowN_MyField', 'BigQueryResult_RowN_a_id')
) v(MyField, a_id)
ON v.a_id = t.a_id;
In general, using UNION when you intend UNION ALL is a bad idea -- because UNION incurs overhead for removing duplicates. In this case, a table constructor is a simpler solution anyway.
Does a query like this work without timeout (using temp table as suggested #JohnHC)?
BEGIN TRANSACTION
;WITH UpdateTempTable AS (
SELECT 'BigQueryResult_Row1_MyField' As MyField, 'BigQueryResult_Row1_a_id' AS a_id
UNION SELECT 'BigQueryResult_Row2_MyField', 'BigQueryResult_Row2_a_id'
UNION SELECT 'BigQueryResult_Row3_MyField', 'BigQueryResult_Row3_a_id'
UNION SELECT 'BigQueryResult_RowN_MyField', 'BigQueryResult_RowN_a_id'
) UPDATE MyTable
SET MyTable.MyField = UpdateTempTable.MyField
FROM MyTable
JOIN UpdateTempTable ON UpdateTempTable.a_id = MyTable.a_id;
COMMIT TRANSACTION
Related
I need to get a value from one database for a customer and update another database with that value.
The procedure below works but I need to have it go through table2 and update every customer in table1 with a matching CustomerID. I hate to use the word loop through but as I said, I am very new to this and lost. I have watched videos, and tried to search with no luck. Can someone point me to a tutorial or tell me if I am trying to do something I shouldn't be?
CREATE PROCEDURE dbo.bhshSample
as
BEGIN;
update table1 set
ptall = (
SELECT TOP (1) nsma1_ans
from table2
where nsma1_code = 'ptall'
order by nsma1_tm
)
where CustomerID = '4'
End;
In php, I would loop and do a select distinct CustomerID from table 2
then do the update using the variable I set but I can't seem to figure it out with stored procedure.
Use a correlated subquery, like this:
update table1 set
ptall = (
SELECT TOP (1) nsma1_ans
from table2
where nsma1_code = 'ptall'
and CustomerID = table1.CustomerID
order by nsma1_tm
)
I've been trying for a few hours (probably more than I needed to) to figure out the best way to write an update sql query that will dissallow duplicates on the column I am updating.
Meaning, if TableA.ColA already has a name 'TEST1', then when I'm changing another record, then I simply can't pick a value for ColA to be 'TEST1'.
It's pretty easy to simply just separate the query into a select, and use a server layer code that would allow conditional logic:
SELECT ID, NAME FROM TABLEA WHERE NAME = 'TEST1'
IF TableA.recordcount > 0 then
UPDATE SET NAME = 'TEST1' WHERE ID = 1234
END IF
But I'm more interested to see if these two queries can be combined into a single query.
I am using Oracle to figure things out, but I'd love to see a SQL Server query as well. I figured a MERGE statement can work, but for obvious reasons you can't have the clause:
..etc.. WHEN NOT MATCHED UPDATE SET ..etc.. WHERE ID = 1234
AND you can't update a column if it's mentioned in the join (oracle limitation but not limited to SQL Server)
ALSO, I know you can put a constraint on a column that prevents duplicate values, but I'd be interested to see if there is such a query that can do this without using constraint.
Here is an example start-up attempt on my end just to see what I can come up with (explanations on it failed is not necessary):
ERROR: ORA-01732: data manipulation operation not legal on this view
UPDATE (
SELECT d.NAME, ch.NAME FROM (
SELECT 'test1' AS NAME, '2722' AS ID
FROM DUAL
) d
LEFT JOIN TABLEA a
ON UPPER(a.name) = UPPER(d.name)
)
SET a.name = 'test2'
WHERE a.name is null and a.id = d.id
I have tried merge, but just gave up thinking it's not possible. I've also considered not exists (but I'd have to be careful since I might accidentally update every other record that doesn't match a criteria)
It should be straightforward:
update personnel
set personnel_number = 'xyz'
where person_id = 1001
and not exists (select * from personnel where personnel_number = 'xyz');
If I understand correctly, you want to conditionally update a field, assuming the value is not found. The following query does this. It should work in both SQL Server and Oracle:
update table1
set name = 'Test1'
where (select count(*) from table1 where name = 'Test1') > 0 and
id = 1234
Can I update multiple rows using one query?
How to union following queries:
UPDATE tablename SET col1='34355' WHERE id='2'
UPDATE tablename SET col1='152242' WHERE id='44'
You can use a virtual map table for this update.
update tablename
inner join (
select '34355' col1, '2' id union all
select '152242' col1, '44' id
) map on map.id = tablename.id
set tablename.col1 = map.col1
Using this pattern allows for easy expansion (just add rows to the map). It also allows MySQL to more predictably choose an index on tablename.id for the normal JOIN operation.
Can you? Sure. Should you? No way.
Think about the person looking at your code in five years. What's more readable, this:
UPDATE tablename SET col1='34355' WHERE id='2';
UPDATE tablename SET col1='152242' WHERE id='44';
or this (The Scrum Meister's answer):
UPDATE tablename SET col1 = IF(id='2', '34355','152242') WHERE id='2' OR id='44';
The second one is shorter, but it's a challenge to figure out exactly what it's doing. If you're worried about race conditions, make it a single transaction (in most modern DBMS):
BEGIN;
UPDATE tablename SET col1='34355' WHERE id='2';
UPDATE tablename SET col1='152242' WHERE id='44';
COMMIT;
That way you can be guaranteed no other query will run when row 2 is updated but row 44 is not.
You can use a OR clause combined with the IF() function (or CASE WHEN... for other RDBMS)
UPDATE tablename SET col1 = IF(id='2', '34355','152242')
WHERE id='2' OR id='44'
Generally the only way you can update multiple rows in a single query is if your where clause matches multiple rows... and then every row will have the same values set.
Past that you can do funky stuff with expressions in your set clauses, but generally it's cleaner to do multiple queries, unless there's a very specific reason you can't.
I use a cursor to iterate through quite a big table. For each row I check if value from one column exists in other.
If the value exists, I would like to increase value column in that other table.
If not, I would like to insert there new row with value set to 1.
I check "if exists" by:
IF (SELECT COUNT(*) FROM otherTabe WHERE... > 1)
BEGIN
...
END
ELSE
BEGIN
...
END
I don't know how to get that row which was found and update value. I don't want to make another select.
How can I do this efficiently?
I assume that the method of checking described above isn't good for this case.
Depending on the size of your data and the actual condition, you have two basic approaches:
1) use MERGE
MERGE TOP (...) INTO table1
USING table2 ON table1.column = table2.column
WHEN MATCHED
THEN UPDATE SET table1.counter += 1
WHEN NOT MATCHED SOURCE
THEN INSERT (...) VALUES (...);
the TOP is needed because when you're doing a huge update like this (you mention the table is 'big', big is relative, but lets assume truly big, +100MM rows) you have to batch the updates, otherwise you'll overwhelm the transaction log with one single gigantic transaction.
2) use a cursor, as you are trying. Your original question can be easily solved, simply always update and then check the count of rows updated:
UPDATE table
SET column += 1
WHERE ...;
IF ##ROW_COUNT = 0
BEGIN
-- no match, insert new value
INSERT INTO (...) VALUES (...);
END
Note that this approach is dangerous though because of race conditions: there is nothing to prevent another thread from inserting the value concurrently, so you may end up with either duplicates or a constraint violation error (preferably the latter...).
This is just psuedo code because I have no idea of your table structure but I think you will understand... basically Update the columns you want then Insert the columns you need. A Cursor operation sounds unnecessary.
Update OtherTable
Set ColumnToIncrease = ColumnToIncrease + 1
FROM CurrentTable Where ColumnToCheckValue is not null
Insert Into OtherTable (ColumnToIncrease, Field1, Field2,...)
SELECT
1,
?
?
FROM CurrentTable Where ColumnToCheckValue is not null
Without a sample, I think this is the best I can do. Bottom line: you don't need a cursor. UPDATE where a match exists (INNER JOIN) and INSERT where one does not.
UPDATE otherTable
SET IncrementingColumn = IncrementingColumn + 1
FROM thisTable INNER JOIN otherTable ON thisTable.ID = otherTable.ID
INSERT INTO otherTable
(
ID
, IncrementingColumn
)
SELECT ID, 1
FROM thisTable
WHERE NOT EXISTS (SELECT *
FROM otherTable
WHERE thisTable.ID = otherTable.ID)
I think you'd be better off using a view for this -- then it's always up to date, no risk of mistakenly double/triple/etc counting:
CREATE VIEW vw_value_count AS
SELECT st.value,
COUNT(*) AS numValue
FROM SOME_TABLE st
GROUP BY st.value
But if you still want to use the INSERT/UPDATE approach:
IF EXISTS(SELECT NULL
FROM SOMETABLE WHERE ... > 1)
BEGIN
UPDATE TABLE
SET count = count + 1
WHERE value = #value
END
ELSE
BEGIN
INSERT INTO TABLE
(value, count)
VALUES
(#value, 1)
END
What about Update statement with inner join to perform +1, and Insert selected rows that do not exist in the first table.
Provide the tables schema and the columns you want to check and update so I can help.
Regards.
Is there any way to update a table within the select_expr part of a mysql select query. Here is an example of what I am trying to achieve:
SELECT id, name, (UPDATE tbl2 SET currname = tbl.name WHERE tbl2.id = tbl.id) FROM tbl;
This gives me an error in mysql, but I dont see why this shouldn't be possible as long as I am not changing tbl.
Edit:
I will clarify why I cant use an ordinary construct for this.
Here is the more complex example of the problem which I am working on:
SELECT id, (SELECT #var = col1 FROM tbl2), #var := #var+1,
(UPDATE tbl2 SET col1 = #var) FROM tbl WHERE ...
So I am basically in a situation where I am incrementing a variable during the select statement and want to reflect this change as I am selecting the rows as I am using the value of this variable during the execution. The example given here can probably be implemented with other means, but the real example, which I wont post here due to there being too much unnecessary code, needs this functionality.
If your goal is to update tbl2 every time you query tbl1, then the best way to do that is to create a stored procedure to do it and wrap it in a transaction, possibly changing isolation levels if atomicity is needed.
You can't nest updates in selects.
What results do you want? The results of the select, or of the update.
If you want to update based on the results of a query you can do it like this:
update table1 set value1 = x.value1 from (select value1, id from table2 where value1 = something) as x where id = x.id
START TRANSACTION;
-- Let's get the current value
SELECT value FROM counters WHERE id = 1 FOR UPDATE;
-- Increment the counter
UPDATE counters SET value = value + 1 WHERE id = 1;
COMMIT;