parameterizing all JOIN data in a large UPDATE operation - sql

I have an app that makes a bunch of updates to objects hydrated with data from an SQL Server table and then writes the updates objects' data back to the DB in one query. I'm trying to convert this into a parameterized query so that I don't have to do manual escaping, conversions, etc.
Here's the most straightforward example query:
UPDATE TestTable
SET [Status] = DataToUpdate.[Status], City = DataToUpdate.City
FROM TestTable
JOIN
(
VALUES --this is the data to parameterize
(1, 0, 'A City'),
(2, 0, 'Another City')
) AS DataToUpdate(Id, [Status], City)
ON DataToUpdate.Id = TestTable.Id
I've also played around with using OPENXML to do this, but I'm still forced to write a bunch of escaping code when adding the values to the query. Any ideas on how to make this more elegant? I am open to ADO.NET/T-SQL solutions or platform-agnostic solutions.
One thought I had (but I don't really like how dynamic this is) is to dynamically create parameters and then add them to an ADO.NET SqlConnection, e.g.
for(int i = 0; i < data.Length; i++)
{
string paramPrefix = string.Format("#Item{0}", i);
valuesString.AppendFormat("{0}({1}Status)", Environment.NewLine, paramPrefix);
var statusParam = new SqlParameter(
string.Format("{0}Status", paramPrefix),
System.Data.SqlDbType.Int)
{ Value = data[i].Status };
command.Parameters.Add(statusParam);
}

I'm not exactly sure how you store your application data (and I don't have enough rep points to post comments) so I will ASSUME that the records are held in an object CityAndStatus which is comprised of int Id, string Status, string City held in a List<CityAndStatus> called data. That way you can deal with each record one at a time. I made Status a string so you can convert it to an int in your application.
With those assumptions:
I would create a stored procedure https://msdn.microsoft.com/en-us/library/ms345415.aspx in SQL Server that updates your table one record at at time.
CREATE PROCEDURE updateCityData (
#Id INT
,#Status INT
,#City VARCHAR(50)
)
AS
BEGIN TRAN
UPDATE TestTable
SET [Status] = #Status
,City = #City
WHERE Id = #Id
COMMIT
RETURN
GO
Then I would call the stored procedure https://support.microsoft.com/en-us/kb/310070 from your ADO.NET application inside a foreach loop that goes through each record that you need to update.
SqlConnection cn = new SqlConnection(connectionString);
cn.Open();
foreach (CityAndStatus item in data)
{
SqlCommand cmd = new SqlCommand("updateCityData",cn);
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue("#Id", item.Id);
cmd.Parameters.AddWithValue("#Status", Convert.ToInt32(item.Status));
cmd.Parameters.AddWithValue("#City", item.User);
cmd.ExecuteNonQuery();
cmd.Dispose();
}
cn.Close();
After that you should be good. The one thing left that might stand in your way is SQL Server makes web application users grant permission to execute stored procedures. So in SQL Server you may have to do something like this to allow your application to fire the stored proc.
GRANT EXECUTE
ON updateCityData
TO whateverRoleYouHaveGivenPermissionToExecuteStoredProcedures
Good Luck

Related

Export SQL Server data from one database to another

I have a SQL server database of a web application, my requirement is to read 2-3 table data from one source database and insert the data in the destination database. My input will be a Name and an ID, based on that I have to read data from the source database and I have to validate whether the similar Name already exists in the destination database. I have to do this via a C# windows application or a web application.
So far in my research, people have recommended using SqlBulkCopy or an SSIS package, I tried to transfer one table data using the following code.
using (SqlConnection connSource = new SqlConnection(csSource))
using (SqlCommand cmd = connSource.CreateCommand())
using (SqlBulkCopy bcp = new SqlBulkCopy(csDest))
{
bcp.DestinationTableName = "SomeTable";
cmd.CommandText = "myproc";
cmd.CommandType = CommandType.StoredProcedure;
connSource.Open();
using (SqlDataReader reader = cmd.ExecuteReader())
{
bcp.WriteToServer(reader);
}
}
The problem I'm facing is that, I have to copy 2 table data, based on table 1, table 2 value like ID(primary key) changes, I have to update this in the SqlDataReader, so in order to get this new ID in the destination database, I have to insert one table data first, then get the ID, then update this ID in my reader object and then do another SqlBulkCopy, this doesn't look like the ideal way to do this, is there any other to do this?
On the source SQL instance I would create a linked server referencing destination/target SQL instance and then I would create a stored procedure within source database thus:
USE SourceDatabase
GO
CREATE PROCEDURE dbo.ExportSomething
#param1 DataType1,
#param2 DataType2, ...
AS
BEGIN
INSERT LinkedServer.DestinationDatabase.dbo.TargetTable
SELECT ...
FROM dbo.Table1 INNER JOIN dbo.Table2 ....
WHERE Col1 = #param1 AND/OR ...
END
GO
Then, the final step is to call this stored procedure from client application.

looping by pairs in SQL

I've looked at the few topics here that I thought would answer this, but they don't seem to be exactly what I'm looking for.
Basically, I have a database that, among other things, has a user table. Due to those very same users, we often end up with duplicates. In other words, one person has two separate accounts and uses both of them until we catch on and merge them into one account, tell them which it is and to stop trying to use the second account. Unfortunately, the decision of which account to keep and which to axe isn't always something that follows a formula, but rather an admin's knowledge of the situation and I'm informed on a line-by-line basis of which stays and goes.
Part of the merge involves modifying several tables by replacing any occurrences of the discarded ID with the ID we're keeping. The last guy to have this job had a script to do this, but it simply assumed that the most recent login was the one to keep and pulled those pairs of IDs and made the mods. I, however, can't rely on that being the case. I have a spreadsheet with pairs of IDs that I would love to be able to run through and process. Until now, I've been doing this all by hand, one at a time.
So, what I'm looking for is something to the effect of:
foreach (x,y) in (oldID1, newID1, oldID2, newID2, ...){
go through tables and change all instances of x to y;
}
Hopefully, that was clear enough.
Thanks!
For simple one-off jobs like this, I like to use LINQPad to whip up a quick script. You could just use an anonymous-typed array for your id-pairs, and execute the script for each pair (assuming SQL Server):
var idpairs = new [] {
new { OldId = 5, NewId = 10 },
new { OldId = 23, NewId = 45 },
new { OldId = 443, NewId = 299 }
};
using (var con = new SqlConnection(connectionString)) {
foreach (var idpair in idpairs) {
var oldId = idpair.OldId;
var newId = idpair.NewId;
// prepare script with old/new id's inserted
var sql = string.Format("blah blah blah {0} blah blah {1} blah",
oldId, newId);
// execute the command
var cmd = con.CreateCommand();
cmd.CommandText = sql;
cmd.ExecuteNonQuery();
}
}
I am going to make a number of assumptions since the information is not completely supplied. One assumption is that the spreadsheet is separated into the fields described above (oldID,newID)
Import the spreadsheet into a table on SQL.
Then, use the following code (first for MS SQL):
DECLARE #oldID int, #newID int
DECLARE CURSOR idCursor STATIC FOR
SELECT oldID,newID FROM idSpreadsheetTable
OPEN idCursor
FETCH NEXT FROM idCursor INTO #oldID,#newID
WHILE (##FETCH_STATUS = 0)
BEGIN
UPDATE Table1 SET idField = #newID WHERE idField = #oldID
... More update table commands
FETCH NEXT FROM idCursor INTO #oldID,#newID
END
CLOSE idCursor
DEALLOCATE idCursor
and here is mySQL syntax:
DECLARE done INT DEFAULT FALSE;
DECLARE oldID,newID int;
DECLARE idCursor CURSOR FOR SELECT oldID,newID FROM idSpreadsheetTable
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
OPEN idCursor;
read_loop: LOOP
FETCH idCursor INTO oldID, newID;
IF done THEN
LEAVE read_loop;
END IF;
UPDATE Table1 SET idField = newID WHERE idField = oldID;
... More update table commands
END LOOP;
CLOSE idCursor;

Passing array to a store procedure

I want to pass an array of 20k IDs to my stored procedure param in order to update a certain table.
Instead of running 20k update queries separately, I want to run 1 query to update all, it should improve my performances.
Any knows I can I pass a param to my stored proc?
I understood that NVARCHAR(MAX) is limited to 8000 chars, is it possible at all to send such a huge data using stored proc param?
Use a Table Value Parameter instead. See Use Table-Valued Parameters (Database Engine). A TVP is exactly as the name implies: a parameter that is a table. You assign to it from your client code a DataTable and the procedure (or you ad-hoc SQL codE) receives the entire DataTable as a parameter.This is an MSDN copied example:
// Assumes connection is an open SqlConnection.
using (connection)
{
// Create a DataTable with the modified rows.
DataTable addedCategories = CategoriesDataTable.GetChanges(
DataRowState.Added);
// Define the INSERT-SELECT statement.
string sqlInsert =
"INSERT INTO dbo.Categories (CategoryID, CategoryName)"
+ " SELECT nc.CategoryID, nc.CategoryName"
+ " FROM #tvpNewCategories AS nc;"
// Configure the command and parameter.
SqlCommand insertCommand = new SqlCommand(
sqlInsert, connection);
SqlParameter tvpParam = insertCommand.Parameters.AddWithValue(
"#tvpNewCategories", addedCategories);
tvpParam.SqlDbType = SqlDbType.Structured;
tvpParam.TypeName = "dbo.CategoryTableType";
// Execute the command.
insertCommand.ExecuteNonQuery();
}

What's the best way to update a List on the database?

If I have the following list in C# that was loaded from the database
List<User> user = GetUsers(foo);
and it was updated and I want to store those changes in the database what's the best way of doing it using SQL? It should insert the records added to that list, updated the modified records and delete the ones that are not present in the collection.
I'm no using the EntityFramework so I need to do this using SQL.
Copy this list to datatable and set datatable RowStat as (modified,deleted,new)
and update datatable using sqldataadapter
Here's an example that adds or inserts a row. It searches for a row with a specific UserID. If the row exists, it uses update to grant the user a point. If the row does not exist, a new row is created with insert.
var connectionString = "Data Source=myServerAddress;" +
"Initial Catalog=myDataBase;User Id=myUsername;Password=myPassword;"
using (var con = new SqlConnection(connectionString))
{
con.Open();
var com = con.CreateCommand();
com.Parameters.AddWithValue("#UserId", userId);
com.CommandText = #"
if exists (select * from YourTable where UserId = #UserId)
update YourTable set TrollPoints = TrollPoints + 1 where UserId = #UserId
else
insert YourTable (UserId, TrollPoints) values (#UserId, 1)
";
com.ExecuteNonQuery();
}
The use of parameters allows the server to chache the execution plan, and also helps against SQL injection.

How can I make a stored procedure return a "dataset" using a parameter I pass?

I'm very new to Stored Procedures.
Say I have a IDCategory (int) and I pass that to a stored procedure. In normal-talk, it'd go:
Find me all the listings with an
IDCategory equal to the IDCategory I'm
telling you to find.
So it would find say 3 listing, and create a table with columns:
IDListing, IDCategory, Price, Seller, Image.
How could I achieve this?
To fill a dataset from a stored procedure you would have code like below:
SqlConnection mySqlConnection =new SqlConnection("server=(local);database=MyDatabase;Integrated Security=SSPI;");
SqlCommand mySqlCommand = mySqlConnection.CreateCommand();
mySqlCommand.CommandText = "IDCategory";
mySqlCommand.CommandType = CommandType.StoredProcedure;
mySqlCommand.Parameters.Add("#IDCategory", SqlDbType.Int).Value = 5;
SqlDataAdapter mySqlDataAdapter = new SqlDataAdapter();
mySqlDataAdapter.SelectCommand = mySqlCommand;
DataSet myDataSet = new DataSet();
mySqlConnection.Open();
mySqlDataAdapter.Fill(myDataSet);
Your connection string will be different and there are a few different ways to do this but this should get you going.... Once you get a few of these under your belt take a look at the Using Statement. It helps clean up the resources and requires a few less lines of code. This assumes a Stored Procedure name IDCategory with one Parameter called the same. It may be a little different in your setup.
Your stored procedure in this case will look something like:
CREATE PROC [dbo].[IDCategory]
#IDCategory int
AS
SELECT IDListing, IDCategory, Price, Seller, Image
FROM whateveryourtableisnamed
WHERE IDCategory = #IDCategory
Here's a link on Stored Procedure basics:
http://www.sql-server-performance.com/articles/dba/stored_procedures_basics_p1.aspx
Here's a link on DataSets and other items with ADO.Net:
http://authors.aspalliance.com/quickstart/howto/doc/adoplus/adoplusoverview.aspx
Have a table in your database that contains those 5 fields you wish and query it.
Example:
Select IDListing, IDCategory, Price, Seller, Image
From [listingtable] --whatever your table is called
where IDCategoryID = #IDCategoryID
Entire stored procedure:
CREATE PROCEDURE sp_Listing_Get
#IDCategory int
AS
DECLARE #categoryid
SET #categoryid = #IDCategory
BEGIN
SELECT t.idlisting,
t.idcategory,
t.price,
t.seller,
t.image
FROM [databaseName].dbo.LISTING t
WHERE t.idcategoryid = #categoryid
END
Replace [databaseName] with the name of your database. The benefit to using the two period format is that the sproc will return results as long as the user who is executing the sproc has access to the table (and database).
The #categoryid is used to deal with SQL Servers parameter sniffing issue.