Export SQL Server data from one database to another - sql

I have a SQL server database of a web application, my requirement is to read 2-3 table data from one source database and insert the data in the destination database. My input will be a Name and an ID, based on that I have to read data from the source database and I have to validate whether the similar Name already exists in the destination database. I have to do this via a C# windows application or a web application.
So far in my research, people have recommended using SqlBulkCopy or an SSIS package, I tried to transfer one table data using the following code.
using (SqlConnection connSource = new SqlConnection(csSource))
using (SqlCommand cmd = connSource.CreateCommand())
using (SqlBulkCopy bcp = new SqlBulkCopy(csDest))
{
bcp.DestinationTableName = "SomeTable";
cmd.CommandText = "myproc";
cmd.CommandType = CommandType.StoredProcedure;
connSource.Open();
using (SqlDataReader reader = cmd.ExecuteReader())
{
bcp.WriteToServer(reader);
}
}
The problem I'm facing is that, I have to copy 2 table data, based on table 1, table 2 value like ID(primary key) changes, I have to update this in the SqlDataReader, so in order to get this new ID in the destination database, I have to insert one table data first, then get the ID, then update this ID in my reader object and then do another SqlBulkCopy, this doesn't look like the ideal way to do this, is there any other to do this?

On the source SQL instance I would create a linked server referencing destination/target SQL instance and then I would create a stored procedure within source database thus:
USE SourceDatabase
GO
CREATE PROCEDURE dbo.ExportSomething
#param1 DataType1,
#param2 DataType2, ...
AS
BEGIN
INSERT LinkedServer.DestinationDatabase.dbo.TargetTable
SELECT ...
FROM dbo.Table1 INNER JOIN dbo.Table2 ....
WHERE Col1 = #param1 AND/OR ...
END
GO
Then, the final step is to call this stored procedure from client application.

Related

parameterizing all JOIN data in a large UPDATE operation

I have an app that makes a bunch of updates to objects hydrated with data from an SQL Server table and then writes the updates objects' data back to the DB in one query. I'm trying to convert this into a parameterized query so that I don't have to do manual escaping, conversions, etc.
Here's the most straightforward example query:
UPDATE TestTable
SET [Status] = DataToUpdate.[Status], City = DataToUpdate.City
FROM TestTable
JOIN
(
VALUES --this is the data to parameterize
(1, 0, 'A City'),
(2, 0, 'Another City')
) AS DataToUpdate(Id, [Status], City)
ON DataToUpdate.Id = TestTable.Id
I've also played around with using OPENXML to do this, but I'm still forced to write a bunch of escaping code when adding the values to the query. Any ideas on how to make this more elegant? I am open to ADO.NET/T-SQL solutions or platform-agnostic solutions.
One thought I had (but I don't really like how dynamic this is) is to dynamically create parameters and then add them to an ADO.NET SqlConnection, e.g.
for(int i = 0; i < data.Length; i++)
{
string paramPrefix = string.Format("#Item{0}", i);
valuesString.AppendFormat("{0}({1}Status)", Environment.NewLine, paramPrefix);
var statusParam = new SqlParameter(
string.Format("{0}Status", paramPrefix),
System.Data.SqlDbType.Int)
{ Value = data[i].Status };
command.Parameters.Add(statusParam);
}
I'm not exactly sure how you store your application data (and I don't have enough rep points to post comments) so I will ASSUME that the records are held in an object CityAndStatus which is comprised of int Id, string Status, string City held in a List<CityAndStatus> called data. That way you can deal with each record one at a time. I made Status a string so you can convert it to an int in your application.
With those assumptions:
I would create a stored procedure https://msdn.microsoft.com/en-us/library/ms345415.aspx in SQL Server that updates your table one record at at time.
CREATE PROCEDURE updateCityData (
#Id INT
,#Status INT
,#City VARCHAR(50)
)
AS
BEGIN TRAN
UPDATE TestTable
SET [Status] = #Status
,City = #City
WHERE Id = #Id
COMMIT
RETURN
GO
Then I would call the stored procedure https://support.microsoft.com/en-us/kb/310070 from your ADO.NET application inside a foreach loop that goes through each record that you need to update.
SqlConnection cn = new SqlConnection(connectionString);
cn.Open();
foreach (CityAndStatus item in data)
{
SqlCommand cmd = new SqlCommand("updateCityData",cn);
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue("#Id", item.Id);
cmd.Parameters.AddWithValue("#Status", Convert.ToInt32(item.Status));
cmd.Parameters.AddWithValue("#City", item.User);
cmd.ExecuteNonQuery();
cmd.Dispose();
}
cn.Close();
After that you should be good. The one thing left that might stand in your way is SQL Server makes web application users grant permission to execute stored procedures. So in SQL Server you may have to do something like this to allow your application to fire the stored proc.
GRANT EXECUTE
ON updateCityData
TO whateverRoleYouHaveGivenPermissionToExecuteStoredProcedures
Good Luck

How to merge two databases on two different servers?

I have two different databases, the client one is attached from a .MDF file to a .\SQLEXPRESS server. The master one is running on a server on another computer called COMPUTER_NAME.
I want to merge these using C# to run a .SQL file. I'll paste my code below for reference, but basically my problem is that if I connect to the server using
string sqlConnectionString = #"Server=.\SQLEXPRESS; Trusted_Connection=True";
Then I can't find the database on COMPUTER_NAME. And if I use
string sqlConnectionString = #"Server=COMPUTER_NAME; Trusted_Connection=True";
It will look for my .MDF file on the C: drive of COMPUTER_NAME, not the local machine.
How can I connect to both of these databases on different servers?
Additional info:
The SQL script I'm using. This worked perfectly back when both the databases were on the same server, but I can't do that anymore.
CREATE DATABASE ClientDB
ON (Filename = 'C:\Clayton.mdf')
, (Filename = 'C:\Clayton_log.ldf')
FOR ATTACH;
-- update the client from the master
MERGE [ClientDB].[dbo].[table] trgt
using [MasterDB].[dbo].[table] src
ON trgt.id = src.id
WHEN matched AND trgt.lastmodified <= src.lastmodified THEN -- if master row is newer
UPDATE SET trgt.[info] = src.[info], ... -- update the client
WHEN NOT matched BY source -- delete rows added by client
THEN DELETE
WHEN NOT matched BY target -- insert rows added by master
THEN INSERT ( [info], ... ) VALUES (src.[info], ... );
-- close all connections to database
ALTER DATABASE ClientDB SET SINGLE_USER WITH ROLLBACK IMMEDIATE;
-- detach database
EXEC sp_detach_db 'ClientDB', 'true';
And I run it using C# like so:
string sqlConnectionString = #"Server=.\SQLEXPRESS; Trusted_Connection=True";
string script = File.ReadAllText(Environment.CurrentDirectory + #"\MergeTotal.sql");
SqlConnection conn = new SqlConnection(sqlConnectionString);
IEnumerable<string> commandStrings = Regex.Split(script, #"^\s*GO\s*$",
RegexOptions.Multiline | RegexOptions.IgnoreCase);
conn.Open();
foreach (string commandString in commandStrings)
{
if (commandString.Trim() != "")
{
using (var command = new SqlCommand(commandString, conn))
{
command.ExecuteNonQuery();
}
}
}
I don't care if the entire process happens in the .SQL or in C# so long as it has the desired effect.
Thanks in advance for any guidance or recommendations.
Linking the servers would help you to be able to access the data simultaneously, if that's the requirement. If you're looking to merge data together, though, I'd suggest you check out sp_generate_merge to pull the data into a merge script for you (very handy for moving data). See also my question on generating merge data here.
Okay, I had to completely throw out the whole .MDF thing. Instead of attaching and reattaching the database from an .MDF, I just set up the database.
Here's my code to initialize the local database on the tablet:
CREATE DATABASE LocalClaytonDB
ON (Filename = 'C:\ProgramData\Clayton\Clayton.mdf')
, (Filename = 'C:\ProgramData\Clayton\Clayton_log.ldf')
FOR ATTACH;
GO
EXEC sp_addlinkedserver #server='Server'
Here's my code to synchronize the two databases:
-- update the client from the master
MERGE [LocalClaytonDB].[dbo].[tableName] trgt
using [Server].[Clayton].[dbo].[tableName] src
ON trgt.id = src.id
WHEN matched AND trgt.lastmodified <= src.lastmodified THEN
-- if the master has a row newer than the client
-- update the client
UPDATE SET trgt.[allColumns] = src.[allColumns],
trgt.[id] = src.[id],
trgt.[lastmodified] = src.[lastmodified]
-- delete any rows added by a client
WHEN NOT matched BY source
THEN
DELETE
-- insert any rows added by the master
WHEN NOT matched BY target
THEN
INSERT ( [allColumns],
[id],
[lastmodified])
VALUES (src. [allColumns],
src.[id],
src.[lastmodified]);
-- now we update the master from the client
-- Note:
-- because the serverDB is a linked server
-- we can't use another MERGE statement, otherwise
-- we get the error: "The target of a MERGE statement
-- cannot be a remote table, a remote view, or a view over remote tables."
UPDATE
serverDB
SET
[allColumns] = [localDB].[allColumns],
[id] = [localDB].[id],
[lastmodified] = [localDB].[lastmodified]
FROM
[Server].[Clayton].[dbo].[tableName] serverDB
INNER JOIN
[LocalClaytonDB].[dbo].[tableName] localDB
-- update where the id is the same but the client is newer than the master
ON serverDB.id = localDB.id
AND localDB.lastmodified >= serverDB.lastmodified

Passing array to a store procedure

I want to pass an array of 20k IDs to my stored procedure param in order to update a certain table.
Instead of running 20k update queries separately, I want to run 1 query to update all, it should improve my performances.
Any knows I can I pass a param to my stored proc?
I understood that NVARCHAR(MAX) is limited to 8000 chars, is it possible at all to send such a huge data using stored proc param?
Use a Table Value Parameter instead. See Use Table-Valued Parameters (Database Engine). A TVP is exactly as the name implies: a parameter that is a table. You assign to it from your client code a DataTable and the procedure (or you ad-hoc SQL codE) receives the entire DataTable as a parameter.This is an MSDN copied example:
// Assumes connection is an open SqlConnection.
using (connection)
{
// Create a DataTable with the modified rows.
DataTable addedCategories = CategoriesDataTable.GetChanges(
DataRowState.Added);
// Define the INSERT-SELECT statement.
string sqlInsert =
"INSERT INTO dbo.Categories (CategoryID, CategoryName)"
+ " SELECT nc.CategoryID, nc.CategoryName"
+ " FROM #tvpNewCategories AS nc;"
// Configure the command and parameter.
SqlCommand insertCommand = new SqlCommand(
sqlInsert, connection);
SqlParameter tvpParam = insertCommand.Parameters.AddWithValue(
"#tvpNewCategories", addedCategories);
tvpParam.SqlDbType = SqlDbType.Structured;
tvpParam.TypeName = "dbo.CategoryTableType";
// Execute the command.
insertCommand.ExecuteNonQuery();
}

What's the best way to update a List on the database?

If I have the following list in C# that was loaded from the database
List<User> user = GetUsers(foo);
and it was updated and I want to store those changes in the database what's the best way of doing it using SQL? It should insert the records added to that list, updated the modified records and delete the ones that are not present in the collection.
I'm no using the EntityFramework so I need to do this using SQL.
Copy this list to datatable and set datatable RowStat as (modified,deleted,new)
and update datatable using sqldataadapter
Here's an example that adds or inserts a row. It searches for a row with a specific UserID. If the row exists, it uses update to grant the user a point. If the row does not exist, a new row is created with insert.
var connectionString = "Data Source=myServerAddress;" +
"Initial Catalog=myDataBase;User Id=myUsername;Password=myPassword;"
using (var con = new SqlConnection(connectionString))
{
con.Open();
var com = con.CreateCommand();
com.Parameters.AddWithValue("#UserId", userId);
com.CommandText = #"
if exists (select * from YourTable where UserId = #UserId)
update YourTable set TrollPoints = TrollPoints + 1 where UserId = #UserId
else
insert YourTable (UserId, TrollPoints) values (#UserId, 1)
";
com.ExecuteNonQuery();
}
The use of parameters allows the server to chache the execution plan, and also helps against SQL injection.

Insert INTO in vb.net

I have 2 databases. In first one I have 10 tables. Second one is only 1 table. I would like to select 1 columns from each table from 1st database and Insert INTO another database. How can I manage this using INSERT INTO statement in VB.net?
I deleted my previous answer saying that you have to manually copy over the data. For now, let's assume you want to do this with a SELECT INTO statement.
The following code shows you how to execute a SQL command on your database using a ADO.NET connection and command object:
' Open a connection to your database (e.g. in a SQL Server): '
Using connection As IDbConnection = New SqlConnection("<Connection string>")
connection.Open()
Try
' Define the SQL command to be executed here: '
Dim command As IDbCommand = connection.CreateCommand()
command.CommandText = "SELECT <...> INTO <...>"
' Execute the command: '
command.ExecuteNonQuery()
Finally
connection.Close()
End Try
End Using
I hope this helps:
From sql side, you'll just need to write a stored procedure to insert into (ten) hash tables and select/insert them into your target table.
In Vb.net, you'll need: a connection object and a command object to call your stored procedure