I would like to run a job that automatically snapshots a DB1 (a stored procedure) and merges the result with a table in DB2. Basically I would like to query DB1 from DB2.
What is the best way to do this? They are run on two different SQL Servers in two different resource groups in Azure.
At the moment it won't let me created a linked server - tells me the sp does not exist when I try create it.
You have two databases run on two different SQL Servers in two different resource groups in Azure.
Basically you would like to query DB1 from DB2.
For Azure SQL database, you want to query across different database, you can use Azure SQL Database elastic query.
Summary:
The elastic query feature (in preview) enables you to run a Transact-SQL query that spans multiple databases in Azure SQL Database. It allows you to perform cross-database queries to access remote tables, and to connect Microsoft and third-party tools (Excel, Power BI, Tableau, etc.) to query across data tiers with multiple databases. Using this feature, you can scale out queries to large data tiers in SQL Database and visualize the results in business intelligence (BI) reports.
You can using the bellow code to query from the remote database:
EXEC sp_execute_remote
N'MyExtSrc',
N'select count(w_id) as foo from warehouse'
For more details, please see: Stored procedure for remote T-SQL execution: sp_execute_remote.
You can also reference this blog: Is it Possible to call Functions and Stored Procedures of One database In another database- Azure Sql server.
With elastic query, you can call the stored procedure in DB1, merges the result with a table in DB2
Hope this helps.
I have 11 databases in which I'm having tables contains User Details i.e. all employee details. There I have a column "Status"(which is 1 for Active and 0 for Inactive). I have a regular tasks for updating "Status" column value 0 or 1 for mentioned employees and for that, I have to go through all the databases then User table then I have to update. The same task i have to do for all the database and it consumes a lot of time.
If I will get a short Query or Procedure that I have to run once and will do all updation at once then, it would be a great help.
I see a couple of possible options.
You could build an SSIS package to connect to each database and do the necessary updates provided the criteria of which employees to update and what to update them to could be found within the database or some external source such as a text file.
Alternatively, you could use SQLCMD mode in SQL Server Management Studio and then within your SQL script use CONNECT command to switch to each server and database something like this...
:CONNECT Server1
USE Database1
--put your update SQL script
:CONNECT Server2
USE Database2
--put your update SQL script
...
These links provide some further information on using SQLCMD mode...
Connecting to multiple servers in a Query Window using SQLCMD
SQL Server SQLCMD Basics
Noel
As you mentioned, you have 11 databases.
Problem : First, you are using very bad approach for database design,
What really Happens : When you are using multiple databases and you need to check in every database, then the server needs to connect to different database again and again, which takes very more time compared to switching into the tables, because of connection handling.
Solution : In your case, you have only one option to connect different databases in loops and then run the query in the loop for every DB.
Suggestion : you should keep all the data in the same database, you can use an extra column in tables to keep track your data to different entities.
I am trying to find a way of linking data from two servers in SSRS report builder 2.0
ultimately using one dataset to query 2 databases in 2 different servers.
The SQL that i have is very basic as shown below and i can run the query easily in SQL management studio.
select * from
[server1].[DES].[dbo].[Daily] dr
LEFT JOIN [server2].[VES].[dbo].[Rou] mr
ON dr.ID = mr.id
Also the access i have to the reporting server is read only so i cant really make any administrative changes to the configuration.
I have explored the connection string to create a connection but it only allows me to connect to one server for each dataset. what i am trying to do is use one dataset
Any help will be greatly appreciated as i am a junior in SQL and SSRS
Using SSRS and MS SQL 2008 R2
if you executing this query from your SSMS it means this other server
is a linked server. then it should be no different to execute it from
SSRS.
Yes it is right you can only add one server to your data source of
your dataset whether its a shared data source or embedded.
But for instance if you have a data source pointing to say Server A
when you executing queries which will be pulling data from Server A
and also from server B you will Use fully Qualified name for the
Objects from server B and two part name from server A.
something like this ...
SELECT *
FROM Schema.Table_Name A INNER JOIN [ServerNameB].DataBase.Schema.TableName B
ON A.ColumnA = B.ColumnA
obviously ServerB has to be a Linked Server.
I can think of two possible solutions
Option 1:
Creating a stored procedure on server1 (With a query accessing the another server). And call this stored procedure from SSRS (Dataset).
Option 2:
Create two dataset with two different connection strings on the RDL (one for Server1 and another for Server2).
Use the "LookUpDataSet" function in Report Builder to merge the results using the key fields.
Option 2 is preferable.
I want something like :
insert into server2.database1.table1 select * from server1.database1.table1
both tables are exactly the same.
how can I Copy data between two server instances?
SQL - Linked Server
If both servers are SQL Server, you can set up Linked servers - I would suggest using an SQL account for security there.
Then you can simply perform
insert into server2.database1.dbo.table1
select * from server1.database1.dbo.table1 where col1 = 'X'
If you run the query in SQL Management studio connected to server1, and current database set to database1, you won't need the prefix
server1.database1.dbo.
Also, the linked server would be configured on server1, to connect to server2 (rather than the other way around).
If you have the correct OLE DB drivers, this method can also work between different types of RDBMS (ie. non-SQL Server ones).
Open Query
Note: Beware not to rely on linked servers too much especially for filtering, and for joins across servers, as they require data to be read in full to the originating RDBMS before any conditions can be applied. Many complications can arise from Linked Servers, so read up before you embark, as even version differences might cause headaches.
I recommend you use the OPENQUERY command for SQL Servers to get around such limitations. Here's an example, but you should find help specific to your needs through further research:
insert into server2.database1.dbo.table1
select * from OPENQUERY(server1, 'select * from database1.dbo.table1 where col1 = ''X''');
The above code is more efficient, filtering the data on the source server (and using available indexes), before pumping the data through, saving bandwidth/time/resources of both the source and destination servers.
(Also note the double quote '', is an escape sequence to produce a single quote.)
SQL - Temporarily on the same server
Would enable (note the underscore):
insert into server2_database1.dbo.table1
select * from database1.dbo.table1
Still within the SQL query domain. If you can temporarily move the database on server2 to server1, then you won't need the linked server. A rename of the database would appear to be required while co-locating on server1. Achieving such co-location could use various methods, I suggest shrinking database files before proceeding with either:
Backup/Restore - Backup on server2, Restore on server1 (with different name) - perform insert as described above, but without the server1 or server2 prefixes. Then reverse - backup on server1, restore on server2/
Detach/Attach - Rename database, Detach on server2, (compress), copy files to server 1, (decompress), attach on server1, perform insert. Then reverse...
In either case, SQL Server version could be a barrier. If server1 is of a lower SQL version, then both backup and detach/attach methods will likely fail. This can be worked around by moving the server1 database to server2, which may or may not be more suitible.
Other Methods
May be suitable, non-SQL/TSQL method failing favorable environmental factors for previously mentioned methods. And if you have the correct access (OLE DB Drivers, etc..), this method can also work between different types of RDBMS (ie. non-SQL Server ones), and data-sources (such as XML, flatfiles, Excel Spreadsheets...)
SSIS Explicitly with Business Development Management Studio - direct datapump or using delimited file intermeditary.
SSIS Implicitly through SQL Management Studio, by right clicking the database1 on server1 > Tasks > Export, then completing the wizard. May work direct to server2, or using a flat-file intermeditary.
.Net Programming with SqlBulkInsert (I believe the SSIS datapump uses such an object), I can go into more detail about this, if it interests you.
Eg. of SQLBulkInsert (psedo-C# code)
SqlConnection c = new SqlConnection("connectionStringForServer1Database1Here");
SqlConnection c2 = new SqlConnection("connectionStringForServer2Database1Here");
c.Open();
SqlCommand cm = new SqlCommand(c);
cm.CommandText = "select * from table1;";
using (SqlDataReader reader = cm.ExecuteReader())
{
using (SqlBulkInsert bc = new SqlBulkInsert(c))
{
c2.Open();
bc.DestinationTable = "table1";
bc.WriteToServer(reader);
}
}
Pretty cool huh? If speed/efficiency is a concern - SqlBulkInsert based approaches (Such as SSIS) are the best.
Update - Modifying the destination table
If you need to update the destination table, I recommend that you:
Write to a staging table on the destination database (a temporary table, or proper table which you truncate before and after process), the latter is preferable. The former may be your only choice if you don't have CREATE TABLE rights. You can perform the transfer using any one of the above options.
Run a MERGE INTO command as per your requirements from the staging table to the destination table. This can Insert, Update and Delete as required very efficiently.
Such a whole process could be enhanced with a sliding window (changes since last checked), only taking recently changed rows in the source an applying to the destination, this complicates the process, so you should at least accomplish the simpler one first. After completing a sliding window version, you could run the full-update one periodically to ensure there are no errors in the sliding window.
To copy data between two different servers you have several options:
Use linked servers.
Use the data import export wizard.
Use a third party tool such as Red Gate SQL Data Compare.
Similar to Todd C# SqlBulkCopy
Generally this is easier than creating linked servers.
Create a unit test and run the below, if you have triggers then be careful and you will need ALTER permissions.
[Test]
public void BulkCopy()
{
var fromConnectionString = #"fromConnectionString";
var destinationConnectionString = #"destConnectionString2";
using (var testConnection = new SqlConnection(fromConnectionString))
{
testConnection.Open();
var command = new SqlCommand("select * from MyTable;", testConnection);
using (var reader = command.ExecuteReader())
{
using (var destinationConnection = new SqlConnection(destinationConnectionString))
{
using (var bc = new SqlBulkCopy(destinationConnection))
{
destinationConnection.Open();
bc.DestinationTableName = "dbo.MyTable";
bc.WriteToServer(reader);
}
}
}
}
}
}
The best way to do this would be to create a "linked server".
And then you can use below statement into your insert statement in order to define your table
[linkedserver].databasename.dbo.tablename
On Server A add a linked server (B)
http://msdn.microsoft.com/en-us/library/ms188279.aspx
Then you can transfer data between the two.
Export table data from one SQL Server to another
HTH
First You need to add the server
Eg. Server 1 and Server 2
sp_addlinkedserver 'Server-2'
then copy your data from that server to your server by using following query
In Server-1 Write
select * INTO Employee_Master_bkp
FROM [Server-2].[DB_Live].[dbo].[Employee_Master]
If you need an alternative without using Linked Servers, my favorite option is use the command line BCP utility.
With this bulk copy tool, you can export the data to a flat file, copy the file across the network and import it (load it) onto the target server.
https://learn.microsoft.com/en-us/sql/tools/bcp-utility
I have database connection to database DB1. The only thing I could do - execute any t-sql statements including using stored procedures. I want to export the specific table (or even the specific rows of specific table) to my local database. As you can read abve, DBs are on diffrent servers meaning no direct connection is possible. Therefore question: Is it possible to write query that returns the other query to execute on local server and get data? Also note, that table contains BLOBs. Thanks.
If you have SQL Server Management Studio, you can use the data import function on your local database to get the data. It works as long as you have Read/Select access on the tables you are trying to copy.
If you have Visual Studio you can use the database tools in there to move data between two servers as long as you can connect to both from your workstation.
Needs Ultimate or Premium though:
http://msdn.microsoft.com/en-us/library/dd193261.aspx
RedGate has some usefull tools too:
http://www.red-gate.com/products/sql-development/sql-compare/features
Maybe you should ask at https://dba.stackexchange.com/ instead.
If you can login to the remote db (where you can only issue t-sql), you may create linked server on your local server to the remote and use it later directly in queries, like:
select * from [LinkedServerName].[DatabaseName].[SchemaName].[TableName]