I have two tables Table A and B that is present in a Azure SQL Database. I have the same database clone running in my local, but I want to populate the data that is present in Azure by using SSMS Export Data option. While using that option I specify the source and destination and then choose the option of "Write a query to specify the data to transfer"
And then I add the query "Select * from A where Condition1" and select the destination table here:
The issue is if I have 5 tables to export data from, I have to do this whole process 5 times, only difference is the queries and destination tables. Anyone has any idea how can I do this whole thing faster by some other means? I just need to copy data using some select statements with where clauses.
As per the Official Documentation
When you select  Write a query to specify the data to transfer, you can only copy the results of one query to one destination table.
So, you have to repeat the entire process for multiple times if you want to export data like that.
You can use the following ways for importing and exporting data:
Use Transact-SQL statements.
Use BCP (Bulk Copy Program) from the command prompt.
If you want to design a custom data import, you can use SQL Server Integration Services.
Use Azure Data factory.
Use BACPAC file. Refer this material by accu web hosting to know about it. Rather than querying before exporting the data, instead you can delete the unwanted data in destination database after exporting using delete statement.
REFERENCE:
Import & export data from SQL Server & Azure SQL Database - SQL Server | Microsoft Docs
Related
I am in need of suggestion to move data from a particular table in one azure sql database to the other azure sql database which has the same table structure without using elastic query
Using SQL Server Management Studio to connect to SQL azure database, right click the source database and select generate scripts.
During the wizard, after have select the tables that you want to output to a query window, then click advanced. About half way down the properties window there is an option for "type of data to script". Select that and change it to "data only", then finish the wizard.
The heck the script, rearrange the inserts for constraints, and change the using at the top to run it against my target DB.
Then right click on the target database and select new query, copy the script into it, and run it.
This will migrate the data.
Please consider using the "Transfer SQL Server Objects task" in SSIS. You can learn all the advantages it provides on this article.
You can use PowerShell to query each database and move data between them as needed. Here's an example article on how to get this done.
Using PowerShell when working with Azure has a number of other benefits in what you can do and can control as well. It's a good choice to spend time learning.
In the source database I created SPs to select the data from the tables.
In the target database I created table types (which would be available in programmability) for the tables with the same structure as in the source.
I used Azure function to move the data into table type from source.
In the target database I created SPs to insert data into the tables from their respective table types.
After ensuring the transfer of data, I would be deleting those records moved to the target in the source database and for this I created SPs.
I cannot use linked server.
Both databases on both servers have the same structure but different data.
I have 10k rows to transfer from the DB on one server to the same DB on the other. I cannot restore the DB on the other server as it will take a huge amount of space that I don't have on the other server.
So, I have 2 options that I don't know how to execute:
Backup and restoring only one table - the table is linked to many other tables but these other tables exist on the other server too. Can I somehow delete or remove the other tables or make a backup only over one table?
I need to transfer 10k rows. How is it possible to create 10k insert queries based on selected 10k rows?
Can I somehow delete or remove the other tables or make a backup only over one table?
No you can not do this, unfortunately
How is it possible to create 10k insert queries based on selected 10k rows?
Right-click on Database -> Tasks -> Generate scripts -> (Introduction) Next
Chose Select specific database objects -> Tables, chose table you need -> Next
Advanced -> Search for Types of data script change from Schema only (by default) to Data only -> OK
Chose where to save -> Next -> Next. Wait the operation to end.
This will generate the file with 10k inserts.
Another way is to use Import/Export wizard (the most simple way for one-time-import/export) if you have link between databases.
There are many ways to choose from, here is one way using BCP. That's a tool that ships with SQL Server to Import and Export Bulk Data.
The outlines of the process:
Export the data from the source server to a file using BCP - BCP OUT for a whole table, or BCP QUERYOUT with a query to select the 10k rows you want exported
Copy the file to the destination server
Import the data using BCP on the destination database - BCP IN.
My suggestion would be to export these rows to excel( you can do this by copy pasting your query output) and transfer this to other server and import it there.
this is the official method :-
https://learn.microsoft.com/en-us/sql/relational-databases/import-export/import-data-from-excel-to-sql
and this is the the unofficial method :
http://therealdanvega.com/blog/2010/08/04/create-sql-insert-statements-from-a-spreadsheet.
Here I have assumed that you only need to transfer the transactional data and your reference data is same on both server. So you will need to execute only one query for exporting your data
I would definietely go down the SSIS route once you use SSIS to do a task like this you will not use anything else very simple to script up. You can use any version and it will be a simple job and very quick.
Open new SSIS project in available visual studio version/s there are many different but even a 2008 version will do this simple task you may have to install integration services or something similar used to be called bids (business information development studio in 2008) (anything up to 2015 although support is nearly there in 2017)
add a data flow task
double click the data flow task
Bottom of screen add two connection managers (1 to source and 1 to destination database)
add oledb source pointing to source database table
add oledb destination pointing to destination database table
drag line between the source and destination (should auto map all columns if the same name)
hit Start and the data will flow very quickly
you have create DbInstaller. using dbInstaller you have share whole database. Dbinstaller work both ado.Net and Entity Frame Work but I have using Entity Frame Work.
you can do it by sql server query
first select first database like
Use database1 --- this will be your first database
after select first database we will put our table row in temp table by this query
select * into #Temp from select * from table1
now we select second database and insert temp table data into second database table by this code
use secondDatabaseName
INSERT INTO tableNameintoinsert (col1, col2, )
SELECT col1, col2 FROM #temp;
In sql server, From my desktop I connected to the server. And I want to move a data from a database to another. I have used both select into and import wizard. But import wizard seems to be slow. Why?
Is there any methodology changes for transferring data ?
Select into is a SQL query, and it is executed directly.
Import and Export Wizard is a tool which invokes Integration Services (SSIS).
Wizard is slow, but can use various data sources
More about export/import wizard
https://msdn.microsoft.com/en-US/en-en/library/ms141209.aspx
Topic about select into and export/import wizard
https://social.msdn.microsoft.com/forums/sqlserver/en-US/e0524b2a-0ea4-43e7-b74a-e9c7302e34e0/super-slow-performance-while-using-import-export-wizard
I agree with Andrey. The Wizard is super slow. If you perform a Google search on "sql server import and export wizard slow", you will receive nearly 50k hits. You may want to consider a couple of other options.
BCP Utility
Note: I have used this on a number occasions. Very fast processing.
The bcp utility bulk copies data between an instance of Microsoft SQL Server and a data file in a user-specified format. The bcp utility can be used to import large numbers of new rows into SQL Server tables or to export data out of tables into data files. Except when used with the queryout option, the utility requires no knowledge of Transact-SQL. To import data into a table, you must either use a format file created for that table or understand the structure of the table and the types of data that are valid for its columns.
Example:
BULK INSERT TestServer.dbo.EmployeeAddresses
FROM 'D:\Users\Addresses.txt';
GO
OPENROWSET(BULK) Function
The OPENROWSET(BULK) function connects to an OLE DB data source to restore data and it allows access to a remote data by connecting to a remote data source.
Example:
INSERT INTO AllAddress(Address)
SELECT * FROM OPENROWSET(
BULK 'D:\Users\Addresses.txt',
SINGLE_BLOB) AS x;
Reference
https://msdn.microsoft.com/en-us/library/ms175915.aspx
http://solutioncenter.apexsql.com/sql-server-bulk-copy-and-bulk-import-and-export-techniques/
Mysql Store data into many places and it stores data in Small chunk of files for faster retrieve and when we use export wizard what it does is write all metadata and data to our RAM first and depending on our system and increases overhead and same happen in case of importing, and Select into is fast because mysql has to create inbuilt replica of the database that already exist.
in real life, Select into is like photocopy of a page whereas wizard is like re-writing the page manually.
Please Let me know if any best approach available for these two SSIS Activities.
migrating access data to SQL Server table
multiple Access dbs are used as a source to migrate the data into a SQL table.
SQL server table to SQL server table
Is this a one time job that should it be done periodically?
If one time job, then you can use SQL server Import Data utility to accomplish both things easily.
If it is just copy of data from source to destination without any modification to data, then there is no need to use SSIS.
Use the SQL server Import and export wizard, at the end don't click on run now, and save the package , you can schedule these packages to do the same for you,
Yes you can copy from access to SQL table, and from SQL table to SQL table.
http://msdn.microsoft.com/en-us/library/ms140052%28v=sql.105%29.aspx
How do I copy data from multiple tables within one database to another database residing on a different server?
Is this possible through a BTEQ Script in Teradata?
If so, provide a sample.
If not, are there other options to do this other than using a flat-file?
This is not possible using BTEQ since you have mentioned both the databases are residing in different servers.
There are two solutions for this.
Arcmain - You need to use Arcmain Backup first, which creates files containing data from your tables. Then you need to use Arcmain restore which restores the data from the files
TPT - Teradata Parallel Transporter. This is a very advanced tool. This does not create any files like Arcmain. It directly moves the data between two teradata servers.(Wikipedia)
If I am understanding your question, you want to move a set of tables from one DB to another.
You can use the following syntax in a BTEQ Script to copy the tables and data:
CREATE TABLE <NewDB>.<NewTable> AS <OldDB>.<OldTable> WITH DATA AND STATS;
Or just the table structures:
CREATE TABLE <NewDB>.<NewTable> AS <OldDB>.<OldTable> WITH NO DATA AND NO STATS;
If you get real savvy you can create a BTEQ script that dynamically builds the above statement in a SELECT statement, exports the results, then in turn runs the newly exported file all within a single BTEQ script.
There are a bunch of other options that you can do with CREATE TABLE <...> AS <...>;. You would be best served reviewing the Teradata Manuals for more details.
There are a few more options which will allow you to copy from one table to another.
Possibly the simplest way would be to write a smallish program which uses one of their communication layers (ODBC, .NET Data Provider, JDBC, cli, etc.) and use that to take a select statement and an insert statement. This would require some work, but it would have less overhead than trying to learn how to write TPT scripts. You would not need any 'DBA' permissions to write your own.
Teradata also sells other applications which hides the complexity of some of the tools. Teradata Data Mover handles provides an abstraction layer between tools like arcmain and tpt. Access to this tool is most likely restricted to DBA types.
If you want to move data from one server to another server then
We can do this with the flat file.
First we have fetch data from source table to flat file through any utility such as bteq or fastexport.
then we can load this data into target table with the help of mload,fastload or bteq scripts.