I have an SQL table which is used for logging purpose(There are lakhs of records in the table). I need to purge the table (Take a back up of the data and need to clear the table data).
Is there a standard way of doing it where I can automate it.?
You can do this within SQL Server Management Studio, by:
right clicking Database > Tasks > Generate Script
You can then select the table you wish to script out and also choose to include any associated objects, such as constraints and indexes.
Attaching an image which will give you the step by step procedure,
image_bkp_procedure
PFB the stackoverflow link which will give you more insight on this,
Table-level backup
And your automation requirement,
You can download bcp utility which copies data between an instance of Microsoft SQL Server and a data file in a user-specified format.
Sample syntax to export,
bcp "select * from [MyDatabase].dbo.Customer " queryout "Customer.bcp" -N -S localhost -T -E
You can automate this query by using any scheduling mechanism (UNIX etc)
Simply we can create a job that runs once in a month
--> That backups data in another table like archive table
--> Then deletes data in the main table
Its primitive partitioning I guess, this way it will be more flexible when you need to select data from the past deleted one i.e. now on archive table where you have backed up
Related
I cannot use linked server.
Both databases on both servers have the same structure but different data.
I have 10k rows to transfer from the DB on one server to the same DB on the other. I cannot restore the DB on the other server as it will take a huge amount of space that I don't have on the other server.
So, I have 2 options that I don't know how to execute:
Backup and restoring only one table - the table is linked to many other tables but these other tables exist on the other server too. Can I somehow delete or remove the other tables or make a backup only over one table?
I need to transfer 10k rows. How is it possible to create 10k insert queries based on selected 10k rows?
Can I somehow delete or remove the other tables or make a backup only over one table?
No you can not do this, unfortunately
How is it possible to create 10k insert queries based on selected 10k rows?
Right-click on Database -> Tasks -> Generate scripts -> (Introduction) Next
Chose Select specific database objects -> Tables, chose table you need -> Next
Advanced -> Search for Types of data script change from Schema only (by default) to Data only -> OK
Chose where to save -> Next -> Next. Wait the operation to end.
This will generate the file with 10k inserts.
Another way is to use Import/Export wizard (the most simple way for one-time-import/export) if you have link between databases.
There are many ways to choose from, here is one way using BCP. That's a tool that ships with SQL Server to Import and Export Bulk Data.
The outlines of the process:
Export the data from the source server to a file using BCP - BCP OUT for a whole table, or BCP QUERYOUT with a query to select the 10k rows you want exported
Copy the file to the destination server
Import the data using BCP on the destination database - BCP IN.
My suggestion would be to export these rows to excel( you can do this by copy pasting your query output) and transfer this to other server and import it there.
this is the official method :-
https://learn.microsoft.com/en-us/sql/relational-databases/import-export/import-data-from-excel-to-sql
and this is the the unofficial method :
http://therealdanvega.com/blog/2010/08/04/create-sql-insert-statements-from-a-spreadsheet.
Here I have assumed that you only need to transfer the transactional data and your reference data is same on both server. So you will need to execute only one query for exporting your data
I would definietely go down the SSIS route once you use SSIS to do a task like this you will not use anything else very simple to script up. You can use any version and it will be a simple job and very quick.
Open new SSIS project in available visual studio version/s there are many different but even a 2008 version will do this simple task you may have to install integration services or something similar used to be called bids (business information development studio in 2008) (anything up to 2015 although support is nearly there in 2017)
add a data flow task
double click the data flow task
Bottom of screen add two connection managers (1 to source and 1 to destination database)
add oledb source pointing to source database table
add oledb destination pointing to destination database table
drag line between the source and destination (should auto map all columns if the same name)
hit Start and the data will flow very quickly
you have create DbInstaller. using dbInstaller you have share whole database. Dbinstaller work both ado.Net and Entity Frame Work but I have using Entity Frame Work.
you can do it by sql server query
first select first database like
Use database1 --- this will be your first database
after select first database we will put our table row in temp table by this query
select * into #Temp from select * from table1
now we select second database and insert temp table data into second database table by this code
use secondDatabaseName
INSERT INTO tableNameintoinsert (col1, col2, )
SELECT col1, col2 FROM #temp;
I want to get a backup of a single table with its data from a database in SQL Server using a script.
How can I do that?
SELECT * INTO mytable_backup FROM mytable
This makes a copy of table mytable, and every row in it, called mytable_backup. It will not copy any indices, constraints, etc., just the structure and data.
Note that this will not work if you have an existing table named mytable_backup, so if you want to use this code regularly (for example, to backup daily or monthly), you'll need to run drop mytable_backup first.
You can use the "Generate script for database objects" feature on SSMS.
Right click on the target database
Select Tasks > Generate Scripts
Choose desired table or specific object
Hit the Advanced button
Under General, choose value on the Types of data to script. You can select Data only, Schema only, and Schema and data. Schema and data includes both table creation and actual data on the generated script.
Click Next until wizard is done
There are many ways you can take back of table.
BCP (BULK COPY PROGRAM)
Generate Table Script with data
Make a copy of table using SELECT INTO, example here
SAVE Table Data Directly in a Flat file
Export Data using SSIS to any destination
You can create table script along with its data using following steps:
Right click on the database.
Select Tasks > Generate scripts ...
Click next.
Click next.
In Table/View Options, set Script Data to True; then click next.
Select the Tables checkbox and click next.
Select your table name and click next.
Click next until the wizard is done.
For more information, see Eric Johnson's blog.
Put the table in its own filegroup. You can then use regular SQL Server built in backup to backup the filegroup in which in effect backs up the table.
To backup a filegroup see:
https://learn.microsoft.com/en-us/sql/relational-databases/backup-restore/back-up-files-and-filegroups-sql-server
To create a table on a non-default filegroup (its easy) see:
Create a table on a filegroup other than the default
Another approach you can take if you need to back up a single table out of multiple tables in a database is:
Generate script of specific table(s) from a database (Right-click database, click Task > Generate Scripts...
Run the script in the query editor. You must change/add the first line (USE DatabaseName) in the script to a new database, to avoid getting the "Database already exists" error.
Right-click on the newly created database, and click on Task > Back Up...
The backup will contain the selected table(s) from the original database.
To get a copy in a file on the local file-system, this rickety utility from the Windows start button menu worked:
"C:\Program Files (x86)\Microsoft SQL Server\110\DTS\Binn\DTSWizard.exe"
I have developed a SQL Express database. I need to backup all but one table in that database in an automated way. I was thinking i could write a SQL script to do this, trigger it using sqlcmd from a batch file but not sure how to write that SQL script.
I was also thinking, if nothing else possible, i could create a second db that has the tables i want to backup then i write a script that copys data 'into' the second db and then do a auto backup of that entire db. This has the disadvantage of having a procrastinated unpacking of that backup when wanting to use it - its not a small install script.
Is this a possibility, is it the only option or is there tools for SQL Express to do this?
There is no option to exclude just one table while backing up .Few things i could think of
1.Right click database ->Tasks ->generate scripts ->exclude the table you want and choose to save the script and run this every time
2.you could also choose Export option,but since you are using SQL Express,you wont have the option to save this package
Keep the large table in a different database and just backup the original database. You can still use the large table even in a different database, i.e.
SELECT *
FROM MyDb.dbo.SomeTable s
JOIN OtherDb.dbo.LargeTable l
ON (expression);
How do I copy data from multiple tables within one database to another database residing on a different server?
Is this possible through a BTEQ Script in Teradata?
If so, provide a sample.
If not, are there other options to do this other than using a flat-file?
This is not possible using BTEQ since you have mentioned both the databases are residing in different servers.
There are two solutions for this.
Arcmain - You need to use Arcmain Backup first, which creates files containing data from your tables. Then you need to use Arcmain restore which restores the data from the files
TPT - Teradata Parallel Transporter. This is a very advanced tool. This does not create any files like Arcmain. It directly moves the data between two teradata servers.(Wikipedia)
If I am understanding your question, you want to move a set of tables from one DB to another.
You can use the following syntax in a BTEQ Script to copy the tables and data:
CREATE TABLE <NewDB>.<NewTable> AS <OldDB>.<OldTable> WITH DATA AND STATS;
Or just the table structures:
CREATE TABLE <NewDB>.<NewTable> AS <OldDB>.<OldTable> WITH NO DATA AND NO STATS;
If you get real savvy you can create a BTEQ script that dynamically builds the above statement in a SELECT statement, exports the results, then in turn runs the newly exported file all within a single BTEQ script.
There are a bunch of other options that you can do with CREATE TABLE <...> AS <...>;. You would be best served reviewing the Teradata Manuals for more details.
There are a few more options which will allow you to copy from one table to another.
Possibly the simplest way would be to write a smallish program which uses one of their communication layers (ODBC, .NET Data Provider, JDBC, cli, etc.) and use that to take a select statement and an insert statement. This would require some work, but it would have less overhead than trying to learn how to write TPT scripts. You would not need any 'DBA' permissions to write your own.
Teradata also sells other applications which hides the complexity of some of the tools. Teradata Data Mover handles provides an abstraction layer between tools like arcmain and tpt. Access to this tool is most likely restricted to DBA types.
If you want to move data from one server to another server then
We can do this with the flat file.
First we have fetch data from source table to flat file through any utility such as bteq or fastexport.
then we can load this data into target table with the help of mload,fastload or bteq scripts.
We have a partitioned view with 12 member tables. The partitioning column is a date, we have one table for each month. Data are continually being inserted into the table of the current month. Older tables are constant.
At the beginning of each next month a new table is going to be created and added to the view. At the same time we are going to remove the oldest table from the database but we have to preserve the data somehow because in the future we might have to reinsert those data into the database temporarily for analysis and comparison. For example in june 2011 when the partitioned view will have member tables from july 2010 to june 2011 we might have to reinsert the data of june 2009 and june 2010 temporarily.
My question is: How would you do this 'preserve' and then the 'reinsert' operation? Is there a recommended way or a well-known pattern for this?
We are using SQL Server 2005 Standard Edition. I'm a novice in database administration.
(EDIT: Why might someone use partitioning and a new table for each month? See SQL Server 2000 Partitioned Views and SQL Server 2005 Partitioned Tables and Indexes)
Probably not the answer you want, but I'll say it anyways - this is typically what you would use a data warehouse for. Shuffle those old months off to the data warehouse and do your reporting & analysis there.
with data storage being so cheap these days why not keep the all the data in one table filtering it by date. Put an index on that date to help with performance as the table size grows.
If that is not an option I would just create a back up of the table that you can restore from later or create insert statements from the data you can save to a sql file and run at any time.
Well. I don't know if I understand 100% what you mean. But as I understand it, you have one table for each month of data, then use a view to merge several tables together (using unions I guess ?). If that's the case I wouldn't do anything except keep creating new tables for each month, and modify the view to remove old tables like you said. Then to "reinsert" old data I would just create a new view that includes all tables or modify the current if feasible. There should be no reason to delete the old data from the database.
Why not use the bulk copy tool (bcp) to export the contents of your table to a file.
You can then zip it up and keep it somewhere safe until you need to use the bcp tool again to recreate the table.
To export
bcp yourdb..Stuff200901 OUT Stuff200901.bcp -T -S yourdbsvr -r "\n" -t "|" -c
To import (note you'll need to create the table)
bcp yourdb..Stuff200901 in Stuff200901.bcp -c -S yourdbsvr -T -t "|" -k -e err.txt
Type bcp to get a full list of options at the command prompt.