I have a data dump that I manually initiate and I want to automate things now that they are working well. I have a system that exports data into Excel that I ultimately want to import into a SQL table.
I have a ssis package that I used for the import and saved it for re-use later. I just manually ran it and it works well. Now I would like to have it run either when invoked by a file watcher or schedule or some thing so that all I need to do is over-write the excel file and have it trigger the ssis to run its import.
Any ideas on how to make this happen?
SQL Server does its scheduling with SQL Agent, so try creating a schedule in that to do what you want.
Related
Is there a way to export data from a MS Excel file into a SQL Server table automatically? Maybe this is done using a script of some kind.
If it's not possible to be completely automated, perhaps there's a way to do it using minimal user effort. (For example clicking a button or link)
There is a MS Excel spreadsheet where the data keeps having to be manually exported to SQL Server.
I've done this using Excel to Access before, but not too certain on how to do it using SQL Server (MS).
*MS Office 2013 and MS SQL Server 2012.
The other answers are ok. I just want to suggest an additional alternative.
If it is just 1 specific Excel file that is frequently updated, I would consider using VBA. For example, write some VBA code in Excel that uploads changes to the database when the spreadsheet is saved (or the user presses a button).
The problem with using a scheduled job is that Excel is basically a single user application. If someone has the spreadsheet open or is doing something in it when the scheduled job runs or moves the spreadsheet to a different folder, then the job may fail.
This way you also get the updated data in your database in something close to real time instead of waiting on a job to run. This might take more time and effort to set up though than some of the other answers.
You can use SQL Server Agent to run a scheduled job that imports data from an Excel worksheet into a SQL Server table.
The import is relatively straightforward to do using Integration Services, but if you've not used either of these before you might need to do some reading up on it.
You can do the following:
You need to create an SSIS package and then create a job to run the package.
The easiest way to create the SSIS package is with "Import and Export Data" tool of SQL Server. It has a nice step by step wizard.
You set everything it asks you from the source and the destinations. Until you get here, select the "Save SSIS Package":
Then you only have to create the job to run it :)
I want to import CSV files and invoices from SAP R/3 system into a SQL database. The database will be used for reporting purpose only, please tell me what will be the best possible way, which database to use and anything else that will be relevant to me in this context? and I am novice so please help....Thanks:)
If you are routinely importing CSV files then I recommend getting them comma delimited (or whatever delimiter you choose) and going the route of making an SSIS package with a corresponding SQL Agent Job that runs daily to check for the file and run it if it finds it.
Info on SSIS package creation:
http://smallbusiness.chron.com/import-csv-ssis-46849.html
If this is a one time load then I would recommend just using the import export wizard built into SQL Server.
https://msdn.microsoft.com/en-us/library/ms140052.aspx
Pretty easy to use the import export wizard too. Right click the database > tasks > import data. This will launch the wizard and will walk you through the one time import.
Adding Microsoft's official SSIS guide as well:
https://msdn.microsoft.com/en-us/library/ms169917.aspx
I'm looking for a simple solution (beginner to SQL) to allow the import of data from my .csv file to my SQL DB.
I have a third party program that is updating my .csv file every 30 seconds and I want to put that updating information into my SQL DB. I tried the importing & exporting wizard but it didn't work due to the .csv file being utilized by the other third party program.
Getting the information into the SQL DB doesn't need to be in real time it could just retrieve all the information when opening a saved sql query file.
Thank you!
OPENROWSET is the simplest one if you get that working in your env for CSV file. i have seen lot of issues with what OS, and what version of MS office installed with 32bit or 64 bit.
but bit more work and you will be all set with creating a small SSIS package to import that CSV in to table. execute that SSIS using SQL JOB at desired interval. later if you needed more complex insert/update you can always modify the package.
This is the case of producer consumer problem where one process is writing data and another one is reading it.
Whatever you do you need to setup some kind of lock on this file that process can check if file is available for reading/writing. If import/export wizard had issues with concurrency then probably other processes will also.
Another option is to always create new file to write into and have reader process to always read from the newest one and delete it after processing.
One more thing you’ll have to take care of is reading from same file multiple times. You need some way to mark the records that have already been read so these are not inserted twice.
All of the above is needed if this needs to be a fully automated and unattended process.
If not you can just manually create a copy of CSV file and then use import/export wizard to import the data.
Here is another resource you can check out for importing CSV into SQL Server
http://blog.sqlauthority.com/2008/02/06/sql-server-import-csv-file-into-sql-server-using-bulk-insert-load-comma-delimited-file-into-sql-server/
I have a database on one server that I need to copy to another server. I can do this manually using the Export Data task, which is fine for a one time export, but I would like to speed this up as it is going to be repeated.
The database will always contain the same set of tables, I just need to get a copy of this database with it's tables and their data from one server to another.
I'd like to create some sort of reusable tool that allows you to specify the source and target database servers and then copies this specific database from one to another. Is this possible?
The Export Data task in SQL 2005 and later uses SQL Server Integration Services (SSIS) under the hood. You can save the package you're already using and run it on a schedule or on demand. You can also edit it (once it is saved) using the Business Intelligence Development Studio (BIDS).
At the end of the Export wizard (on the "Save and Run Package" screen), you can tick the "Save SSIS Package" check-box to store the package either within SQL server or on the file system. The file system is probably simpler.
Once you have the package you can execute it from the command line using the dtexec tool, or from a SQL Agent job using an Execute SSIS task.
SSIS is too big a subject to cover in full here - there are decent tutorials within SQL server books online if you need more details - alternatively, as another SO question if you get stuck.
does anyone know really good step by step tutorial to create a package to import tab delimited flat file into a table using SSIS and then schedule it to run as an SQL job at specified time?
Or maybe someone can show me how to do it?
The easiest way to get started with SSIS is, using Management Studio, right click on the database you want to import into, under Tasks, select Import data. This kicks off a wizard which is quite straight forward. At the end of the wizard, you have the option to save the package to the server or to a file. If you save it to a file, you can then create a job and set the step type to be Sql Server Integration Services Package. You can then configure the parts of your package and schedule the job as normal.