i used management studio to import data from Excel to a SQL table, and on the final step saved the ssis package to the database.
now i want to re-run the saved package on a new excel file i get as a data dump from another system.
the excel file is identical and only the data changes.
how can i re-run the saved ssis package on the new file?
i am just over-writing the old excel file with the new one and expect the ssis job to just pick up everything and run as it did the first time.
You should be able to run DTExecUI from the command prompt. From here select the server you want to use and the package.
Related
I am just starting on SSIS and created a simple package to start off with. I ran the import/export wizard from SSDT where I am copying table contents from a table to an excel spreadsheet. I eventually get the table contents on the spreadsheet but I have to run the Execute SQL task first, then run the Data Flow task second manually. Not sure why the data Flow task does not pick up after the Execute SQL task completes. The connection is set to run on success. I googled the heck out of this and cannot find an answer. I am running Visual Studio 2013 Shell (Integrated) version 12.0.21005. Any answer (or clue) would be greatly appreciated! Thank you!
I have a few SQL queries to run in Microsoft SQL Server Management Studio 2018 to export data as .txt files. The .txt files are then imported to ElasticSearch via Logstash. At the moment, I'm running these queries manually via a many clicks operation:
Right click on the database
Click Tasks > Export Data
Choose my data source from SQL Server Native Client 11.0
Choose my destination as Flat File Destination.
Type my desired filename to store in the output folder.
Choose to Write a query to specify the data to transfer Copy and Paste my SQL Statement Finally a few next buttons and run it.
My question is, is there an API or a way to automatically run these queries periodically (once a day at 12 pm), and store it in a particular folder?
Thanks
You can save the Export defintion as an SSIS package
If you're running on your PC then you can run SSIS from a scheduled a task using DTExec
If you're running it on the SQL server than you can configure a SQL job
You can try one of the following:
SQLCMD command line SQL tool to send your query output to text file using -o switch. it can be scheduled using SQL Server Agent Job
Or you can use the SQL Job Step Advanced tab to specify an output file to receive the out from your job step.
I'm looking for a simple solution (beginner to SQL) to allow the import of data from my .csv file to my SQL DB.
I have a third party program that is updating my .csv file every 30 seconds and I want to put that updating information into my SQL DB. I tried the importing & exporting wizard but it didn't work due to the .csv file being utilized by the other third party program.
Getting the information into the SQL DB doesn't need to be in real time it could just retrieve all the information when opening a saved sql query file.
Thank you!
OPENROWSET is the simplest one if you get that working in your env for CSV file. i have seen lot of issues with what OS, and what version of MS office installed with 32bit or 64 bit.
but bit more work and you will be all set with creating a small SSIS package to import that CSV in to table. execute that SSIS using SQL JOB at desired interval. later if you needed more complex insert/update you can always modify the package.
This is the case of producer consumer problem where one process is writing data and another one is reading it.
Whatever you do you need to setup some kind of lock on this file that process can check if file is available for reading/writing. If import/export wizard had issues with concurrency then probably other processes will also.
Another option is to always create new file to write into and have reader process to always read from the newest one and delete it after processing.
One more thing you’ll have to take care of is reading from same file multiple times. You need some way to mark the records that have already been read so these are not inserted twice.
All of the above is needed if this needs to be a fully automated and unattended process.
If not you can just manually create a copy of CSV file and then use import/export wizard to import the data.
Here is another resource you can check out for importing CSV into SQL Server
http://blog.sqlauthority.com/2008/02/06/sql-server-import-csv-file-into-sql-server-using-bulk-insert-load-comma-delimited-file-into-sql-server/
I have a database on one server that I need to copy to another server. I can do this manually using the Export Data task, which is fine for a one time export, but I would like to speed this up as it is going to be repeated.
The database will always contain the same set of tables, I just need to get a copy of this database with it's tables and their data from one server to another.
I'd like to create some sort of reusable tool that allows you to specify the source and target database servers and then copies this specific database from one to another. Is this possible?
The Export Data task in SQL 2005 and later uses SQL Server Integration Services (SSIS) under the hood. You can save the package you're already using and run it on a schedule or on demand. You can also edit it (once it is saved) using the Business Intelligence Development Studio (BIDS).
At the end of the Export wizard (on the "Save and Run Package" screen), you can tick the "Save SSIS Package" check-box to store the package either within SQL server or on the file system. The file system is probably simpler.
Once you have the package you can execute it from the command line using the dtexec tool, or from a SQL Agent job using an Execute SSIS task.
SSIS is too big a subject to cover in full here - there are decent tutorials within SQL server books online if you need more details - alternatively, as another SO question if you get stuck.
I have a data dump that I manually initiate and I want to automate things now that they are working well. I have a system that exports data into Excel that I ultimately want to import into a SQL table.
I have a ssis package that I used for the import and saved it for re-use later. I just manually ran it and it works well. Now I would like to have it run either when invoked by a file watcher or schedule or some thing so that all I need to do is over-write the excel file and have it trigger the ssis to run its import.
Any ideas on how to make this happen?
SQL Server does its scheduling with SQL Agent, so try creating a schedule in that to do what you want.