Can anyone tell me the best practice to insert data into a SQL Server database periodically? Say once a day. I get data once in a day from an external source such as a .csv file.
Are there any service programming api that SQL Server supports?
Assuming you're not using SQL Express you can create an SSIS package to do the import, or a BULK LOAD or something like that, and then run the import once a day using a SQL Agent job that uses a daily schedule.
You could also create a Windows Scheduled Task. The task could run a script that connects to SQL Server and runs an SSIS Package, a BULK LOAD, etc. It could also execute a custom program that loads the data.
Related
I work for a customer who has his own database management team and I need to deploy my new web application version within a SQL Server 2008 script (I am not in position to execute any actions on their systems). I can't back up myself their database and I'm not sure they'll do it so if I delete all data it's gonna be terrible.
Therefore, I'm looking for a solution to back-up if possible the database, extract the existing datas, execute the new statements of my script, and re-include the datas saved in the database.
It is possible to do this in a SQL server script ?
More generally, how can I safely update schemas and datas of a SQL Server database within a script without losing all data inside ?
PS :
Currently, I'm using in the dev environment the initial database schema and the newest. So, I use Visual Studio 2012 with Data Tools to make Schema comparison and generate the update script.
You start with a copy of the database schema with sample data against which you can develop and test.
Then you write scripts (perhaps with the aid of a tool like SSDT) that updates the schema to be compatible with the new version of your application, retaining the data in the database.
You deliver these tested schema modification scripts along with the new version of your application for the customer's administrators to test and apply to the target database as part of the application upgrade.
I have a package that has a main sql execute task. Within that task I need to call two databases (A,B) each on a different server. There is a linked server(B) within the main server(A) I am using. I can't split them out however due to they join on each other many times on many different tables in separate scripts. The main script is a large amount of unions pulling data in to report off of in a later task.
I have my linked server call in my sql script but it says it can't connect. Is there a way I can connect to this linked server as I do when I run the sql script from sql server? Or is there a way to make a connection within the SSIS package and refer to it somehow within a task or in the script?
Make sure Proxy Account has setup proper Login credential/user with permission to execute your task at main server(A) and able to access linked server(B).
Use Log Viewer at SSMS to see the error message.
See example at following URL:-
https://www.mssqltips.com/sqlservertip/2163/running-a-ssis-package-from-sql-server-agent-using-a-proxy-account/
I have a google doc that I'm using to update a table in a sql table and at the moment I do it manually; copy and paste the data from the google doc. into excel, remove the columns that I don't need then save as a CSV and upload.
I'm wondering if there is a way that I can automate this? The report it's for is daily and the upload is becoming a bit of a bore!
You can also go the python-route by utilising the following python libraries: gspread, oauth2client, pandas and sqlalchemy (source). If you want to automate it, use cron or crontab to run your python-script on a hourly, daily, weekly or monthly basis.
What DBMS do you use?
If you use Microsoft SQL Server, you can use SSIS (SQL Server Integration Services), the easiest way to do this for a novice would be to right-click on the database in SQL Server Management Studio --> Tasks --> Import Data, then follow the wizard to set up the data source (flat file / Excel file etc.) and the data destination (DB table). Once you've finished with the wizard, you can save the package and re-use it later on (or even automate it via SQL Agent job). You can edit the package in Visual Studio (aka SQL Server Data Tools) prior to deployment/automation if needed.
More on SSIS: https://msdn.microsoft.com/en-us/library/ms169917.aspx
I have a database on one server that I need to copy to another server. I can do this manually using the Export Data task, which is fine for a one time export, but I would like to speed this up as it is going to be repeated.
The database will always contain the same set of tables, I just need to get a copy of this database with it's tables and their data from one server to another.
I'd like to create some sort of reusable tool that allows you to specify the source and target database servers and then copies this specific database from one to another. Is this possible?
The Export Data task in SQL 2005 and later uses SQL Server Integration Services (SSIS) under the hood. You can save the package you're already using and run it on a schedule or on demand. You can also edit it (once it is saved) using the Business Intelligence Development Studio (BIDS).
At the end of the Export wizard (on the "Save and Run Package" screen), you can tick the "Save SSIS Package" check-box to store the package either within SQL server or on the file system. The file system is probably simpler.
Once you have the package you can execute it from the command line using the dtexec tool, or from a SQL Agent job using an Execute SSIS task.
SSIS is too big a subject to cover in full here - there are decent tutorials within SQL server books online if you need more details - alternatively, as another SO question if you get stuck.
I have a data dump that I manually initiate and I want to automate things now that they are working well. I have a system that exports data into Excel that I ultimately want to import into a SQL table.
I have a ssis package that I used for the import and saved it for re-use later. I just manually ran it and it works well. Now I would like to have it run either when invoked by a file watcher or schedule or some thing so that all I need to do is over-write the excel file and have it trigger the ssis to run its import.
Any ideas on how to make this happen?
SQL Server does its scheduling with SQL Agent, so try creating a schedule in that to do what you want.