I am just starting to use the Enterprise version of SQL Server 2014. I am completely new to Integration Services and need to use it to call stored procedures asynchronously.
Is using SSIS different from using SQL Server Agent? I am aware of SQL jobs and agents to execute stored procedures asynchronously. Is there any other way to do it using SSIS?
What I really mean to ask is Is there any other way to use SSIS for asynchrnous execution besides this?
Also, Can I get guidance on how to start with it as I am completely raw with SSIS.
Yes. Just create your tasks (such as Execute SQL Task which can invoke stored procedures) in parallel. Make sure the arrows don't lead into each other. SSIS will invoke the tasks simultaneously.
Related
At this time I have one server with a few databases. I have to move Transact-SQL jobs to SSIS packages (company's policy, so I have to do it).
The advantage of the T-SQL is that I can identify really fast if we have modifications (msdb.dbo.sysjobsteps). If I moved the scripts to SSIS Execute SQL tasks, it will be really hard.
I read a lot of post about this topic, but I didn't find the answer.
If I put the scripts into stored procedures and use it in Data Flow tasks, will I lose performance? If yes, do you have any idea how do I search quickly in SSIS packages without opening those?
f I put the scripts into stored procedures and use it in Data Flow tasks, will I lose performance?
Shouldn't. And dynamic SQL in stored procedures will be identical to a TSQL job step.
If yes, do you have any idea how do I search quickly in SSIS packages without opening those?
SSIS packages are XML files. And SSIS projects can be easily managed in a source control system with Visual Studio. This really is a better way.
I need to collect data from a SQL Server table, format it, and then put it into a different table.
I have access to SQL Server but cannot setup triggers or scheduled jobs.
I can create tables, stored procedures, views and functions.
What can I setup that will automatically collect the data and insert it into a SQL Server table for me?
I would probably create a stored procedure to do this task.
In the stored procedure you can create a CTE or use temp tables (depending on the task) and do all the data manipulation you require and once done, you can use the SELECT INTO statement to move all the data from the temp table into the SQL Server table you need.
https://www.w3schools.com/sql/sql_select_into.asp
You can then schedule this stored procedure to run at a time desired by you
A database is just a storage container. It doesn't "do" things automatically all by itself. Even if you did have the access to create triggers, something would have to happen to the table to cause the trigger to fire, typically a CRUD operation on the parent table. And something external needs to happen to initiate that CRUD operation.
When you start talking about automating a process, you're talking about the function of a scheduler program. SQL Server has one built in, the SQL Agent, and depending on your needs you may find that it's appropriate to enlist help from whoever in your organization does have access to it. I've worked in a couple of organizations, though, that only used the SQL Agent to schedule maintenance jobs, while data manipulation jobs were scheduled through an outside resource. The most common one I've run across is Control-M, but there are other players in that market. I even ran across one homemade scheduler protocol that was just built in C#.NET that worked great.
Based on the limitations you lay out in your question, and the comments you've made in response to others, it sounds to me like you need to do socialize your challenge within your organization to find out what their routine mechanism is for setting up data transfers. It's unlikely that this is the first time it's come up, unless the company was founded in the last week or two. It will probably require that you set up your code, probably a stored procedure or maybe an SSIS package, and then work with someone else, perhaps a DBA or a Site Operations team or some such, to get that process automated to fire when you need it to, whether through an Agent job or maybe a file listener.
Well you have two major options, SP and SSIS.
Both of them can be scheduled to run at a given time with a simple Job from the SQL Server Agent. Keep in mind that if you are doing this on a separate server you might need to add the source server as a Linked Server so you can access it from the script.
I've done this approach in the past and it has worked great. Note, for security reasons, I am not able to access the remote server's task scheduler, so I go through the SQL Server Agent:
Run a SQL Server Agent on a schedule of your choice
Use the SQL Server Agent to call an SSIS Package
The SSIS Package then calls an executable which can pull the data you want from your original table, evaluate it, and then insert a formatted version of it, one record at a time. Alternatively, you can simply create a C# script within the SSIS package via a Script Task.
I hope this helps. Please let me know if you need more details.
I am trying to migrate a database from a sql server into Azure. This database have 2 rather simple TSQL script that inserts data. Since the SQL Agent does not exist on Azure, I am trying to find an alternative.
I see the Automation thing, but it seems really complex for something as simple as running SQL scripts. Is there any better or at least easier way to do this ?
I was under the impression that there was a scheduller for that for I can't find it.
Thanks
There are several ways to run a scheduled Task/job on the azure sql database for your use case -
If you are comfortable using the existing on-premise sql sever agent you can connect to your azure sql db(using linked servers) and execute jobs the same way we used to on on-premise sql server.
Use Automation Account/Runbooks to create sql jobs. If you see marketplace you can find several examples on azure sql db(backup,restore,indexing jobs..). I guess you already tried it and does not seem a feasible solution to you.
Another not very famous way could be to use the webjobs(under app service web app) to schedule tasks(can use powershell scripts here). The disadvantage of this is you cannot change anything once you create a webjob
As #jayendran suggested Azure functions is definitely an option to achieve this use case.
If some how out of these if you do not have options to work with the sql directly , there is also "Scheduler Job Collection" available in azure to schedule invocation of HTTP endpoints, and the sql operation could be abstracted/implemented in that endpoint. This would be only useful for less heavy sql operations else if the operation takes longer chances are it might time out.
You can use Azure Functions to Run the T-SQL Queries for Schedule use Timely Trigger.
You can use Microsoft Flow (https://flow.microsoft.com) in order to create a programmed flow with an SQL Server connector. Then in the connector you set the SQL Azure server, database name, username and password.
SQL Server connector
There are many options but the ones that you can use to run a T-SQL query daily are these:
SQL Connector options
Execute a SQL Query
Execute stored procedure
You can also edit your connection info in Data --> Connections menu.
Is it possible to have a stored procedure or set of SQL code run periodically by specifying a time-based trigger in SQL Server?
Not directly, but look at SQL Jobs.
A job is a specified series of operations performed sequentially by SQL Server Agent. A job can perform a wide range of activities, including running Transact-SQL scripts, command-line applications, Microsoft ActiveX scripts, Integration Services packages, Analysis Services commands and queries, or Replication tasks. Jobs can run repetitive tasks or those that can be scheduled, and they can automatically notify users of job status by generating alerts, thereby greatly simplifying SQL Server administration.
(emphasis mine)
You can use a SQL Agent job. IF you have a requirement to run on Express editions, that lack SQL Agent, you can use dialog timers and activation.
You can use a SQL job to run any SQL on a schedule. If you are needing to do something a little more dynamic you can control the jobs (creation, scheduling, removing etc) from SQL itself. This provides an immense amount of flexibility.
Some more info on controlling jobs with TSQL here.
Is it possible to call a stored procedure from another stored procedure asynchronously?
Edit: Specifically I'm working with a DB2 database.
Executive summary: Yes, if your database has a message queue service.
You can push a message onto a queue and the queue processor will consume it asynchronously.
Oracle: queues
Sql Server: service broker
DB2: event broker
For "pure" stored procedure languages (PL/Sql or T-Sql) the answer is no, since it works against the fundamental transaction model most databases have.
However, if your database has a queuing mechanism, you can use that to get the same result.
With MS Sql Server 2005, try the Service Broker and/or CLR stored procedures. I don't think there's anything built directly into TSQL.
It sounds like you need to put some scheduled jobs in place with Cron (or windows equiv). You could use the initial stored proc call to set some kind of flag in the DB, which is then checked periodically by a cron job. If you need to have a specific delay before the 2nd job executes, you should be able to do that by having the task scheduled by the cron job.