Afternoon,
Is it possible to execute a SSIS package using MVC? What I am creating is a web application and it would have a button, once clicked the SSIS package runs.
The SSIS project is already set up and deployed on the MSSQL server.
SQL Server 2012+ offers a fantastic mechanism for managing packages and their execution via the Integration Services catalog, SSISDB.
The following code provides an example of running the package (Package2.dtsx) from the MyProjectName SSIS project living under the Demo folder with an IntensityLevel package parameter turned up to 11.
DECLARE #execution_id bigint;
EXEC SSISDB.catalog.create_execution
#package_name = N'Package2.dtsx'
, #execution_id = #execution_id OUTPUT
, #folder_name = N'Demo'
, #project_name = N'MyProjectName'
, #use32bitruntime = False
, #reference_id = NULL;
DECLARE #var0 int = 11;
EXEC SSISDB.catalog.set_execution_parameter_value
#execution_id
, #object_type = 30
, #parameter_name = N'IntensityLevel'
, #parameter_value = #var0;
DECLARE #var1 smallint = 1;
EXEC SSISDB.catalog.set_execution_parameter_value
#execution_id
, #object_type = 50
, #parameter_name = N'LOGGING_LEVEL'
, #parameter_value = #var1;
EXEC SSISDB.catalog.start_execution
#execution_id;
The easy way to get a sample of how the above SQL should be built out is to open SQL Server Management Studio (SSMS) and configure a run of the SSIS package. Navigate to the Integration Services Catalog and find the package you want to run. Right click and select Execute...
The Configuration menu opens up and find the parameter(s) you want to specify. Provide a sample value but DO NOT CLICK OK. Instead, click that Script button and specify script to new window (or clipboard)
Now you have the exact commands that SSMS would have issued to run your package. Take that code, use your parameterization method of choice for having your MVC program stubbing in the correct runtime value for your parameter and then wrap all of that TSQL up in a simple database call (ole, ado, odbc it won't matter)
You could just create a stored procedure which you can call from your MVC app with SQLClient or so..
In this stored procedure you can then launch the SSIS package.(see below link for more detailed description on how to do this)
https://www.mssqltips.com/sqlservertip/2992/how-to-execute-an-integration-services-ssis-package-from-a-sql-server-stored-procedure/
I created a stored procedure which runs the SSIS package. Within MVC it calls the stored procedure -
[HttpPost]
[ValidateAntiForgeryToken]
public async Task<IActionResult> Index()
{
//code that updates DB
#region Run Stored Procedure
//connect to the SQL server
var connection = new SqlConnection(_configuration.GetConnectionString("DatabaseConnection"));
//command that runs procedure on the SQL server
var command = new SqlCommand("RebuildSelection", connection)
{
CommandType = CommandType.StoredProcedure,
CommandText = "RebuildSelection"
};
//get text from stored procedure to show success/error messages
SqlParameter text = new SqlParameter("#Text", SqlDbType.NVarChar, 1000)
{
//output as its displayed to the user
Direction = ParameterDirection.Output
};
//add the params
command.Parameters.Add(text);
connection.Open();
//run query
command.ExecuteNonQuery();
//used to return success/error messages to user
ViewData["Message"] = text.Value;
connection.Close();
#endregion
return View();
}
This link was a big help for creating the procedure.
This one helped in returning message to the user.
C# can be used to execute a package deployed to SSISDB as follows. The initial connection created in the SqlConnection object will be the server where the package is deployed to. In the example below, the ObjectType of 30 is used for a package parameter. This can be changed to 20 for a parameter at the project level.
using System.Data.SqlClient;
using Microsoft.SqlServer.Management.IntegrationServices;
using System.Collections.ObjectModel;
string folder = "Folder of Package";
string project = "Project Of Package";
string packageName = "PackageName.dtsx";
string packageParameter = "ParameterValue";
// server where package is deployed
SqlConnection connString = new SqlConnection(#"Data Source=ServerWherePackageDeployed;Initial Catalog=SSISDB;Integrated Security=SSPI;");
IntegrationServices ssisConnString = new IntegrationServices(connString);
//create PackageInfo object for package to execute
Microsoft.SqlServer.Management.IntegrationServices.PackageInfo package = ssisConnString.Catalogs["SSISDB"].Folders[folder].Projects[project].Packages[packageName];
Collection <Microsoft.SqlServer.Management.IntegrationServices.PackageInfo.ExecutionValueParameterSet> parameterList = new Collection<Microsoft.SqlServer.Management.IntegrationServices.PackageInfo.ExecutionValueParameterSet>();
//set logging level to basic
parameterList.Add(new Microsoft.SqlServer.Management.IntegrationServices.PackageInfo.ExecutionValueParameterSet { ObjectType = 50, ParameterName = "LOGGING_LEVEL", ParameterValue = 1 });
parameterList.Add(new Microsoft.SqlServer.Management.IntegrationServices.PackageInfo.ExecutionValueParameterSet { ObjectType = 30, ParameterName = "ParameterName", ParameterValue = packageParameter });
package.Execute(false, null, parameterList);
Related
how does one programmatically create a localdb .mdf?
acceptable solutions exclude visual studio, ssms, aspnet_regsql.
a naive stab at a solution might look like this:
static void Main(string[] args)
{
using (var con = new SqlConnection(#"Integrated Security=SSPI;Data Source=(LocalDb)\v11.0;AttachDbFilename=test.mdf"))
{
con.Open();
using (var cmd = new SqlCommand("CREATE DATABASE test", con))
{
cmd.CommandType = CommandType.Text;
cmd.ExecuteNonQuery();
}
}
}
but of course, this fails in SqlConnection.Open with the error
An attempt to attach an auto-named database for file test.mdf failed. A database with the same name exists, or specified file cannot be opened, or it is located on UNC share.
You cannot connect to a database if the specified .mdf doesn't exist.
So... how do you create one?
Had to piece together several answers from Stackoverflow and the great Getting Started with SQL Server 2012 Express LocalDB article from #AaronBertrand
Code assumes Dapper.NET is installed:
PM> Install-Package Dapper
Programmatic creation:
var dbServerName = "SERVER_NAME";
var dbName = "DATABASE_NAME";
var infoResult = SqlLocalDbCommand($"info {dbServerName}");
var needsCreated = infoResult?.Trim().EndsWith($"\"{dbServerName}\" doesn't exist!");
if (needsCreated.GetValueOrDefault(false))
{
var createResult = SqlLocalDbCommand($"create {dbServerName} -s");
var success = createResult?.Trim().EndsWith($"\"{dbServerName}\" started.");
if (false == success)
{
var msg = $"Failed to create database:{Environment.NewLine}{createResult}"
throw new ApplicationException(msg);
}
var master = $#"Server=(localdb)\{dbServerName};Integrated Security=True;"
using (var conn = new SqlConnection(master))
{
var result = conn.Execute($"CREATE DATABASE {dbName}");
}
var #new = $#"Server=(localdb)\{dbServerName};Integrated Security=True;Database={dbName}"
using (var conn = new SqlConnection(#new))
{
//verify i can access my new database
var tables = conn.Query($"SELECT * FROM {dbName}.INFORMATION_SCHEMA.Tables");
}
}
Helper (thanks T30):
/// <summary>
/// Executes a command against SqlLocalDB
/// </summary>
/// <remarks></remarks>
/// <param name="arguments">The arguments to pass to SqlLocalDB.exe</param>
/// <returns></returns>
/// <exception cref="System.ApplicationException">Error returned from process</exception>
private static string SqlLocalDbCommand(string arguments)
{
var process = new Process
{
StartInfo =
{
FileName = "SqlLocalDB",
Arguments = arguments,
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardError = true
}
};
process.Start();
//* Read the output (or the error)
var output = process.StandardOutput.ReadToEnd();
Console.WriteLine(output);
var err = process.StandardError.ReadToEnd();
Console.WriteLine(err);
process.WaitForExit();
if (err.Exists()) throw new ApplicationException(err); //Is LocalDB installed?
return output;
}
Note that with this solution you won't see the mdf files, i'm sure they exist in some user folder but the key take away is that you'll connect by the connection string
(localdb)\SERVER_NAME;Integrated Security=True;Database=DATABASE_NAME
So I take it what you actually want to do is create a database called test in your LocalDB instance, but you don't have an MDF file already created for this database?
If that's the case, the code you have will fail at the connection phase since you've asked it to attach your test.mdf file.
What you would normally do in this situation is make a connection to the master database initially, and then run the create database statement which will create the test database with it's associated MDF file, maybe try changing your connection string so it looks more like this and then running again:
Integrated Security=SSPI;Data Source=(localdb)\V11.0;Initial Catalog=master
I Have a database that we want to partially sync data out of into another database (on Azure).
I have been looking at Sync Framework 2.1 and believe it can solve the problem, however i cannot figure it out from the online documentation.
We have the restraint that we cannot change the schema of the database however we are on SQL 2008 R2 which means that we can use track changes.
I am looking for some advise on how this might be achieved.
currently i have a SyncOrchestrator
var orch = new SyncOrchestrator
{
LocalProvider = new SampleServerSyncProvider(),
RemoteProvider = new SampleClientSymcProvider(),
Direction = SyncDirectionOrder.Upload
};
and then a sync provider
public class SampleServerSyncProvider : DbServerSyncProvider
{
private String SQLLocalConnection = "valid connection string";
public SampleServerSyncProvider()
{
SqlConnection serverConn = new SqlConnection(SQLLocalConnection);
Connection = serverConn;
Connection.Open();
var cmTableSyncAdapter = new SqlSyncAdapterBuilder
{
Connection = serverConn,
ChangeTrackingType = ChangeTrackingType.SqlServerChangeTracking,
SyncDirection = SyncDirection.Bidirectional,
TableName = "my table"
};
SyncAdapters.Add(cmTableSyncAdapter.ToSyncAdapter());
}
}
Currently i am getting an error that talks about initializing the connection. But I cannot find an initialize method on any of the objects
System.InvalidOperationException : Cannot create a SyncAdapter for table 'My table' by using
SqlSyncAdapterBuilder because the connection to the server has not yet
been initialized. Initialize the Connection property of the
SqlSyncAdapterBuilder before you call any of the SqlSyncAdapterBuilder
methods
SQL Change Tracking is only supported on the older offline providers (SqlClientSyncProvider/DbServerSyncProvider/SyncAgent). The newer providers you're trying to use (SqlSyncProvider/SyncOrchestrator) requires a custom change tracking. You cannot mix and match the database sync providers.
have you looked at using SSIS instead?
I am first time trying to use filestream to store pdf files on file system using varbinary(MAX) column type of DB.
I have followed following steps.
enabled filestream feature on SQL server 2008 R2.
Create a filegroup for BLOB storage
created table with blob column of type varbinary(max)
Now, I want to use file upload control to select file and when click on upload button it should save the pdf file. Also, how to retrieve the file?
I have tried following code
protected void btnFSupload_Click(object sender, EventArgs e)
{
SqlConnection cn = null;
SqlTransaction tx = null;
SqlCommand cmd = null;
SqlCommand cmd2 = null;
bool bCommit = false;
try
{
// read in the file to be saved as a blob in the database
FileStream input = new FileStream(#"D:\swami.pdf", FileMode.Open, FileAccess.Read);
byte[] buffer = new byte[(int)input.Length];
input.Read(buffer, 0, buffer.Length);
cn = new SqlConnection("server=at-hetal01\\sqlexpress;Initial Catalog=practice;Integrated Security=true;");
cn.Open();
tx = cn.BeginTransaction();
cmd = new SqlCommand("dbo.stp_AddBLOB", cn, tx);
cmd.CommandType = System.Data.CommandType.StoredProcedure;
SqlDataReader r = cmd.ExecuteReader(System.Data.CommandBehavior.SingleRow);
r.Read();
string id = r[0].ToString();
string path = r[1].ToString();
r.Close();
// get the transaction context
cmd2 = new SqlCommand("SELECT GET_FILESTREAM_TRANSACTION_CONTEXT()", cn, tx);
Object obj = cmd2.ExecuteScalar();
byte[] txCtx = (byte[])obj;
// open the filestream to the blob
SafeFileHandle handle = OpenSqlFilestream(path,DESIRED_ACCESS_WRITE,SQL_FILESTREAM_OPEN_NO_FLAGS,txCtx,(UInt32)txCtx.Length,0);
// open a Filestream to write the blob
FileStream output = new FileStream(handle,FileAccess.Write,buffer.Length,false);
output.Write(buffer,0,buffer.Length);
output.Close();
if (handle != null && !handle.IsClosed)
handle.Close();
bCommit = true;
}
catch (Exception ex)
{
Response.Write(ex.Message);
}
finally
{
if (cn != null)
{
switch (bCommit)
{
case true:
tx.Commit();
break;
case false:
tx.Rollback();
break;
}
cn.Close();
}
}
}
Above code shows error as below
The operating system returned the error '0xc000003a({Path Not Found} The path %hs does not exist.)' while attempting 'NtCreateFile' on 'D:\DB\FS\d11132f8-c2a8-452d-ae0c-208164a550d7\beb8e1f1-8116-440b-870b-7cef4281a15d\0000001c-000000e4-010d'. The statement has been terminated.
So, any clue on this?
If you have altered your table using SSMS table designer, the FILESTEAM column attribute will be lost producing the path not found. Make sure the FILESTREAM attribute is set for the file field by running the follwoing statement in your database:
select SERVERPROPERTY('FilestreamShareName') as ShareName,
SERVERPROPERTY('FilestreamConfiguredLevel') as ConfiguredLevel,
SERVERPROPERTY('FilestreamEffectiveLevel') as EffectiveLevel
You'll need to alter the table via a script and NOT SSMS to tie your varchar(max)/filestream field to the FileGroup you should have already created.
When I ran into this issue, I found the answer on StackOverflow, but can't seem to find it again for the reference.
I know this is old, but for future reference:
We checked the SERVERPROPERTY values that #BMP suggested. They were configured correctly, so that didn't help.
However, we went ahead and turned OFF the windows file share part of the file streaming access. Once this was done, the error went away.
In our case it was a web app running on the exact same machine as the sql server which exhibited the problem. I'm not sure if the web app's app pool user didn't have access to the file share created by windows or not.
The details were:
Windows 2003 Server (x86)
IIS 6
SQL Server 2008 R2 Express
UPDATE: Apparently this worked for a few days. It's not working any more.
I want to be able to have a sql query in my DTSX package and I want to be able to have some sort of prompt to update the value of a null column. See what I have below:
UPDATE SF1411
SET [QuoteNumber] = '123456'
, [ItemNumber] = '123654-100'
, [DeleteItem] = 'NO'
WHERE [QuoteNumber] = '0'
I want to be able to be prompted for the QuoteNumber and ItemNumber, then have the script update as needed. Is this possible and if so how can I do it?
This can be acheived as below: This will be in your intial script component.
System.Windows.Forms.Form frm = new Form();
TextBox txt = new TextBox();
Button inputset = new Button();
public void Main()
{
inputset.Text = "Set Variable Value";
inputset.Width = 200;
inputset.Height = 100;
inputset.Click += new EventHandler(inputset_Click);
txt.Name = "Input";
frm.Controls.Add(txt);
frm.Controls.Add(inputset);
frm.ShowDialog();
MessageBox.Show(Dts.Variables["Value1"].Value.ToString());
Dts.TaskResult = (int)ScriptResults.Success;
}
void inputset_Click(object sender, EventArgs e)
{
Dts.Variables["Value1"].Value = Convert.ToInt32(txt.Text);
frm.Close();
}
This should be the initial component in your package to set the variable value or construct you SQL Command.
In general, an SSIS package is not used interactively. Your cleanest solution is a custom solution that gets the input from the user, and then launches the SSIS package.
A simpler alternative is using Package Configurations. You can store the user input in an external location (XML file, SQL Server database, and others) and the SSIS package will load the value at run time.
How is it possible to execute a direct SQL command to an ADO.NET connected database?
I want to send a DBCC SHRINKDATABASE to the SQL server, to compress the current datafile after a big deletion process. The function ObjectContext::CreateQuery returns a parser error after the DBCC command. Is there any other way to shrink the database file, or another way to send the SQL command directly to the SQL Server?
I'd just send this as raw SQL:
using (SqlCommand command = new SqlCommand("DBCC SHRINKDATABASE(0)", connection))
{
command.ExecuteNonQuery();
}
Another way is to put the DBCC SHRINKDATABASE in a stored procedure and call the stored procedure from your code.
thanks Mark, I just had to figure out how I can obtain the SqlConnection.
Here is the complete sourcecode (ok could be written a little bit nicer with an extension method):
public ActionResult Optimize()
{
using (BXPartsEntities Entities = new BXPartsEntities())
{
System.Data.EntityClient.EntityConnection eConnection = Entities.Connection as System.Data.EntityClient.EntityConnection;
eConnection.Open();
var SqlConnection = eConnection.StoreConnection as SqlConnection;
if (SqlConnection == null)
throw new ArgumentException("StoreConnection shall be SQL Connection");
using (SqlCommand command = new SqlCommand("DBCC SHRINKDATABASE(0)", SqlConnection))
{
command.ExecuteNonQuery();
}
eConnection.Close();
}
return Content("Done");
}