I have about 85 SSIS packages that are using the same connection manager.
I understand that each package has its own connection manager.
I am trying to decide what would be the best configurations approach to simply set the connectionstring of the connection manager based on the server the packages are residing on.
I have visited all kinds of suggestions online, but cannot find anywhere the practice where I can simply copy the configuration from one package to the rest of the packages.
There are obviously many approaches such as XML file, SQL Server, Environment Variable, etc.
All the articles out there are pointing to use an Indirect method by using XML or SQL approach. Why would using an environment variable for just holding a connection string is such a bad approach?
Any suggestions are highly appreciated.
Thanks!
Why would using an environment variable for just holding a connection string is such a bad approach?
I find the environment variable or registry key configuration approach to be severely limited by the fact that it can only configure one item at a time. For a connection string, you'd need to define an environment variable for each catalog on a given server. Maybe it's only 2 or 3 and that's manageable. We had a good 30+ per database instance and we had multi-instanced machines so you can see how quickly this problem explodes into a maintenance nightmare. Contrast that with a table or xml based approach which can hold multiple configuration items for a given configuration key.
...best configurations approach to simply set the connectionstring of the connection manager based on the server the packages are residing on.
If you go this route, I'd propose creating a variable, ConnectionString and using it to configure the property. It's an extra step but again I find it's easier to debug a complex expression on a variable versus a complex expression on a property. With a variable, you can always pop a breakpoint on the package and look at the locals window to see the current value.
After creating a variable named ConnectionString, I right click on it, select Properties and set EvaluateAsExpression equal to True and the Expression property to something like "Data Source="+ #[System::MachineName] +"\\DEV2012;Initial Catalog=FOO;Provider=SQLNCLI11.1;Integrated Security=SSPI;"
When that is evaluated, it'd fill in the current machine's name (DEVSQLA) and I'd have a valid OLE DB connection string that connects to a named instance DEV2012.
Data Source=DEVSQLA\DEV2012;Initial Catalog=FOO;Provider=SQLNCLI11.1;Integrated Security=SSPI;
If you have more complex configuration needs than just the one variable, then I could see you using this to configure a connection manager to a sql table that holds the full repository of all the configuration keys and values.
...cannot find anywhere the practice where I can simply copy the configuration from one package to the rest of the packages
I'd go about modifying all 80something packages through a programmatic route. We received a passel of packages from a third party and they had not followed our procedures for configuration and logging. The code wasn't terribly hard and if you describe exactly the types of changes you'd make to solve your need, I'd be happy to toss some code onto this answer. It could be as simple as the following. After calling the function, it will modify a package by adding a sql server configuration on the SSISDB ole connection manager to a table called dbo.sysdtsconfig for a filter named Default.2008.Sales.
string currentPackage = #"C:\Src\Package1.dtsx"
public static void CleanUpPackages(string currentPackage)
{
p = new Package();
p.app.LoadPackage(currentPackage, null);
Configuration c = null;
// Apply configuration Default.2008.Sales
// ConfigurationString => "SSISDB";"[dbo].[sysdtsconfig]";"Default.2008.Sales"
// Name => MyConfiguration
c = p.Configurations.Add();
c.Name = "SalesConfiguration";
c.ConfigurationType = DTSConfigurationType.SqlServer;
c.ConfigurationString = #"""SSISDB"";""[dbo].[sysdtsconfig]"";""Default.2008.Sales""";
app.SaveToXml(sourcePackage, p, null);
}
Adding a variable in to the packages would not take much more code. Inside the cleanup proc, add code like this to add a new variable into your package that has an expression like the above.
string variableName = string.Empty;
bool readOnly = false;
string nameSpace = "User";
string variableValue = string.Empty;
string literalExpression = string.Empty;
variableName = "ConnectionString";
literalExpression = #"""Data Source=""+ #[System::MachineName] +""\\DEV2012;Initial Catalog=FOO;Provider=SQLNCLI11.1;Integrated Security=SSPI;""";
p.Variables.Add(variableName, readOnly, nameSpace, variableValue);
p.Variables[variableName].EvaluateAsExpression = true;
p.Variables[variableName].Expression = literalExpression;
Let me know if I missed anything or you'd like clarification on any points.
Related
I created my vb.net project to .exe file.
During installation on another machine, one can change the location of installing package to any drive.
In my project, I have set my app.config to point the Database that is available in c:\project.
If I suppose, while installation, when I change the location of installation to **d:** or anywhere, I get invalid access db.
What I want is:
I want to reconfigure my app.config file automatically, by detecting its current db location.
Imports System.Configuration
Imports System.Configuration.ConfigurationManager
Dim config As System.Configuration.Configuration = ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.None)
config.ConnectionStrings.ConnectionStrings("MyProject.Properties.Settings.MyProjectConString").ConnectionString = "Provider=Microsoft.Jet.OLEDB.4.0;DataSource=|DataDirectory|\SampleDB;Persist Security Info=True;"
config.Save(ConfigurationSaveMode.Modified)
Where MyProject.Properties.Settings.MyProjectConString is the name of your project and connection string.
Although this is too late to answer as the question is very old but I think this might help someone else in the future.
So, there is a way to change the Connection String value in the runtime. Since connection string is a read-only item like all other items that are on Application Scope under My.Settings so we can't change it using My.Setting.ConnectionString = "Something". But here is the code by which you can solve the issue and change any My.Settings item that is on Application Scope in the runtime.
So the code is,
My.Settings.Item("ConnectionString") = "Something"
simple...
MsgBox(My.Settings.Item("remoteAddress").ToString)
My.Settings.Item("remoteAddress") = "abcserver.servebbs.net"
My.Settings.Save()
MsgBox(My.Settings.Item("remoteAddress").ToString)
You have three options:
1.) Create and use a folder in C:\Databse and set your connection string at design time.
2.)Add the database to the project's data source at design time, then use '|Data Directory|\mydb.mdb' as your connection string.
3.) And if you use sqlserver, you don't need to worry about the location of the database once you have attached the database to sqlserver. You only need to use the proper connection string eg 'Data Source=.; Database = mydb; Integrated Security = False; Username=myusername; Password = mypassword; User Instance = false'.
The above is an example of a sql server with SQL Authentication mode as login, if you use Windows Authentication, set Integrated Security = True and remove both username and password.
This is my first post to this precious website. I am a new learner of vb.net. I am working on a simple purchase project, where i got some errors. But the first thing is which baffled me is:
This is my connection string at module level, on the developed machine.
Public strCn As String = "Data Source = (local); Initial Catalog = PSys; Integrated Security = false; User ID = sa; Password = 123;"
Is it mandatory that Data Source must be the original name of the System Name. I mean If i use (local) or using ( . ), so will it work or not? Because when i copy my project to any other system for further development so every time i need to change the Data source, otherwise i get the error that: "Network-related or instance-specific error occurred......."
Kindly guide me that what i need to do.
When you are developing an application which uses a database server such as MsSQL it is not wise to install the server along with your application in every pc which is installed to. For example what are you going to do if a customer has a local network with 10 computers? Are you going to install SQL server in all 10 of them? And if so what if they need to share data?
So your best approach (based on common practice by other applications) will be to allow the user to install the SQL server where he wants and let him configure your application and point it to the server's location. If you follow that path then the configuration of your application can be in the setup application or in the application itself.
Now about the development phase, I had a similar situation in which I needed to develop the same application in two different computers. What I did was to install the SQL server in both of them with a named instance "sqlexpress" then in the application I used the
Data.SqlClient.SqlConnectionStringBuilder
class to build the connection string. I did something like this:
Public Function getDevConnectionString() As String
Dim csb As New Data.SqlClient.SqlConnectionStringBuilder(My.Settings.dbConnectionString) '<-My original cs in app settings
csb.DataSource = My.Computer.Name & "\sqlexpress"
Return csb.ConnectionString
End Function
Whenever I need a connection string I simply call getDevConnectionString() which returns the connection string based on the computer name plus the sql server instance name. For example:
Dim cs As String
#If DEBUG Then
cs = getDevConnectionString()
#Else
cs = getReleaseConnectionString()
#End If
where getReleaseConnectionString() is the function that returns your connection string configured by the customer.
Hope this point you the right direction...
I'm seeing an issue with an SSIS (SQL Server 2005) job where I'm getting the following error:
The file name "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=\UNC\FOLDERS\filename.xls;Extended Properties="EXCEL 8.0;HDR=YES";" specified in the connection was not valid.
My searching around this site and others indicates that the most common cause of this is a permissions error but I don't believe that's the case in this situation since any number of files have successfully been processed through this implementation.
Here's an overview of the setup:
Vendors FTP files to us on a daily basis that a Windows service picks up, copies to a temporary directory and then calls SSIS jobs on those files. There are two SSIS jobs for each vendor one for a snapshot data feed and one for a transaction listing.
There are currently over 50 different SSIS jobs in the overall process. All of them work except for one specific transaction job which fails with the above error in a script task step. Files come in at least daily with unique file names so I grab the job, determine the vendor based off the source directory and then the file type based off indicators in the file name to determine which SSIS job to call. Since file names change every day, when the service calls the SSIS job, I pass in a series of parameters including the vendor file name so it can properly connect to the file.
Each job begins with a script task that sets necessary variable values for the rest of the job. For example, since the vendor file name changes with each run, I pass in the vendor file name through the SSIS variables collection then set the connection string of a datasource using that file name as the DataSource in the string. It is at that point of the script task that the above error occurs. Here's the task script code where the error occurs:
Dts.Connections("Transactions File").ConnectionString = _
Dts.Variables("ConnectionString").Value.ToString().Replace("##FILE_PATH##", sourceFilePath)
The ConnectionString value is: Provider=Microsoft.Jet.OLEDB.4.0;Data Source=##FILE_PATH##;Extended Properties="EXCEL 8.0;HDR=YES";
The sourceFilePath is the full UNC path to the vendor file in the processing directory
I don't believe it's a permissions error since all the other files going through this process (using the same holding directory for processing) are working. It shouldn't be an issue of the file not existing since again it follows the same process as every other file and I have verified the file properly ends up in the correct directory. I also considered that the connection string might be too long, but the filepath ends up at 109 characters and even with a shorter (<90) full path, the same error occurs.
Is there anything else you can you think of for me to look at? Thanks for any help.
Based on the information presented, you are doing everything correct. If you're new to SSIS, one thing I'd suggest, is that you get a copy of the excellent add-in BIDSHelper. It has great features that can really save you time especially with regard to configurations and expressions.
I created a reference package that had an Excel Connection Manager pointing to C:\ssisdata\so_paulsmithjr.xls and wired everything up.
At this point, I know things are working so it was time to make the package move. I created the following variables and their values
CurrentFile - C:\ssisdata\so_paulsmithjr.xls
PlaceHolder - ##FILE_PATH##
TemplateConnection - Provider=Microsoft.Jet.OLEDB.4.0;Data Source=##FILE_PATH##;Extended Properties="Excel 8.0;HDR=YES";
A fourth variable is set to be an expression (Right click on variable, properties window. Set Evaluate as Expression = True & Expression is below)
CurrentConnection - REPLACE(#[User::TemplateConnection], #[User::PlaceHolder], #[User::CurrentFile])
I compared the CurrentConnection value to the ReferenceConnection (which is the original value of the Excel Connection Manager's connection string) and things were a match. At this point, if I were to change the value of CurrentFile to C:\ssisdata\so_paulsmithjr - Copy.xls, that would automatically be reflected in the value of CurrentConnection.
The final trick would be to use an Expression on the Excel Connection Manager. Again, right click on the CM and under Properties, there will be Expressions. It won't expand as there is nothing under it. Instead click the ellipses and then select ConnectionString property and select the ellipses again and this time drag down the #[User::CurrentFile] variable. Click OK x2 and now your connection manager is set to use wherever the CurrentConnection variable specifies.
Does that work any better?
In SQL Server 2005, is there a way to specify more than one connection string from within a .NET Application, with one being a primary preferred connection, but if not available it defaults to trying the other connection (which may be going to a diff DB / server etc)?
If nothing along those exact lines, is there anything we can use, without resorting to writing some kind of round-robin code to check connections?
Thanks.
We would typically use composition on our SqlConnection objects to check for this. All data access is done via backend classes, and we specify multiple servers within the web/app.config. (Forgive any errors, I am actually writing this out by hand)
It would look something like this:
class MyComponent
{
private SqlConnection connection;
....
public void CheckServers()
{
// Cycle through servers in configuration files, finding one that is usable
// When one is found assign the connection string to the SqlConnection
// a simple but resource intensive way of checking for connectivity, is by attempting to run
// a small query and checking the return value
}
public void Open()
{
connection.Open();
}
public ConnectionState State
{
get {return connection.State;}
set {connection.State = value;}
}
// Use this method to return the selected connection string
public string SelectedConnectionString
{
get { return connection.ConnectionString; }
}
//and so on
}
This example includes no error checking or error logging, make sure you add that, so the object can optionally report which connections failed and why.
Assuming that you'd want to access the same set of data, then you'd use clustering or mirroring to provide high availability.
SQLNCLI provider supports SQL Server database mirroring
Provider=SQLNCLI;Data Source=myServer;Failover Partner=myMirrorServer
Clustering just uses the virtual SQL instance name.
Otherwise, I can't quite grasp why you'd want to do this...
Unfortunately there are no FCL methods that do this - you will need to implement this yourself.
I am using the HybridSessionBuilder supplied by Palermo and his team .. link ..
We have our staging environments set up so that the url will be one of the following based on the environment
web-test.company.com
web-cert.company.com
web.company.com
what we normally do is take a look at the url and if it has "-test" we use the test configurations and so on (connection strings, etc).
This is the first project that uses nhibernate in this type of environment. What would be a good way to tell the Session Builder to use the correct hibernate cfg (I will build 1 for each environment).
The HybridSessionBuilder lives in an infrastructure layer and is injected into repositories via StructureMap.
Here's how I select a single configuration file using the HybridSessionBuilder:
public Configuration GetConfiguration()
{
var configuration = new Configuration();
string cfgFile = Path.GetDirectoryName(Assembly.GetAssembly(this.GetType()).CodeBase) +
"\\com.Data.nHibernate.cfg.xml";
configuration.Configure(cfgFile);
configuration.AddAssembly("com.Data");
return configuration;
}
If you want to select configuration files based on the URL I would just identify the call stack that leads to this function and pass in either an enum value or the config file's name directly.