I am trying to do some SQL stuff (back a database, restore a database, run some script files) as part of an MSBuild script. The Sql2008.Database task from the MSBuild Extension pack seemed to be a nice easy way of doing this, however i cannot find a way to specify a server instance or username and password. I find it hard to believe that it is limited to the default instance on the machine you are building on and want to connect as the user MSBuild is running but i can't find a way to do anything else. This is the documentation I am working from. Can anyone explain how to do this or do i need to look at a different approach?
There are UserName and UserPassword properties you can use:
<MSBuild.ExtensionPack.Sql2008.Database
MachineName="$(SQLServer)"
TaskAction="Backup"
DatabaseItem="$(SQLDatabaseName)"
DataFilePath="$(SQLBackupLocation)"
UserName="$(SQLUserName)"
UserPassword="$(SQLPassword)" />
I discovered that the MSBuild.ExtensionPack.BaseTask class has the properties MachineName UserName and UserPassword. SQL2008.Database class inherits from this class. I am used to MSDN style documentation where the inherited members are documented on the derived class so i did not think to look for them there although i should have. It does seem a strange place for them though as they are meaningless in the context of many of the other classes that inherit from MSBuild.ExtensionPack.BaseTask.
Doesnt that task have the MachineName member? Think thats just the sql instance name you need.
I use the task MSBuild.ExtensionPack.SqlServer.SqlExecute and a sql statement to do everything else
e.g.
<MSBuild.ExtensionPack.SqlServer.SqlExecute TaskAction="ExecuteReader"
Sql="RESTORE DATABASE $(DatabaseName) FROM DISK = N'$(RestoreFileDestination)\Source\$(Branch)\Build\$(DatabaseFile)' WITH FILE = 1,MOVE N'Accelerate' TO N'$(dataDir)\$(DatabaseName)_1.LDF', MOVE N'Accelerate_log' TO N'$(logsDir)\$(DatabaseName)_2.LDF', NOUNLOAD, REPLACE, KEEP_CDC, STATS = 10"
ConnectionString="Data Source=Localhost\SQLExpress;Initial Catalog=master;Integrated Security=True"
CommandTimeout="660">
They'd never think of all the options i seem to need....
Related
Is there a way to make the logged user (on superset) to make the queries on impala?
I tried to enable the "Impersonate the logged on user" option on Databases but with no success because all the queries run on impala with superset user.
I'm trying to achieve the same! This will not completely answer this question since it does not still work but I want to share my research in order to maybe help another soul that is trying to use this instrument outside very basic use cases.
I went deep in the code and I found out that impersonation is not implemented for Impala. So you cannot achieve this from the UI. I found out this PR https://github.com/apache/superset/pull/4699 that for whatever reason was never merged into the codebase and tried to copy&paste code in my Superset version (1.1.0) but it didn't work. Adding some logs I can see that the configuration with the impersonation is updated, but then the actual Impala query is with the user I used to start the process.
As you can imagine, I am a complete noob at this. However I found out that the impersonation thing happens when you create a cursor and there is a constructor parameter in which you can pass the impersonation configuration.
I managed to correctly (at least to my understanding) implement impersonation for the SQL lab part.
In the sql_lab.py class you have to add in the execute_sql_statements method the following lines
with closing(engine.raw_connection()) as conn:
# closing the connection closes the cursor as well
cursor = conn.cursor(**database.cursor_kwargs)
where cursor_kwargs is defined in db_engine_specs/impala.py as the following
#classmethod
def get_configuration_for_impersonation(cls, uri, impersonate_user, username):
logger.info(
'Passing Impala execution_options.cursor_configuration for impersonation')
return {'execution_options': {
'cursor_configuration': {'impala.doas.user': username}}}
#classmethod
def get_cursor_configuration_for_impersonation(cls, uri, impersonate_user,
username):
logger.debug('Passing Impala cursor configuration for impersonation')
return {'configuration': {'impala.doas.user': username}}
Finally, in models/core.py you have to add the following bit in the get_sqla_engine def
params = extra.get("engine_params", {}) # that was already there just for you to find out the line
self.cursor_kwargs = self.db_engine_spec.get_cursor_configuration_for_impersonation(
str(url), self.impersonate_user, effective_username) # this is the line I added
...
params.update(self.get_encrypted_extra()) # already there
#new stuff
configuration = {}
configuration.update(
self.db_engine_spec.get_configuration_for_impersonation(
str(url),
self.impersonate_user,
effective_username))
if configuration:
params.update(configuration)
As you can see I just shamelessy pasted the code from the PR. However this kind of works only for the SQL lab as I already said. For the dashboards there is an entirely different way of querying Impala that I did not still find out.
This means that queries for the dashboards are handled in a different way and there isn't something like this
with closing(engine.raw_connection()) as conn:
# closing the connection closes the cursor as well
cursor = conn.cursor(**database.cursor_kwargs)
My gut (and debugging) feeling is that you need to first understand the sqlalchemy part and extend a new ImpalaEngine class that uses a custom cursor with the impersonation conf. Or something like that, however it is not simple (if we want to call this simple) as the sql_lab part. So, the trick is to find out where the query is executed and create a cursor with the impersonation configuration. Easy, isnt'it ?
I hope that this could shed some light to you and the others that have this issue. Let me know if you did find out another way to solve this issue, or if this comment was useful.
Update: something really useful
A colleague of mine succesfully implemented impersonation with impala without touching any superset related, but instead working directly with the impyla lib. A PR was open with the code to change. You can apply the patch directly in the impyla src used by superset. You have to edit both dbapi.py and hiveserver2.py.
As a reminder: we are still testing this and we do not know if it works with different accounts using the same superset instance.
I created my vb.net project to .exe file.
During installation on another machine, one can change the location of installing package to any drive.
In my project, I have set my app.config to point the Database that is available in c:\project.
If I suppose, while installation, when I change the location of installation to **d:** or anywhere, I get invalid access db.
What I want is:
I want to reconfigure my app.config file automatically, by detecting its current db location.
Imports System.Configuration
Imports System.Configuration.ConfigurationManager
Dim config As System.Configuration.Configuration = ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.None)
config.ConnectionStrings.ConnectionStrings("MyProject.Properties.Settings.MyProjectConString").ConnectionString = "Provider=Microsoft.Jet.OLEDB.4.0;DataSource=|DataDirectory|\SampleDB;Persist Security Info=True;"
config.Save(ConfigurationSaveMode.Modified)
Where MyProject.Properties.Settings.MyProjectConString is the name of your project and connection string.
Although this is too late to answer as the question is very old but I think this might help someone else in the future.
So, there is a way to change the Connection String value in the runtime. Since connection string is a read-only item like all other items that are on Application Scope under My.Settings so we can't change it using My.Setting.ConnectionString = "Something". But here is the code by which you can solve the issue and change any My.Settings item that is on Application Scope in the runtime.
So the code is,
My.Settings.Item("ConnectionString") = "Something"
simple...
MsgBox(My.Settings.Item("remoteAddress").ToString)
My.Settings.Item("remoteAddress") = "abcserver.servebbs.net"
My.Settings.Save()
MsgBox(My.Settings.Item("remoteAddress").ToString)
You have three options:
1.) Create and use a folder in C:\Databse and set your connection string at design time.
2.)Add the database to the project's data source at design time, then use '|Data Directory|\mydb.mdb' as your connection string.
3.) And if you use sqlserver, you don't need to worry about the location of the database once you have attached the database to sqlserver. You only need to use the proper connection string eg 'Data Source=.; Database = mydb; Integrated Security = False; Username=myusername; Password = mypassword; User Instance = false'.
The above is an example of a sql server with SQL Authentication mode as login, if you use Windows Authentication, set Integrated Security = True and remove both username and password.
I've been googling around around and read this article
C# How to get SQL Server installation path programatically?
and this is exactly what i need in VB.NET , however i'm not good in translating this code into VB.NET Code. So, any help would greatly appreciated. Thank you.
Note : I'm using SQL Server 2005 and Visual Basic 2008
While the question was originally titled about retrieving SQL Server's installation path, I felt it was more about a code translation problem (the solution already existed, just not in the right language).
But then I thought that the method in the original code was fairly blunt.
Evan provided you with what I assume is a workable translation of the existing solution. But probably a much easier way to perform this specific task - assuming you just need to find the installation path for an instance you're already connected to, and assuming that a user who can read the registry will also have VIEW SERVER STATE permissions - is to issue this simple query against the DMV sys.dm_os_loaded_modules from your program:
SELECT name
FROM sys.dm_os_loaded_modules
WHERE name LIKE '%sqlservr.exe';
This will give you something like this:
C:\Program Files\Microsoft SQL Server\MSSQL11.SQL2012\MSSQL\Binn\sqlservr.exe
You have some parsing to do, depending on exactly what you're after (e.g. do you want to stop at MSSQL, or Binn?), but this is much easier than reading the registry or other methods that are out there IMHO.
I just used a code converter ... There are only basic things that need to be changed ..
Using sqlServerKey As RegistryKey = Registry.LocalMachine.OpenSubKey("SOFTWARE\Microsoft\Microsoft SQL Server")
For Each subKeyName As String In sqlServerKey.GetSubKeyNames()
If subKeyName.StartsWith("MSSQL.") Then
Using instanceKey As RegistryKey = sqlServerKey.OpenSubKey(subKeyName)
Dim instanceName As String = instanceKey.GetValue("").ToString()
If instanceName = "MSSQLSERVER" Then
'say
Dim path__1 As String = instanceKey.OpenSubKey("Setup").GetValue("SQLBinRoot").ToString()
path__1 = Path.Combine(path__1, "sqlserver.exe")
Return path__1
End If
End Using
End If
Next
End Using
If you were to just read a quick article on C#, you would notice that strings are declared differently, and minor syntax discrepancies exist such as foreach vs for each
You can read here for some more common differences.
I use a very good (offline) tool, called Convert .NET Free
It's from www.fishcodelib.com
Here's a direct link to the latest release (as of 19/04/14) Size: 2.06MB, File: Zip :
[Direct Link]
Hope this is of some use ;)
P.S. This software requires .NET Framework 4.5.
This almost never fails! :) Good Luck
http://www.developerfusion.com/tools/convert/csharp-to-vb/
I have about 85 SSIS packages that are using the same connection manager.
I understand that each package has its own connection manager.
I am trying to decide what would be the best configurations approach to simply set the connectionstring of the connection manager based on the server the packages are residing on.
I have visited all kinds of suggestions online, but cannot find anywhere the practice where I can simply copy the configuration from one package to the rest of the packages.
There are obviously many approaches such as XML file, SQL Server, Environment Variable, etc.
All the articles out there are pointing to use an Indirect method by using XML or SQL approach. Why would using an environment variable for just holding a connection string is such a bad approach?
Any suggestions are highly appreciated.
Thanks!
Why would using an environment variable for just holding a connection string is such a bad approach?
I find the environment variable or registry key configuration approach to be severely limited by the fact that it can only configure one item at a time. For a connection string, you'd need to define an environment variable for each catalog on a given server. Maybe it's only 2 or 3 and that's manageable. We had a good 30+ per database instance and we had multi-instanced machines so you can see how quickly this problem explodes into a maintenance nightmare. Contrast that with a table or xml based approach which can hold multiple configuration items for a given configuration key.
...best configurations approach to simply set the connectionstring of the connection manager based on the server the packages are residing on.
If you go this route, I'd propose creating a variable, ConnectionString and using it to configure the property. It's an extra step but again I find it's easier to debug a complex expression on a variable versus a complex expression on a property. With a variable, you can always pop a breakpoint on the package and look at the locals window to see the current value.
After creating a variable named ConnectionString, I right click on it, select Properties and set EvaluateAsExpression equal to True and the Expression property to something like "Data Source="+ #[System::MachineName] +"\\DEV2012;Initial Catalog=FOO;Provider=SQLNCLI11.1;Integrated Security=SSPI;"
When that is evaluated, it'd fill in the current machine's name (DEVSQLA) and I'd have a valid OLE DB connection string that connects to a named instance DEV2012.
Data Source=DEVSQLA\DEV2012;Initial Catalog=FOO;Provider=SQLNCLI11.1;Integrated Security=SSPI;
If you have more complex configuration needs than just the one variable, then I could see you using this to configure a connection manager to a sql table that holds the full repository of all the configuration keys and values.
...cannot find anywhere the practice where I can simply copy the configuration from one package to the rest of the packages
I'd go about modifying all 80something packages through a programmatic route. We received a passel of packages from a third party and they had not followed our procedures for configuration and logging. The code wasn't terribly hard and if you describe exactly the types of changes you'd make to solve your need, I'd be happy to toss some code onto this answer. It could be as simple as the following. After calling the function, it will modify a package by adding a sql server configuration on the SSISDB ole connection manager to a table called dbo.sysdtsconfig for a filter named Default.2008.Sales.
string currentPackage = #"C:\Src\Package1.dtsx"
public static void CleanUpPackages(string currentPackage)
{
p = new Package();
p.app.LoadPackage(currentPackage, null);
Configuration c = null;
// Apply configuration Default.2008.Sales
// ConfigurationString => "SSISDB";"[dbo].[sysdtsconfig]";"Default.2008.Sales"
// Name => MyConfiguration
c = p.Configurations.Add();
c.Name = "SalesConfiguration";
c.ConfigurationType = DTSConfigurationType.SqlServer;
c.ConfigurationString = #"""SSISDB"";""[dbo].[sysdtsconfig]"";""Default.2008.Sales""";
app.SaveToXml(sourcePackage, p, null);
}
Adding a variable in to the packages would not take much more code. Inside the cleanup proc, add code like this to add a new variable into your package that has an expression like the above.
string variableName = string.Empty;
bool readOnly = false;
string nameSpace = "User";
string variableValue = string.Empty;
string literalExpression = string.Empty;
variableName = "ConnectionString";
literalExpression = #"""Data Source=""+ #[System::MachineName] +""\\DEV2012;Initial Catalog=FOO;Provider=SQLNCLI11.1;Integrated Security=SSPI;""";
p.Variables.Add(variableName, readOnly, nameSpace, variableValue);
p.Variables[variableName].EvaluateAsExpression = true;
p.Variables[variableName].Expression = literalExpression;
Let me know if I missed anything or you'd like clarification on any points.
I just started working with an application that I inherited from someone else and I'm having some issues. The application is written in C# and runs in VS2010 against the 3.5 framework. I can't run the application on my machine to debug because it will not recognize the way they referenced their parameters when writing their DB queries.
For instance wherever they have a SQL or DB2 query it is written like this:
using (SqlCommand command = new SqlCommand(
"SELECT Field1 FROM Table1 WHERE FieldID=#FieldID", SQLconnection))
{
command.Parameters.AddWithValue("FieldID", 10000);
SqlDataReader reader = command.ExecuteReader();
...
If you will notice the "parameters.AddWithValue("FieldID", 10000);" statement does not include the "#" symbol from the original command text. When I run it on my machine I get an error message stating that the parameter "FieldID" could not be found.
I change this line:
command.Parameters.AddWithValue("FieldID", 10000);
To this:
command.Parameters.AddWithValue("#FieldID", 10000);
And all is well... until it hits the next SQL call and bombs out with the same error. Obviously this must be a setting within visual studio, but I can't find anything about it on the internet. Half the examples for SQL parameter addition are written including the "#" and the other half do not include it. Most likely I just don't know what to search for.
Last choice is to change every query over to use the "#" at the front of the parameter name, but this is the transportation and operations application used to manage the corporation's shipments and literally has thousands of parameters. Hard to explain the ROI on your project when the answer to the director's question "How's progress?" happens to be "I've been hard at it for a week and I've almost started."
Has anyone run into this problem, or do you know how to turn this setting off so it can resolve the parameter names without the "#"?
Success! System.Data is automatically imported whenever you create a .NET solution. I removed this reference and added it back to make sure that I had the latest version of this library and that fixed the issue. I must have had an old version of this library that was originally pulled in... only thing I can figure.
Its handled by the .NET Framework data providers not Visual Studio.
It depends on the data source. Look here:Working with Parameter Placeholders
You can try working with System.Data.Odbc provider and using the question mark (?) place holder. In thios case dont forget to add the parameters in the same order they are in the query.