When running the 'migrate' command of the flyway, the standard way to access DB is by providing user and password. In my case, it's Azure SQL DB and instead of user/password I have only MSI token. Is it possible to pass the token to the command line? Looking at the parameters of the 'migrate' command I couldn't find anything like that.
Azure Active Directory MSI Authentication is supported and please see the Microsoft documentation as it has more details about how these work with JDBC URLs, (link) to download the driver and the applicable connection string properties and client requirements but please note:
Supported since driver version v7.2, authentication=ActiveDirectoryMSI can be used to connect to an Azure SQL Database/Data Warehouse from inside of an Azure Resource with "Identity" support enabled. Optionally, msiClientId can also be specified in the Connection/DataSource properties along with this authentication mode, which must contain the Client ID of a Managed Service Identity to be used to acquire the accessToken for establishing the connection.
You could run this from an Azure AD Federated on-premise host or Azure host using the following java client example and then run the migrate command.
Related
I have a requirement where in I need to parameterize my Azure SQL/Azure Synapse linked service using 2 parameters 'DBServer' and 'DBName'. I configured my collaboration branch as below and everything works as expected in my Synapse workspace.
Even the linked service ARM template shows the connection string values as expected.
However when I publish the code (using the publish button in Synapse workspace), the code in the Synapse Live mode does not contain these parameters defined. Please see the snippet below.
Even the code in the TemplateParameter file of the publish branch is blank.
I tried removing the parameters and put in the actual values for Server and DBName, it still shows no values in the publish branch/Live mode. While I understand that the git mode does not store secrets, this case of mine does not have any secrets and I am authenticating via Managed Identity.
Is there a solution to deal with this scenario? I would not prefer to store the connection string in KeyVault and refer it back in the linked service as it does not make the solution completely dynamic.
I created Azure sql database and created synapse pool in Azure portal. I created SQL linked service in synapse pool with managed identity authentication.
to authenticate with managed identity, I created user in SQL database with synapse pool name using below query
create user [SynapsePoolName] from external provider;
ALTER ROLE db_owner ADD MEMBER [SynapsePoolName];
Image for reference:
Linked service:
I published the Linked Service with publish button.
Image for reference:
It published successfully.
When I open the linked service after publish, I am getting the parameters.
Image for reference:
I got the parameters in template also.
Image for reference:
It worked in my machine kindly check from your side.
I have a database running on an on-premise SQL Server instance. I've set up a SQL agent to backup the database every night and store in a container in Azure. However, I'm seeing the following error after the job runs:
Message
Executed as user: NT SERVICE\SQLSERVERAGENT. Cannot open backup device 'https://mystorageaccount.blob.core.windows.net/mystoragecontainer/20200102/MYDATABASE_0.bak'. Operating system error 50(The request is not supported.). [SQLSTATE 42000] (Error 3201) BACKUP DATABASE is terminating abnormally. [SQLSTATE 42000] (Error 3013)
The Azure storage account is Storage (general purpose v1).
SQL Server 13.0.5233.0
Microsoft SQL Server Management Studio 14.0.17213.0
Microsoft Analysis Services Client Tools 14.0.1016.232
Microsoft Data Access Components (MDAC) 10.0.14393.0
Microsoft MSXML 3.0 6.0
Microsoft Internet Explorer 9.11.14393.0
Microsoft .NET Framework 4.0.30319.42000
Operating System 6.3.14393
Is there a way of configuring NT SERVICE\SQLSERVERAGENT to connect to the Azure storage container?
All the comments are valid to a certain extent. I finally fixed (term used loosely) backing up my SQL Server database to an Azure storage container using SAS (shared access signature) credentials.
Deleted the existing credential in SQL Server (under Security > Credentials)
In Azure, created an access policy under Storage Account > Container. It's important to define start and expiration dates/times, and time zones along with read, write level.
In Azure, Generate a SAS token for the container. Ensure you set the appropriate start and expiration dates/times along with the timezone. Don't rely on just UTC.
Upload a file to the container to make sure it's all ok.
In SQL Server, create the credentials. The secret should be the SAS token without the preceding ?, so just "sv=...." .
IF NOT EXISTS
(SELECT * FROM sys.credentials
WHERE name = 'https://mystorageaccount.blob.core.windows.net/mycontainer')
CREATE CREDENTIAL [https://mystorageaccount.blob.core.windows.net/mycontainer]
WITH IDENTITY = 'SHARED ACCESS SIGNATURE',
SECRET = 'sv=_my_sas_key_without_?';
Backup the database. The URL and container must match with the credentials - in this case, https://mystorageaccount.blob.core.windows.net/mycontainer .
BACKUP DATABASE [mydatabase]
TO URL = 'https://mystorageaccount.blob.core.windows.net/mycontainer/mydatabase_03012020120400.bak'
WITH FORMAT,
COMPRESSION,
STATS=5,
BLOCKSIZE=65536,
MAXTRANSFERSIZE=4194304;
GO
This seems like access related issue.To be able to back up to Azure Blob Storage, your backup command must have a valid credential that has access to the blob storage. This can be done in two ways:
Review your credentials, make sure your access key is correct and, if you’re using SAS, make sure that the access policy is defined and you are pointing to the correct container that it has access to. For more information on URL backups as well as script samples, please refer:
https://learn.microsoft.com/en-us/sql/relational-databases/backup-restore/sql-server-backup-to-url?view=sql-server-2017
Shared Access Signature
Identity and Access Key
Additional reference:
https://blog.pythian.com/how-to-fix-sql-backup-to-url-failure-operating-system-error-50/
Hope it helps.
We are trying to run sql files from Jenkins Linux machine using Liquibase to the Microsoft SQL server db and getting the error as:
ERROR: Build step failed with exception java.lang.RuntimeException:
Error getting database connection using driver
com.microsoft.sqlserver.jdbc.SQLServerDriver using url
jdbc:sqlserver://*********.database.windows.net:***;database=*********;encrypt=true;trustServerCertificate=false;hostNameInCertificate=*.database.windows.net;loginTimeout=30;Authentication=ActiveDirectoryIntegrated'
at org.jenkinsci.plugins.liquibase.evaluator.AbstractLiquibaseBuilder.createJdbcConnection(AbstractLiquibaseBuilder.java:236)
at org.jenkinsci.plugins.liquibase.evaluator.AbstractLiquibaseBuilder.createLiquibase(AbstractLiquibaseBuilder.java:164)
at org.jenkinsci.plugins.liquibase.evaluator.AbstractLiquibaseBuilder.perform(AbstractLiquibaseBuilder.java:129)at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78)at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)at hudson.model.Build$BuildExecution.build(Build.java:205)at hudson.model.Build$BuildExecution.doRun(Build.java:162)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
at hudson.model.Run.execute(Run.java:1720)at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)at hudson.model.ResourceController.execute(ResourceController.java:98)at hudson.model.Executor.run(Executor.java:404)
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: **Azure
Active Directory is only supported on Windows operating
systems.**at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:1289)at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:773)
We have tried connecting directly with SQL server authentication and it was successful but while using Azure service principal credential we are getting this error. How do we use Azure SPN to work with Liquibase?
Are you sure you want to connect using active directory integrated to SQL Azure? You should try to connect using the admin server login of SQL Azure or using a login created on SQL Azure. If that is the case, please look how to create the connection string on the samples contained on below article.
https://learn.microsoft.com/en-us/azure/sql-database/sql-database-connect-query-java
Hope this helps.
Regards,
Alberto Morillo
I'm using a IBM MobileFirst SQL adapter to get data from a AWS Oracle RDS.
In order to connect to the sql adapter, we need to provide the username, password, url and port. The connection works, however, how should I hash it so that no one can read it? This is to make it more secure. Is this possible?
<driverClass>oracle.jdbc.driver.OracleDriver</driverClass>
<url>jdbc:oracle:thin:#xxx.rds.amazonaws.com:1525/SchemaSample</url>
<user>Username</user>
<password>Password</password>
</dataSourceDefinition>
First, you're using a very old build of 7.1 and your team should consider updating both the client and server versions.
Second, just to clarify, you don't "connect to the sql adapter". Your client sends a request to the MobileFirst Server, to use the adapter in order to connect to the AWS Oracle RDS. It is the request of the adapter to the AWS that you want its properties hashed, or encrypted.
As for encrypting properties, this depends on the application server you've installed MobileFirst Server on (WAS, WAS liberty, or Tomcat). In general, the process is as follows:
If you're using Tomcat, you can create a custom property in the worklight.properties file, encrypt it as instructed here and then re-deploy the .war file.
If you're using WAS or WAS liberty, you can also create such custom property in worklight.properties, and then expose it as a JNDI entry in the application server. You can then encrypt it using the encryption tools provided by each application server.
See here for an example how to set it up: https://stackoverflow.com/a/25159260/1530814
Once you have it setup, you can now take care of the encryption by following the documentation of the tool for your application server.
Encryption tools by each application server (search for "Encoding the JDNI properties")
I'm using a IBM MobileFirst SQL adapter to get data from a AWS Oracle RDS.
In order to connect to the sql adapter, we need to provide the username, password, url and port. The connection works, however, how should I hash it so that no one can read it? This is to make it more secure. Is this possible?
<driverClass>oracle.jdbc.driver.OracleDriver</driverClass>
<url>jdbc:oracle:thin:#xxx.rds.amazonaws.com:1525/SchemaSample</url>
<user>Username</user>
<password>Password</password>
</dataSourceDefinition>
First, you're using a very old build of 7.1 and your team should consider updating both the client and server versions.
Second, just to clarify, you don't "connect to the sql adapter". Your client sends a request to the MobileFirst Server, to use the adapter in order to connect to the AWS Oracle RDS. It is the request of the adapter to the AWS that you want its properties hashed, or encrypted.
As for encrypting properties, this depends on the application server you've installed MobileFirst Server on (WAS, WAS liberty, or Tomcat). In general, the process is as follows:
If you're using Tomcat, you can create a custom property in the worklight.properties file, encrypt it as instructed here and then re-deploy the .war file.
If you're using WAS or WAS liberty, you can also create such custom property in worklight.properties, and then expose it as a JNDI entry in the application server. You can then encrypt it using the encryption tools provided by each application server.
See here for an example how to set it up: https://stackoverflow.com/a/25159260/1530814
Once you have it setup, you can now take care of the encryption by following the documentation of the tool for your application server.
Encryption tools by each application server (search for "Encoding the JDNI properties")