I have different projects that are consuming many WCF services. I am using VSTS to automate deployments. Those services target different URLs (endpoint addresses) based on the environment where they are going to be deployed.
I am trying to use web deploy with VSTS release management as suggested in this link:WebDeploy with VSTS, which proposes to create:
Parmeters.xml
Then, add new task "Replace Tokens" with the specified variable for each environment.
However, i don't guess this will work for me, because it generate tokens only for app settings keys (which is not my case).
Is there is a work around or any other suggestion that could help me to do the configuration part?
"Replace Tokens" task can works with any config file in your project and what content to be replaced is also controlled by you.
For example, if you want to replace a URL in "myconfig.config" file. You can set the URL in the config file to "#{targeturl}#", and add a "Replace Tokens" task in your definition with the following settings: (You can change the token prefix and suffix, but remember to update it accordingly in the config file since the task find the strings to replace base on it)
And then create a variable "targeturl" in the definition with the actual URL value:
Now, when you start the build/release, the string "#{targeturl}#" in "myconfig.config" file will be replaced with "www.test.com".
Related
I want to use the configuration setup for my .NET Core Web API. I installed the Microsoft.Extensions.Configuration package for the DI container.
First of all I have 4 config files
appsettings.json
Whenever all three environments use the same config value, this is the file where to put it
appsettings.Development.json
Basic config values for development purposes only. E.g. database connection points to localhost and token secret is "secret", example:
.
{
"Database": {
"ConnectionString": "Server=localhost;Port=3306;Database=db;Uid=root;Pwd=admin;Pooling=true;"
}
}
appsettings.Staging.json
Almost the same as the development file
appsettings.Production.json
Things are different here. I can't put sensitive information to that file, e.g. token secret. These values should come from the environment variable
So in my code I can access the config values via dependency injection
public class MyClass
{
public MyClass(IConfiguration configuration)
{
string databaseConnectionString = configuration["Database:ConnectionString"];
}
}
but what if the code runs in production mode? The information doesn't exist in the production file so I would have to read from the environment variables.
Would I have to create an environment variable called Database:ConnectionString and .NET Core maps all the system environment variables into the configuration file during startup if they don't exist? Or how would I pass in sensitive data to the configuration?
With the default builder, ASP.NET Core will load the configuration from multiple sources, where later sources have the chance to overwrite earlier ones. The default sources in non-development environments are the following:
General JSON configuration from appsettings.json
Environment-specific JSON configuration from appsettings.<Environment>.json
Environment variables, e.g. ConnectionStrings:DefaultConnection or ConnectionStrings__DefaultConnection (both map to the same configuration path)
Command-line arguments
So you have the ability to overwrite the configuration from the JSON files by default using both environment variables and command line arguments.
When it comes to production use, there are also other means to protect the secrets. For example, you could simply edit the appsettings.Production.json during deployment, so that the values will never leave the machine itself.
There are several solutions for this. But obviously you want to keep things such as connection strings away from version controlled files.
If running locally i would sugest using the user secrets functionality in visual studio.
However you can also set environment variables from the cli. This is an example from the documentation about configuration:
set MyKey="My key from Environment"
set Position__Title=Environment_Editor
set Position__Name=Environment_Rick
dotnet run
Of course when running in azure the key vault is a good place to put these kinds of secrets and also has great integration into .Net.
Shortly my scenario is to test a remote API if there was any changes in the called APIs, like some parameter removed or something like that.
To get this info I need to have a token.
My problem is, I can't store it in the Database and use windowsCredentials, because in the AzurePipeline the build agents has no access and connection to the Database. And if I pass the token through variables in the pipeline then I won't have the token when I run the code in local.
appSetting is stored in Git so it is not safe.
Any idea on this?
Thanks!
UserSecrets + Environment variables are the key here. Appsettings.json is for configuration that is non-sensitive, but there is a concept called user secrets (see link below) that will allow you to have stored an appsettings.json equivalent just on your machine and not in git. When specifying info in it it should override or add onto anything in your appsettings.json.
If that info is also needed for production, then environment variables should be used. Instead of a file, single configurations can be specified/overridden using environment variables.
This is all accomplished ONLY if the aspnet core web server configuration is setup to accept from all of these places. The default setup from the template should accomplish this but read the links below to make sure that your setup works.
All the configuration and best practices can be found here.
Non sensitive info/Defaults: appsettings.json
Sensitive info for devs/dev specific info: UserSecrets
Sensitive info for prod: Environment variables
I am using database connector component, with vault component to store the database credentials. Now as per the documentation of both components i have created different properties file for each environment to store the encrypted credentials for diff env.
Following is the structure of my mule project
Now the problem with this structure is that i have to build new deployable zip file whenever i have to update the database credentials for any environment.
I need a solution where i can keep all credentials encrypted and centralized and i don't have to create a build every time after updated the credentials, We can afford to restart the server, but building new zip and deploying is really cumbersome.
Second problem we have this approach is a developer needs to know the production db to update it in properties file, this is also a security issue.
Please suggest alternate approach for credentials management for mule projects.
I'm going to recommend you do NOT try to change the secure solution provided to you by MuleSoft. To alleviate the need for packaging and deployment, you would have to extract the properties files outside of the deployment and this would be a huge risk. Regardless of where you store the property files within the deployment if you change the files, you have to package and re-deploy. I see the only solution to your problem as moving the files outside of the deployment and securely storing them. Mule has provided a solution while it may be cumbersome, they are securing these files first with encryption and secondly within the server container. You can move out the property files but you have to provide a custom implementation and you will be assuming great risk to your protected resources.
Set a VM arguement e.g. environment.type=local for local machine on your anypoint studio.
Read this variable in wherever you are reading your properties file in a way that environment type is read dynamically such as below.
" location="classpath:properties/sample-app-${environment.type}.properties" doc:name="Secure Property Placeholder"/>
In order to set the environment type on your production server(or wherever you are using mule runtime), open \conf\wrapper.conf and add the arguement wrapper.java.additional.=-Dserver.type=production. If you already have any property in this file, you may need to set the value of n appropriately. For example 13 or 14.
This way you don't need to generate different deployment artefacts for different environment because correct properties file is picked by using environment specific VM arguement.
What is the best practice approach to local user generated content when using Microsoft WebDeploy and Team City to deploy fixes to a site?
Using the deployment process described by Troy Hunt:
http://www.troyhunt.com/2010/11/you-deploying-it-wrong-teamcity_26.html
When changes are made to a site the WebDeploy agent updates the site including removing old files that are no longer needed - which is great. However in the case where a site contains user generated data (say users can upload an image which is stored as a file on disk or a simple CMS where page content files can be updated by the user), what is the best practice to prevent these files being deleted by the deployment agent?
Is there an ignore flag for certain folders?
Should the user files be stored outside the root of the deployed website (Is this a security risk)?
You basically need to use MSDeploy's skip rules. This will tell MSDeploy to ignore certain files, folder, or subfolders etc.
It depends on where you implement these to what the syntax will look like. But you have the following options:
If your publishing through VS.Net using a publishing profile you can include skip rules here (I've taken this approach and seen it work fine). This SO question should point you in the right direction - MSDeploy skip rules when using MSBuild PublishProfile with Visual Studio 2012
If your using a vs.net web solution (website / web application) I later found out you can also implement skip rules in the web.config. Although the following article is a bit old the approach may still be viable - How to write skip and replace rules for MSDeploy (I havent used, or tested this approach)
Last, but not least, you could use MSDeploy skip rule on the command line itself. So assuming you execute msdeply directly (as opposed to via msbuild) you would need to append a skip parameter with the relevant attributes you require. Further information can be found at: Demystifying MSDeploy skip rules or Web Deploy Operation Settings (Look for the skip command reference, about 2/3 down the page) (Using publishing profiles with MSBuild ultimately makes this call for you, i've seen it in action working by using the first approach above).
Hope that helps!
In Visual Studio 2012, using publish profiles along with web deploy simplifies the deployments quite a bit. However it still is missing few things or may be I don't know how to use it yet.
I prefer to use the NTLM authentication without storing the username and password (especially) in the publish profiles. How can this be done? If I leave the username and password empty, I am prompted for it. Is there a way like manually modifying the .pubxml files?
Why is the username/password stored in PublishProfileName.pubxml that I have checked in the source control and not in PublishProfileName.pubxml.user that is local to each user? I could at least save the username but obviously don't want that to be checked in.
The Configuration itself is not part of PublishProfileName.pubxml but is stored in PublishProfileName.pubxml.user as LastUsedBuildConfiguration.
Same for the Platform as last point.
I am also missing support for multi-server deployments. I am currently forced to use batch files in addition to Publish Profiles.
EDIT
The command line that works fine for publishing is
MSBuild.Exe MyProject.sln /p:Configuration=QA /p:DeployOnBuild=true;PublishProfile=PublishToQA;AllowUntrustedCertificate=true /p:authType=NTLM /p:UserName=
In this I would like to omit the /p:Configuration=QA if the configuration becomes part of the publish profile itself.
Some answers to your questions.
I prefer to use the NTLM authentication without storing the username and password (especially) in the publish profiles. How can
this be done? If I leave the username and password empty, I am
prompted for it. Is there a way like manually modifying the .pubxml
files?
Your authentication is typically driven by how Web Deploy is hosted. By default if you are using the Web Management Service then you are using IIS users for auth. With IIS users you can control which users have permissions to specific sites/apps. You can configure WMSVC to use windows auth as well though. If you have issues using VS for those scenarios let me know.
If you are using the Remote Agent service to host Web Deploy then in this case you'll be using windows auth.
Why is the username/password stored in PublishProfileName.pubxml that I have checked in the source control and not in
PublishProfileName.pubxml.user that is local to each user? I could
at least save the username but obviously don't want that to be checked
in.
We have another mechanism for you to determine what information is private/shared. With the exception of the password all publish info is shared (and checked in by default). In order to simplify the design you can either have a publish profile which is shared, or one which is not shared at all. There is no in-between in which you have a profile that some fields are shared and other not. Password is special cased here and encrypted on a per-user/per-machine basis in the .pubxml.user file.
If you'd like to have a private publish profile then you can simply not check in the .pubxml file which corresponds to the publish profile. These are stored in the Properties\PublishProfiles (or My Project\PublishProfiles for VB) and just exclude them from the project and don't check the files in. The publish dialog looks for the profiles on disk, not just the ones which are in the project. Everything should continue to work.
We don't support the concept of selectively storing values in the .pubxml.user file. The publish dialog will only store a set number of values in that file. Instead of
The Configuration itself is not part of PublishProfileName.pubxml but is stored in
PublishProfileName.pubxml.user as LastUsedBuildConfiguration.
Same for the Platform as last point.
This was a mistake it should have been stored in the .pubxml file, not the .pubxml.user file. We have since fixed this, but haven't had a chance to release the update yet.
The Configuration property cannot be set in the publish profile. The Configuration property is a core part of the build process. To be more specific, the reason why we didn't call this property Configuration is because the .pubxml file is imported into the definition of the .csproj/.vbproj during a build & publish. Since other properties are defined based on Configuration you cannot change the value once it's been set. I just blogged with way too much detail on this subject at http://sedodream.com/2012/10/27/MSBuildHowToSetTheConfigurationProperty.aspx. This limitation is an MSBuild thing not a publish limitation. For command line you should specify Configuration in the following way:
msbuild.exe myproj.csproj /p:...(other properties)... /p:Configuration=
I am also missing support for multi-server deployments. I am currently forced to use batch files in addition to Publish Profiles.
We don't have direct support for this, but if you expand on your needs I may be able to help. FYI I have an extension which you may be interested in. I have posted a 5 min video to http://sedodream.com/2012/03/14/PackageWebUpdatedAndVideoBelow.aspx.
You are free (and encouraged) to manually edit your pubxml files, so feel free to remove the password.
To switch to NTLM, change AuthType to NTLM in the first PropertyGroup.
Platform and Configuration remain build configuration, the user file just stores them so Visual Studio knows what the last configuration you deployed was.
By multi-server, do you mean a web farm? If so, you might try looking at the Web Farm Framework which basically performs MSDeploy syncs from the primary server to the others.
Alternatively, you could switch to the command line and use postSync to upload and execute a batch file on the remote server that triggers the other deployments from there.