So I'm writing a Facebook App using Rails, and hosted on Heroku.
On Heroku, you deploy by pushing your repo to the server.
When I do this, I'd like it to automatically change a few dev settings (facebook secret, for example) to production settings.
What's the best way to do this? Git hook?
There are a couple of common practices to handle this situation if you don't want to use Git hooks or other methods to modify the actual code upon deploy.
Environment Based Configuration
If you don't mind having the production values your configuration settings in your repository, you can make them environment based. I sometimes use something like this:
# config/application.yml
default:
facebook:
app_id: app_id_for_dev_and_test
app_secret: app_secret_for_dev_and_test
api_key: api_key_for_dev_and_test
production:
facebook:
app_id: app_id_for_production
app_secret: app_secret_for_production
api_key: api_key_for_production
# config/initializers/app_config.rb
require 'yaml'
yaml_data = YAML::load(ERB.new(IO.read(File.join(Rails.root, 'config', 'application.yml'))).result)
config = yaml_data["default"]
begin
config.merge! yaml_data[Rails.env]
rescue TypeError
# nothing specified for this environment; do nothing
end
APP_CONFIG = HashWithIndifferentAccess.new(config)
Now you can access the data via, for instance, APP_CONFIG[:facebook][:app_id], and the value will automatically be different based on which environment the application was booted in.
Environment Variables Based Configuration
Another option is to specify production data via environment variables. Heroku allows you to do this via config vars.
Set up your code to use a value based on the environment (maybe with optional defaults):
facebook_app_id = ENV['FB_APP_ID'] || 'some default value'
Create the production config var on Heroku by typing on a console:
heroku config:add FB_APP_ID=the_fb_app_id_to_use
Now ENV['FB_APP_ID'] is the_fb_app_id_to_use on production (Heroku), and 'some default value' in development and test.
The Heroku documentation linked above has some more detailed information on this strategy.
You can explore the idea of a content filter, based on a 'smudge' script executed automatically on checkout.
You would declare:
some (versioned) template files
some value files
a (versioned) smudge script able to recognize its execution environment and generate the necessary (non-versioned) final files from the value files or (for more sensitive information) from other sources external to the Git repo.
Related
I have a typescript/node-based application where the following line of code is throwing an error:
const res = await s3.getObject(obj).promise();
The error I'm getting in terminal output is:
❌ Missing credentials in config, if using AWS_CONFIG_FILE, set AWS_SDK_LOAD_CONFIG=1
CredentialsError: Missing credentials in config, if using AWS_CONFIG_FILE, set AWS_SDK_LOAD_CONFIG=1
However, I do actually have a credentials file in my .aws directory with values for aws_access_key_id and aws_secret_access_key. I have also exported the values for these with the variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. I have also tried this with and without running export AWS_SDK_LOAD_CONFIG=1 but to no avail (same error message). Would anyone be able to provide any possible causes/suggestions for further troubleshooting?
Install npm i dotenv
Add a .env file with your AWS_ACCESS_KEY_ID etc credentials in.
Then in your index.js or equivalent file add require("dotenv").config();
Then update the config of your AWS instance:
region: "eu-west-2",
maxRetries: 3,
httpOptions: { timeout: 30000, connectTimeout: 5000 },
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
});
Try not setting AWS_SDK_LOAD_CONFIG to anything (unset it). Unset all other AWS variables. In Mac/linux, you can do export | grep AWS_ to find others you might have set.
Next, do you have AWS connectivity from the command line? Install the AWS CLI v2 if you don't have it yet, and run aws sts get-caller-identity from a terminal window. Don't bother trying to run node until you get this working. You can also try aws configure list.
Read through all the sections of Configuring the AWS CLI, paying particular attention to how to use the credentials and config files at $HOME/.aws/credentials and $HOME/.aws/config. Are you using the default profile or a named profile?
I prefer to use named profiles, but I use more than one so that may not be needed for you. I have always found success using the AWS_PROFILE environment variable:
export AWS_PROFILE=your_profile_name # macOS/linux
setx AWS_PROFILE your_profile_name # Windows
$Env:AWS_PROFILE="your_profile_name" # PowerShell
This works for me both with an Okta/gimme-aws-creds scenario, as well as an Amazon SSO scenario. With the Okta scenario, just the AWS secret keys go into $HOME/.aws/credentials, and further configuration such as default region or output format go in $HOME/.aws/config (this separation is so that tools can completely rewrite the credentials file without touching the config). With the Amazon SSO scenario, all the settings go in the config.
When I publish my .NET core app to a development IIS server, I make a call to the API and the method works fine locally. This method makes a Db call using a connection string stored in
appSettings.json as well as appSettings.Development.json
Observations:
- Yes, I have ASPNETCORE_ENVIRONMENT = Development and I have both an appSettings.json file AND appSettings.Development.json file
- So I started looking at the published files and I had BOTH of these json files in the published folder, even though appSettings.Development.json properties is set to "Build Action" = content and Copy to Output Directory = Do Not Copy
- If I comment out the code that his the Db, and return dummy data, and republish the api, i get the results fine with no complaints about "development mode"
Error I get when calling the API trying to hit the Db
Error.
An error occurred while processing your request.
Request ID:
0HLL1GOCEHH73:00000001
Development Mode
Swapping to the
Development environment displays detailed information about the error that occurred.
The Development environment shouldn't be enabled for deployed applications.
It can result in displaying sensitive information from exceptions to end users.
For local debugging, enable the
Development environment by setting the
ASPNETCORE_ENVIRONMENT environment variable to
Development
and restarting the app.
Questions:
- The Db connection strings are in both
[ update ]
Brain-fart! The db connection strings were using 'trusted', so no wonder they worked locally! Once I put in the credentials, and re-published, things worked like I expected. However, the error message threw me off.
Im still not sure why I have both of those appSettings files published? Which one will it use?
I am assuming your appsetting files are named as following:
appSettings.json
appSettings.dev.json
typically, you have to explicitly set the environment to dev. if you use Visual Studio for development, it sets an environment variable that tells the application to put it in dev mode.
Without seeing the initializing logic, I would say in prod it will use the appSettings.json.
Take a look at this article, it explains configuration in more details.
I'd like store variables in a configuration file so that I can just set them in one place and access them whenever needed.
Where is the place to put such variables in a Dancer app, and how do I access them?
The best way is to have one config.yml file with default global settings, like the following:
# appdir/config.yml
logger: 'file'
layout: 'main'
Your Question: How to access?
Ans: A Dancer application can use the 'config' keyword to easily access the settings within its config file, for instance:
get '/appname' => sub {
return "This is " . config->{appname};
};
This makes keeping your application's settings all in one place simple and easy - you shouldn't need to worry about implementing all that yourself.
You may well want to access your webapp's configuration from outside your webapp. Using Dancer, you can use the values from config.yml and some additional default values:
# bin/script1.pl
use Dancer ':script';
print "template:".config->{template}."\n"; #simple
print "log:".config->{log}."\n"; #undef
Note that config->{log} should result undef error on a default scaffold since you did not load the environment and in the default scaffold log is defined in the environment and not in config.yml. Hence undef.
If you want to load an environment you need to tell Dancer where to look for it. One way to do so, is to tell Dancer where the webapp lives. From there Dancer deducts where the config.yml file is (typically $webapp/config.yml).
# bin/script2.pl
use FindBin;
use Cwd qw/realpath/;
use Dancer ':script';
#tell the Dancer where the app lives
my $appdir=realpath( "$FindBin::Bin/..");
Dancer::Config::setting('appdir',$appdir);
Dancer::Config::load();
#getter
print "environment:".config->{environment}."\n"; #development
print "log:".config->{log}."\n"; #value from development environment
By default Dancer loads development environment (typically $webapp/environment/development.yml). If you want to load an environment other than the default, try this:
# bin/script2.pl
use Dancer ':script';
#tell the Dancer where the app lives
Dancer::Config::setting('appdir','/path/to/app/dir');
#which environment to load
config->{environment}='production';
Dancer::Config::load();
#getter
print "log:".config->{log}."\n"; #has value from production environment
I am trying to set up my first Cloudbees app.
Is there documentation or tutorial that shows how to
a) set variables depending on the environment. e.g. restful end point URLs have to change depending on dev, test or prod
b) initialize the database. We want to initialise the database when we do from dev to test, but not from test to prod.
Thanks
a) you should use application parameters for your DEV/TEST/PROD application to have adequate URL set as system property. parameters are tied to an application ID, so a common pattern is to deploy same binary to myapp-dev, myapp-test, myapp-prod, but change the configuration bindings.
b) use a boolean system property to disable the database migration process on production.
I'm using the web deploy API to deploy a web package (.zip file, created by MSDeploy.exe) to programmatically roll the package out to a server (we need to do some other things before we release the package which is why we're not doing it all in one go using MSDeploy.exe).
Here's the code I have. My question is really to clarify what is happening when this is executed. In the package parameters XML file I have the application name specified ("Default Web Site") but that's about it, there's no other params are specified in there. From testing the server it appears the package gets deployed successfully but my question is are any other settings on the server I'm deploying to getting changed without my knowledge, are any default settings published etc.? Things like security settings, directory browsing etc. that I might not be aware of? The code here seems to deploy the package but I'm anxious about using this on a production environment when I'm so unsure of how this API works. The MS documentation is not helpful (more like non-existant, actually).
DeploymentChangeSummary changes;
string packageToDeploy = "C:/MyPackageLocation.zip";
string packageParametersFile = "C:/MyPackageLocation.SetParameters.xml";
DeploymentBaseOptions destinationOptions = new DeploymentBaseOptions()
{
UserName = "MyUsername",
Password = "MyPassword",
ComputerName = "localhost"
};
using (DeploymentObject deploymentObject = DeploymentManager.CreateObject(DeploymentWellKnownProvider.Package,
packageToDeploy))
{
deploymentObject.SyncParameters.Load(packageParametersFile);
DeploymentSyncOptions syncOptions = new DeploymentSyncOptions();
syncOptions.WhatIf = false;
//Deploy the package to the server.
changes = deploymentObject.SyncTo(destinationOptions, syncOptions);
}
If anyone could clarify that this snippet should deploy a package to a web site application on a server, without changing any existing server settings (unless specified in the SetParameters.xml file) that would be really helpful. Any good resources on using the API or an explanation of how web deployment works behind the scenes would also be much appreciated!
The setparameters file just controls the value for the parameters defined in the package. A package might be doing much more than that. Web deploy has a concept of providers and any given package can have one or more providers.
If you want to make sure that the package is not changing server side settings the best approach you can take is to use the API but make the packages be deployed via Web Management Service. This will give you two benefits:
You can control what providers you allow through.
You can add users and give restricted permissions to them to deploy to their site or their folder etc.
The alternate approach is to:
In the package manually look at the archive.xml and look for the providers in the package. As long as you dont see any of the following providers that can cause server settings change such as apphostconfig or webserver or regkey (this is not a comprehensive list) you should be good. Runcommand is a provider that allows you to execute batch scripts or commands. While it is a good provider for admins themselves you need to consider whether you want to allow packages with such providers to run.
You can do the above mentioned inspection in code by calling getchildren on the deployment object you create out of the package and inspect the providers and the provider paths.