Prevent file differences between live and staging websites AND system scripts (PHP) - apache

I'm trying to find a good way to maintain PHP configuration differences between a dev and live environment. I know how to handle differences when scripts are run by Apache (see below), but not when they are run on the command line, e.g. via cron.
Background info:
What do I mean by "configuration differences"? Like most code shops, we run a non-public 'staging' version of our website where we test code before it goes to the live website. We use Subversion and have the live website as 'Trunk' and 'Staging' as a branch. It makes life easier when code goes from staging to live if the repository version of the files have minimal differences. But obviously, some details need to be different, e.g. the DB connection details.
How configuration differences are solved with Apache
In PHP, we set branch specific variables as follows:
switch ($_SERVER['HTTP_HOST']) {
case 'ourstagingurl.com':
$dbPassword = "blahblah";
break;
default:
$dbPassword = "blahblah";
}
or we put the following in the .htaccess file relevant to the specific site:
php_value dbPassword "blahblah"
Why I can't resolve configuration differences using the CLI?
When a script is run on the CLI, there's no super globals such as $_SERVER. I could include a config file using a absolute path but how can I know whether the script is from live or staging? I could pass in a command line argument that specifies the environment but I was hoping there was a better way?

Hey, I work for a company with a similar setup. What I generally do is set up a scenerio where the staging and live servers both modify a common and generic config file. When the site is deployed on both the staging and live the information will be incorrect and will need to be modified once in each location. However, this will ONLY need to happen once because the parts you modify will be noted as modified by subversion and will not be overwritten by subversion in subsequent svn updates.

$host = isset( $_SERVER['HTTP_HOST'] ) ? $SERVER_['HTTP_HOST'] : php_uname('n');
switch ( $host )
{
case 'ourstagingurl.com':
$dbPassword = "blahblah";
break;
default:
$dbPassword = "blahblah";
}
If php version is => 5.3 then gethostname() could be used instead.

If live and staging are the same server (why don' t you use virtualisation?) then you'll probably have to resort to passing arguments to your scripts ie. call php script.php staging for staging and php script.php live for live site. BTW In case you wouldn' t know, the arguments are accessible using $argv in your PHP script -> http://www.php.net/manual/en/reserved.variables.argv.php.
Though maybe you could get part of the path of the file and use that as your "environment". ie. when file layout is /project/staging/script.php for staging and /project/live/script.php for live it should be easy to get the environment from the script itself using dirname(__FILE __) - extra space after FILE needs to be removed - and stripping the /project folder from it. Pretty stupid but it will work :p.

You can do as Jurassic suggested, or you can set an environment variable and read it in through $_ENV.

In some of my applications, I attempt to include a file that only exists to override values on the development server. I use the dreaded # operator, but you could do an is_file() as well.
#include 'includes/debug.php';
debug.php would attempt to set some variables:
define('DEBUG', true);
define('DB_PASSWORD', 'foobar');
I tell svn to ignore this file. It sticks around in my development checkout, and has a minimal impact on production.
More generally I use hostname checking as it works in both CLI and web apps, but you've stated you have both branches on the same box, so that's out.

Related

SetEnv not updating my environment variable (noob warning)

I am still an Apache noob, and I am trying to set an environment variable that will be used by my Rails application.
I've read https://httpd.apache.org/docs/2.4/mod/mod_env.html#setenv and done some google and SO searches. I have at least determined that the value to be assigned must be in quotes. However, when I run sudo service apache2 restart, the value of SECRET_KEY_BASE is still not correct (viewed via printenv). I don't know what I don't know. Is there some step i'm missing?
In my apache configuration I have:
SetEnv SECRET_KEY_BASE "e10e721..."
# Tell Apache and Passenger where your app's 'public' directory is
DocumentRoot /var/www/some_path
Please let me know what other information I might need to share. Thanks for looking.
There are a few subtle pitfalls here.
First: You can't check it in the terminal you ran that command from. "SetEnv" sets a per-request internal variable that will be copied to CGI-like processes that the server subsequently executes.
Secondly, even if you set a real native environment variable (in e.g. /etc/apache2/envvars) you should not do a restart operation since that will not necessarily reload that particular file. You should do a stop and a start. You still won't see the variable in the command you start the server from, since it was only in the webserver process.
If you want to see the environment of a running process, you can write a basic CGI to dump the environment that was passed down to the CGI script. If you're a PHP user, a basic script with phpinfo() will dump it.
Or, you can determine Apache's process ID with ps and then check /proc/$thepid/env (on Linux).

cannot get environment variables set in Flask application

I tried to set up sensitive information as environment variables in CentOS, and pass them to Flask config file used in main file, i.e. init.py . But it did not work. The Flask application is running under Apache.
I first edit /etc/environment as root user
MAIL_USERNAME="abcde#abc.com"
then logout, login again
Then verify MAIL_USERNAME is set by running
echo $MAIL_USERNAME
This works fine
And in configuration.py, this is how I set MAIL_USERNAME.
MAIL_USERNAME = os.environ.get('MAIL_USERNAME')
for testing purpose,
I print out MAIL_USERNAME
in __init__.py
print(MAIL_USERNAME)
Then from terminal, if I run
python3.4 __init__.py
it print out correct values of MAIL_USERNAME
However, if I tested on web browser, MAIL_USERNAME is just not set. it shows NONE. I verify this by looking Apache log.
Any idea of how this works would be really appreciated.
Thanks
With your CLI, set the environment variable as you want. On Linux and macOS, this is done with export KEY=value.
After that, the environment variable KEY will be available for your Python script or Flask app via os.environ.get('KEY'), like this:
import os
print os.environ.get('key')
>>> value
I had a very similar problem because I used PyCharm terminal to run flask. A similar issue was described and solved here.
My solution was switching to regular cmd (I worked on Windows 10) and just running everything there:
>> set MAIL_USERNAME='bla#example.com'
... (other env variables sets)
>> py manage.py runserver (I run my flask app through a manage script)
I could successfully send an email using my flask app - all the environment variables used in the app were read correctly.
On Linux you can just use export instead of set.
I hope it helps.
Maybe you can use Apache directive PassEnv as mentioned here on Apache's official web documenting how to use environment variables.
There are two kinds of environment variables that affect the Apache HTTP Server.
First, there are the environment variables controlled by the underlying
operating system. These are set before the server starts. They can be used in
expansions in configuration files, and can optionally be passed to CGI scripts
and SSI using the PassEnv directive.
Second, the Apache HTTP Server provides a mechanism for storing information
in named variables that are also called environment variables. This information
can be used to control various operations such as logging or access control.
The variables are also used as a mechanism to communicate with external programs
such as CGI scripts. This document discusses different ways to manipulate and
use these variables.
Although these variables are referred to as environment variables, they are
not the same as the environment variables controlled by the underlying
operating system. Instead, these variables are stored and manipulated in an
internal Apache structure. They only become actual operating system environment
variables when they are provided to CGI scripts and Server Side Include scripts.
If you wish to manipulate the operating system environment under which the server
itself runs, you must use the standard environment manipulation mechanisms
provided by your operating system shell.
I make some of the text cited above bold to make things clearer and maybe easier to explain.
Hope this helps.

Cloudinary Invalid Cloudinary Config Provided

I am new to KeystoneJS and I am having a small problem concerning my deployment on Heroku.
This is my website: http://jeroendruwe.herokuapp.com/, when I navigate to the admin section (http://jeroendruwe.herokuapp.com/keystone/signin)
I get the Invalid Cloudinary Config Provided error
Papertrailapp log: http://pastebin.com/Yn8Pdttz
I've read the documentation (http://keystonejs.com/docs/configuration/#services-cloudinary). The weird thing is that when I try one of these (in keystone.js), the whole site stops working:
keystone.set('cloudinary config', { cloud_name: 'my-cloud', api_key: 'abc', api_secret: '123' });
// or
keystone.set('cloudinary config', 'cloudinary://api_key:api_secret#cloud_name' );
So what I've done at the moment is set the property in the keystone.init(...'cloudinary config': 'cloudinary://...'). I've also added the url to the CLOUDINARY_URL environment variable in the .env file
How can I fix this issue?
Can somebody also explain what the variables in the .env file do? There is 1 in the root and another one in the node_modules/dotenv folder, these files are not pushed to git so how do they get used?
Thanks in advance!
First, let me start by answering your last question first. The .env file is used by the dotenv module, which loads the variables/values in the .env file and makes them available to your application in process.env. Make sure you call the .load() method as early as possible in your code.
var dotenv = require('dotenv');
dotenv.load();
You should also know that Heroku has two other means to configure environment variables (See Configuration and Config Variables). One via your application dashboard and another via their CLI.
Using the Heroku Dashboard, just fill in the NEW_KEY and NEW_VALUE fields, then press Save.
Using the Heroku CLI, just use the heroku config:set command.
$ heroku config:set CLOUDINARY_URL=cloudinary://api_key:api_secret#cloud_name
Adding config vars and restarting myapp... done, v12
CLOUDINARY_URL: cloudinary://api_key:api_secret#cloud_name
If you're using Heroku, I suggest you use one of these to methods to set the CLOUDINARY_URL for your application in production.
Now back to your original question. This error typically means that there's something wrong with the Cloudinary configuration (i.e. it's either incorrect or completely missing). Without seeing the actual code that you're using it would be impossible to pinpoint the exact problem.
I'm going to assume that your replacing the api_key, api_secret and cloud_name with the actual values. That said, I would still double check to make sure those values are correct.
In my Heroku deployments, I use dovenv to set the environment variables in development, and use the either the Heroku Dashboard or CLI to set them in production.
If you're still having difficulties, please post the actual code your using (omitting your actual api key, of course), including the content of your .env file.

How to set up a stageing-enviorment for wordpress/wordpress mu?

I have a wordpress mu-site. I need to set up a test-version of it so that the client can run test on the changes we make, test the plugins with new updates etc.
Anybody who has worked with wordpress know it's a bit off a hassle to move between servers and/or domain-names, due to the absolute paths used. Does anybody have a good solution how to create a stage-enviorment of wordpress?
Here's how I do it + some adjustments I want to make:
Two WP installs on identical environments - dev & production
They each have their own FQDN
Version control (SVN in this case) to handle merges from dev to production
When merging, I don't ever merge database changes. I only merge code, and modify any of the domain specific things during the merge (which really should only be in the DB.)
Recreate any DB changes needed during deployment
There are other ways to do it, but they often require changing the hosts file or access to internal systems. So if you want to be able to show an external client a site, then those methods aren't likely to work.
I also sometimes copy the DB back from production to dev, and just do a find & replace for the FQDN.
You can also dev locally and use the above listed method for staging only.

Make Maven Proxy/Server settings configurable based on location?

So I'm not sure what the best way to accomplish this is, but basically I have a laptop that I use at work for Maven projects. It works fine when I'm at work, but as soon as I walk out of the door of their corporate proxy and maven server, I often have to do alot of hand-fudging of the settings.xml file when I'm at home if I'm not VPN'ed in:
We have a corporate-installed Maven Repository proxy server to store some of our own artifacts and handle being the middle-man for our commonly used artifacts.
We have an http proxy that we use for connecting to the outside world.
Both configurations have been handled by my settings.xml file for setting a single Nexus group and maven proxies. If I'm not connected to the VPN while away from the office, I have to muck around with the settings.xml each time I'm not on it, then switch it back when I am on it.
What solutions have anyone else found to handle this? I've been trying profiles to manage the proxy, but I can't seem to get it to work correctly, and it's starting to look pretty ugly. Are there some settings configurations that can detect when I'm not behind the proxy at work and not use the corporate proxy server or Maven server?
While I can think of some profile based solution to handle the proxy (basically, reading the <active> value from a property defined in a profile), this wouldn't be fully automated (the profile activation do not support network based stuff) unless you can find a file that is present or not depending on your location (in which case, you could use an existing/missing file trigger but this is kinda hacky). Anyway, this would solve only one part of the problem because mirrors can't be declared in profiles (see MNG-3525).
So, instead of trying to control this with a profile, my suggestion would be to use two settings.xml and to pass your settings-home.xml file with the -s command line option when you're at home.
Another option would be to automate the changes in your settings.xml with a script (Groovy would be a good choice as someone reported in MNG-3525).
I found a use environment variables to set nonProxyHosts together with proxy and noproxy shell aliases to be the most convenient solution when switching between networks with proxy and without it.
In settings.xml, configure proxy with
<host>proxy.corporation.int</host>
<port>8080</port>
<nonProxyHosts>${env.MAVEN_NONPROXY}</nonProxyHosts>
Then in ~/.profile set
export MAVEN_NONPROXY_PROXY='*.corporation.int|local.net|some.host.com'
export MAVEN_NONPROXY_NOPROXY='*'
alias proxy="export MAVEN_NONPROXY=\"$MAVEN_NONPROXY_PROXY\" && export all_proxy=http://proxy.corporation.int:8080"
alias noproxy="export MAVEN_NONPROXY=\"$MAVEN_NONPROXY_NOPROXY\" && unset all_proxy"
To do the switch when roaming, you would just execute from a shell:
[me#linuxbox me]$ proxy
or
[me#linuxbox me]$ noproxy
Obviously, both aliases proxy and noproxy can include much more changes than just setup of MAVEN_NOPROXY and all_proxy.
I was frustrated by the same problem: having to manually edit settings.xml when roaming between networks. So much in fact, that I wrote a Maven plugin that enables automatic discovery of proxy settings. The current implementation uses the proxy-vole library written by Bernd Rosstauscher to detect proxy settings based on OS configuration, browser, and environment settings.
I've just released the source code of the plugin on Github, under an Apache 2.0 license: https://github.com/volkertb/autoproxy-maven-plugin
You're welcome to give it a try and to see if it meets your needs. Any feedback or contributions are welcome!
(Note: you don't necessarily have to add the plugin to your project's POM. You can invoke it from the command line as well, after you've installed it. See the README on the site for more details.)
You can set MAVEN_OPTS when you need to activate a proxy:
export MAVEN_OPTS="-Dhttp.proxyHost=my-proxy-server -Dhttp.proxyPort=80 -Dhttp.nonProxyHosts=*.my.org -Dhttps.proxyHost=my-proxy-server -Dhttps.proxyPort=80 -Dhttps.nonProxyHosts=*.my.org"