I have a Dockerfile that has access to a variable that indicates the environment it is being targeted to. Our CICD pipeline makes this environment variable available to the Dockerfile and I can test for a particular environment using "Run if $Environment =".
When I detect a "test" environment, I need to create another environment variable on-the-fly. However, code like this doesn't seem to work:
RUN if $Environment="test" ; then ; /
ENV NewEnvironmentVariable = "test" ; /
fi
The get "ENV" not found when it runs. So obviously, you can't use ENV this way within a RUN .. if.
I CAN however, use bash commands to export the variable, but it's probably creating this export in a different context, so, the Dockerfile doesn't have access to it. I would have thought that exporting it would make the new environment variable to the Docker file (when it returns from the "if" block.
In short, I simply need to evaluate and existing environment variable and if it contains the value I'm looking for it will create a new ENV variable just as if I have done "ENV MyNewVar=1".
Is this possible?
Thanks
Related
I want to access to some environment variables that I set on .bashrc but i cannot get it using System.getenv(), I am using Intellij-idea ultimate on Ubuntu Linux.
On .bashrc i have:
export TWILIO_SMS_SERVICE_SID="XXXXXXXXXX"
export TWILIO_ACCOUNT_SID="XXXXXXXXXX"
export TWILIO_AUTH_TOKEN="XXXXXXXXXX"
Running on command line $ echo $TWILIO_SMS_SERVICE_SID i got the correct value XXXXXXXXXX.
But this System.getenv("TWILIO_SMS_SERVICE_SID") returns null.
When i run this, i got all the environment variables but not the last that i added:
println("Environment variables: ")
System.getenv().forEach { (key, value) -> println("$key: $value") }
Do you know what the causes might be or how to fix it?
p.s: sorry, I have no idea how to do a POC about this, I am open to any suggestion.
Did you start the Java program directly from IntelliJ IDEA? Depending on how you started it, it might not have the environment variables set, as ~/.bashrc might only be evaluated when you start a bash shell. It should work if you start IntelliJ IDEA (or just your Java program) from the bash shell that has the environment variables set.
I am working to set up a gitlab runner for multiple projects, and we want to be able to set up environment variables for all of the projects. I tried to set global variables in the .bashrc for both the gitlab-runner and root users but it did not recognize them during the CI script. What is the correct location to declare global environment variables?
You can also inject environment variables to your gitlab-runner directly in the commandline, as the gitlab-runner exec docker --help states:
OPTIONS: ..
--env value Custom environment variables injected to build environment [$RUNNER_ENV] ..
Here is a small example how I use it in a script:
Change the declarations as needed:
declare jobname="your_jobname"
declare runnerdir="/path/to/your/repository"
Get the env file into a bash array.
[ -f "$runnerdir/env" ] \
&& declare -a envlines=($(cat "$runnerdir/env"))
declare -a envs=()
for env in "${envlines[#]}"; do
envs+=(--env "$env")
done
And finally pass it to the gitlab-runner.
[ -d "$runnerdir" ] && cd "$runnerdir" \
&& gitlab-runner exec docker "${envs[#]}" $jobname \
&& cd -
You can define environment variables to inject in the runner's config.toml file. See the advanced runner configuration documentation in the [[runners]] section.
There doesn't seem to be a way to specify environment variables in the GitLab UI just for a specific runner.
With GitLab 13.1 (June 2020), you now have:
Instance-level CI/CD variables
GitLab now supports instance-level variables.
With this ability to set global variables, you no longer need to manually enter the same credentials repeatedly for all your projects.
This MVC introduces access to this feature by API, and the next iteration of this feature will provide the ability to configure instance-level variables directly in the UI.
See Documentation and issue.
Consider using an external persistent secret storage service like Vault or Keywhiz
Disclaimer: I am not associated nor used any of the above services
I have added export MY_VAR="FOO" to gitlab-runner's .bashrc, and it works.
echo export MY_VAR=\"FOO\" >> /home/gitlab-runner/.bashrc
Check which type of executor do you use? (shell, kubernetes, docker-ssh, parallels...) I use shell executor.
Check what type of shell does gitlab-runner use? (How to determine the current shell I'm working on) And edit the proper rc file for that.
Check the Gitlab CI Runner user.
I suggest dump all environment variables for further debugging, by add env to the .gitlab-ci.yml's script:
#.gitlab-ci.yml
job:
script: env
Making changes in ~/.bash_profile NOT ~/.bashrc.
See my answer
You can easily setup Variables in the GitLab Settings:
Project-level variables can be added by going to your project's Settings > CI/CD, then finding the section called Variables.
To make sure your variables are only used in
See:
https://docs.gitlab.com/ee/ci/variables/#variables
I am using Bamboo 5.6.2
I am defining a plan global variable named "cd" and willing to access it as part of inline script task.
I have tried
echo $BAMBOO_CD
echo `$BAMBOO_CD`
echo "$BAMBOO_CD"
echo $BAMBOO_cd
echo `$BAMBOO_cd`
echo "$BAMBOO_cd"
But none of them are printing anything.
I have already gone through https://confluence.atlassian.com/display/BAMBOO056/Defining+global+variables
Appreciate pointers or an alternative way to define things globally.
I do not have an option of upgrading Bamboo at this stage to use advanced features.
if you declare dev as global variable . In inline script , you can access using $bamboo_dev
In my Dockerfile, I would like to define variables that I can use later in the Dockerfile.
I am aware of the ENV instruction, but I do no want these variables to be environment variables.
Is there a way to declare variables at Dockerfile scope?
You can use ARG - see https://docs.docker.com/engine/reference/builder/#arg
The ARG instruction defines a variable that users can pass at
build-time to the builder with the docker build command using the
--build-arg <varname>=<value> flag. If a user specifies a build
argument that was not defined in the Dockerfile, the build outputs an
error.
Can be useful with COPY during build time (e.g. copying tag specific content like specific folders)
For example:
ARG MODEL_TO_COPY
COPY application ./application
COPY $MODEL_TO_COPY ./application/$MODEL_TO_COPY
While building the container:
docker build --build-arg MODEL_TO_COPY=model_name -t <container>:<model_name specific tag> .
To answer your question:
In my Dockerfile, I would like to define variables that I can use later in the Dockerfile.
You can define a variable with:
ARG myvalue=3
Spaces around the equal character are not allowed.
And use it later with:
RUN echo $myvalue > /test
To my knowledge, only ENV allows that, as mentioned in "Environment replacement"
Environment variables (declared with the ENV statement) can also be used in certain instructions as variables to be interpreted by the Dockerfile.
They have to be environment variables in order to be redeclared in each new containers created for each line of the Dockerfile by docker build.
In other words, those variables aren't interpreted directly in a Dockerfile, but in a container created for a Dockerfile line, hence the use of environment variable.
This day, I use both ARG (docker 1.10+, and docker build --build-arg var=value) and ENV.
Using ARG alone means your variable is visible at build time, not at runtime.
My Dockerfile usually has:
ARG var
ENV var=${var}
In your case, ARG is enough: I use it typically for setting http_proxy variable, that docker build needs for accessing internet at build time.
Christopher King adds in the comments:
Watch out!
The ARG variable is only in scope for the "stage that it is used" and needs to be redeclared for each stage.
He points out to Dockerfile / scope
An ARG variable definition comes into effect from the line on which it is defined in the Dockerfile not from the argument’s use on the command-line or elsewhere.
For example, consider this Dockerfile:
FROM busybox
USER ${user:-some_user}
ARG user
USER $user
# ...
A user builds this file by calling:
docker build --build-arg user=what_user .
The USER at line 2 evaluates to some_user as the user variable is defined on the subsequent line 3.
The USER at line 4 evaluates to what_user as user is defined and the what_user value was passed on the command line.
Prior to its definition by an ARG instruction, any use of a variable results in an empty string.
An ARG instruction goes out of scope at the end of the build stage where it was defined.
To use an arg in multiple stages, each stage must include the ARG instruction.
If the variable is re-used within the same RUN instruction, one could simply set a shell variable. I really like how they approached this with the official Ruby Dockerfile.
You can use ARG variable defaultValue and during the run command you can even update this value using --build-arg variable=value. To use these variables in the docker file you can refer them as $variable in run command.
Note: These variables would be available for Linux commands like RUN echo $variable and they wouldn't persist in the image.
Late to the party, but if you don't want to expose environment variables, I guess it's easier to do something like this:
RUN echo 1 > /tmp/__var_1
RUN echo `cat /tmp/__var_1`
RUN rm -f /tmp/__var_1
I ended up doing it because we host private npm packages in aws codeartifact:
RUN aws codeartifact get-authorization-token --output text > /tmp/codeartifact.token
RUN npm config set //company-123456.d.codeartifact.us-east-2.amazonaws.com/npm/internal/:_authToken=`cat /tmp/codeartifact.token`
RUN rm -f /tmp/codeartifact.token
And here ARG cannot work and i don't want to use ENV because i don't want to expose this token to anything else
I want to pass a system-wide variable to Apache so I can pass it to executed scripts using PassEnv. Basically a script executed Apache executes a shell script, that shell script wont run without the variable being set.
But Ubuntu devs did this in the startup script:
ENV="env -i LANG=C PATH=/usr/local/bin:/usr/bin:/bin"
Resulting in variables from /etc/environment to be discarded. Can I fix this without modifying the startup script?
Turns out you can pass along vars in /etc/apache2/envvars. Still sucks though.
Nope. The value stays empty.