I am working on a Deployment with Bitbucket and got some trouble using the variables. In Bitbucket I set in
Repository Settings / Deployment / Staging:
SOME_VAR = foobar
The bitbucket-pipelines.yml looks like this:
...
definitions:
steps:
- step: &build-some-project
name: 'Build Project'
script:
- echo "\$test='$SOME_VAR';" >> file.php
...
I am not able to pass the value of a variable into the file within the script part via echo, but why? I also put it in double qutes (like echo "\$test='"$SOME_VAR"';"). But the result always is just $test=''; = an empty string. The expected result should be $test='foobar';. Does anybody know how to get this working? The variable is not a secure variable.
// edit
The Repository Variables are working as well. But: I need the same variable with different values for each environment. Also the Repository Variables are accessable for users with permissions to push into the repo.
For the script to understand that the variable to be fetched from the deployment variable mention the deployment type
definitions:
steps:
- step: &build-some-project
name: 'Build Project'
deployment: Staging
script:
- echo "\$test='$SOME_VAR';" >> file.php
Related
I have a react application in which they are getting backend api address by using Environment variable. Below in the example:
this._baseUrl = process.env.API_GATEWAY;
In local development environment, development team create .env. file and set environment variable value in that file, to call backend api and every things work fine, like below.
API_GATEWAY=http://localhost:3000
When i create CI/CD pipeline for same project then every things works fine and application is also successfully deployed on AWS (s3 bucket) but i am not able to change the value of environment variable while building the project using npm, like below:
- script: |
npm run build
displayName: 'npm build'
env:
API_GATEWAY: $(envAppApi)
API_GATEWAY used above is the name of environment variable used in code and $(envAppApi) is variable defined in variable group.
But when application is deployed on AWS then environment variable value not changed and it shows below error.
mutation.js:106 ReferenceError: process is not defined
at new e (http-api.ts:17:42)
at Function.value (http-api.ts:24:12)
at Object.mutationFn (Auth.ts:13:26)
at Object.fn (mutation.js:132:31)
at c (retryer.js:95:31)
at new u (retryer.js:156:3)
at t.executeMutation (mutation.js:126:20)
at mutation.js:86:20
(http-api.ts:17:42) => This is the same line where API_GATEWAY environment variable is set and already showed above.
Problem statement:
Is there is any way that we can update the value of environment variable while creating CI/CD pipeline? so the application run successfully. Thanks.
Note: I don't want to use .env. file in my pipeline for updating environment values in react application.
Is there is any way that we can update the value of environment variable while creating CI/CD pipeline?
Yes. I suggest that you can use RegEx Match & Replace task from RegEx Match & Replace.
This task will use regular expressions to match fields in the file.
Here is an example:
steps:
- task: RegExMatchReplace#2
displayName: 'RegEx Match & Replace'
inputs:
PathToFile: test.js
RegEx: 'this._baseUrl = ([a-zA-Z]+(\.[a-zA-Z]+)+)_[a-zA-Z]+;'
ValueToReplace: ' this._baseUrl = $(envAppApi)'
Then the value will update.
You can use this site to convert the regular expressions : Regex Generator
How to set gitlab-ci varibales through script not just in "varibales" section in .gitlab-ci.yaml?So that I can set variables in one job and use in different job
There is currently no way in GitLab to pass environment variable between stages or jobs.
But there is a request for that: https://gitlab.com/gitlab-org/gitlab/-/issues/22638
Current workaround is to use artifacts - basically pass files.
We had a similar use case - get Java app version from pom.xml and pass it to various jobs later in the pipeline.
How we did it in .gitlab-ci.yml:
stages:
- prepare
- package
variables:
VARIABLES_FILE: ./variables.txt # "." is required for image that have sh not bash
get-version:
stage: build
script:
- APP_VERSION=...
- echo "export APP_VERSION=$APP_VERSION" > $VARIABLES_FILE
artifacts:
paths:
- $VARIABLES_FILE
package:
stage: package
script:
- source $VARIABLES_FILE
- echo "Use env var APP_VERSION here as you like ..."
If you run a script you can set an environment variable
export MY_VAR=the-value
once the environment variable is set it should persist in the current environment.
Now for why you do not want to do that.
A tool like Gitlab CI is meant to achieve repeatability in your
artifacts. Consistency is the matter here. What happens if a second job
has to pick up a variable from the first? Then you have multiple paths!
# CI is a sequence
first -> second -> third -> fourth -> ...
# not a graph
first -> second A -> third ...
\> second B />
How did you get to third? Now if you had to debug third which path do you test? If the build in third is broken who is responsible second A or second B?
If you need a variable use it now, not later in another job/script. Whenever you
want to write a longer sequence of commands make it a script and execute the script!
You can use either Artifact or Cache to achieve this, see the official documentation for more information around Artifact and Cache:
https://docs.gitlab.com/ee/ci/caching/#how-cache-is-different-from-artifacts
I am working with gitlab-ci.
I have set an Environment variable: MY_ENV_VAR (in Project CI/CD Settings menu)
I see a message with says the environnement variable is prefixed by K8S_SECRET_
Here is my .gitlab-ci.yml:
deploy:
only:
- prod
script:
echo ${K8S_SECRET_MY_ENV_VAR}
It does not display the value in job... I see the echo command in green but i have something blank bellow.
I have tried echo $K8S_SECRET_MY_ENV_VAR or echo $SECRET_MY_ENV_VAR. It is the same.
Thanks
The K8S_SECRET_ prefix is to target variables for the Auto Devops feature. Normal variables need no such prefix, so in your example simply reference $MY_ENV_VAR
I can manage to hard code shell scripts in .yml file, so that GitLab runner can execute the .yml contents when someone perform a push to the Git repo.
I would like to explore, if the .yml file can get data from external source (e.g. mysql database), and using loops to generate the shell scripts.
something like this:-
// this is a .yml file
get some data
foreach($data as $datum)
deploy_dev:
stage: deploy
script:
- echo "Deploy to server"
- eval $(ssh-agent)
- ssh-add ~/.ssh/key.pem
- ssh -p22 root#xxxx "do something"
environment:
name: dev
url: http://xxxx
only:
- dev
}
If it is possible, then I can update db data -> .yml -> dynamic stages for CI/CD.
Thanks.
Why don't you write a script (in any language) that does what you want and run just this script inside your CI? You can pass ENV vars to it if you need to get data from the pipeline.
I have a devop directory containing ansible's varible directroy , plabooks and inventory directory
The directory looks like this
|groups_vars
-all.yml
-development.yml
-staging.yml
|inventroy
- staging
- development
configure.yml
deploy.yml
configure.yml and deploy.yml contains task that are applied to either staging or development machines using variable in groups_vars
Know if i call ansible-playbook command with staging inventory. How will it know which variable file to use. The varfile task in not added to configure.yml and deploy.yml
By the way am using an example from the company i work and the example is working I just want to know the magic that is happening it is using the right variable file though the var file is not incuded in the configure.yml nor deploy.yml
Ansible uses a few conventions to load vars files:
group_vars/[group]
host_vars/[host]
So if you have an inventory file that looks like this:
[staging]
some-host.name.com
Then These files will be included (optional extension .yml or .yaml also):
/group_vars/all
/group_vars/staging
/host_vars/some-host.name.com
I think this is the "magic" you are referring to.
You can find more on the subject here: http://docs.ansible.com/playbooks_variables.html#variable-precedence-where-should-i-put-a-variable
And here: http://docs.ansible.com/playbooks_best_practices.html