How to pass variable as parameter, object type, to Azure DevOps template - variables

To reduce duplicates I want to rewrite our Azure DevOps pipelines. The main pipelines looks like:
parameters:
- name: MODULE_Foo
type: boolean
default: false
- name: MODULE_Bar
type: boolean
default: false
...
- name: MODULE_X
type: boolean
default: false
...rest of parameters
- name: BRANCH_NAME
type: string
- name: CHECKOUT_TAG
type: boolean
default: false
extends:
template: template1.yml
parameters:
MODULES: ${{paramaters}}
BRANCH_NAME: ${{parameters.BRANCH_NAME}}
...rest of parameters
The template template1.yml:
parameters:
- name: MODULES
type: object
default: {}
...rest of parameters
stages:
- stage: Build
jobs:
- job: BuildAndDeploy
steps:
- bash: |
MODULES=$(echo "${{ convertToJson(parameters.MODULES) }}" \
| sed -E 's/^([[:space:]]+)/\1"/;s/(:[[:space:]]+)/"\1"/;s/,$/",/;s/([^{},])$/\1"/' \
| jq 'with_entries( select(.key | startswith("MODULE_") ) )' \
)
echo "##vso[task.setvariable variable=MODULES;]$MODULES"
- template: template2.yml
parameters:
MODULES: $(MODULES)
...rest of parameters
And finally template2.yml:
parameters:
- name: MODULES
type: object
default: {}
...rest of parameters
- ${{ each module in parameters.MODULES }}:
- ${{ if and(startsWith(module.Key, 'module'), eq(module.Value, true)) }}:
- bash: |
echo "Module to build: ${{module.key}}, Value: ${{module.value}}"
Now I'm stuck.
How to pass a variable as parameter to a template?
To the template is passed empty string '' for ${{variables.MODULES}} or '$(MODULES)' for $(MODULES). But never generated string. I think that $(MODULES) is valid for this situation.
How to create a valid object, which can be iterated inside each loop in template?
I think that my construction with convertToJson, sed, and jq is not correct.
My goal is to eliminate the MODULE_ parameter group from the templates. Currently, every template contains a duplicate of this parameter section.

Related

Passing variable between jobs in Azure Pipeline with empty result

I am writing an azure pipeline yml requesting to pass variables between jobs but the variables are not passing through. however, it wasn't successful and it returns an empty variable.
here is my pipeline:
jobs:
- job: UpdateVersion
variables:
terraformRepo: ${{ parameters.terraformRepo }}
pool:
vmImage: ubuntu-latest
steps:
- checkout: self
persistCredentials: true
- checkout: ${{ parameters.terraformRepo }}
- task: AzureCLI#2
displayName: PerformVerUpdate
inputs:
azureSubscription: ${{ parameters.azureSubscriptionName }}
scriptType: bash
scriptLocation: inlineScript
inlineScript: |
echo Step 3 result
echo "Reponame $Reponame"
echo "notify $notify"
echo "pullRequestId $pullRequestId"
echo "##vso[task.setvariable variable=pullRequestId;isOutput=true;]$pullRequestId"
echo "##vso[task.setvariable variable=Reponame;isOutput=true;]$Reponame"
echo "##vso[task.setvariable variable=notify;isOutput=true;]true"
Name: PerformVerUpdate
- job: SlackSuccessNotification
dependsOn: UpdateVersion
condition: and(succeeded(), eq(dependencies.UpdateVersion.outputs['PerformVerUpdate.notify'], 'true'))
pool:
vmImage: 'ubuntu-latest'
variables:
- group: platform-alerts-webhooks
- name: notify_J1
value: $[ dependencies.UpdateVersion.outputs['PerformVerUpdate.notify'] ]
- name: pullRequestId_J1
value: $[ dependencies.UpdateVersion.outputs['PerformVerUpdate.pullRequestId'] ]
- name: Reponame_J1
value: $[ dependencies.UpdateVersion.outputs['PerformVerUpdate.Reponame'] ]
steps:
- task: AzurePowerShell#5
displayName: Slack Notification
inputs:
pwsh: true
azureSubscription: ${{ parameters.azureSubscriptionName }}
ScriptType: 'InlineScript'
TargetAzurePs: LatestVersion
inline: |
write-host "Reponame $(Reponame_J1)"
write-host "pullRequest $(pullRequestId_J1)"
I've tried so many different syntax for it but the variables are still not able to pass through between both jobs - e.g. The condition is passing Null result to second job "(Expanded: and(True, eq(Null, 'true'))". Could anyone help with this?
Firstly 'Name' should be 'name' in lowercase
Name: PerformVerUpdate
The rest of syntax seems fine(I have tested it on Bash task because I do not have Azure subscription).
If renaming 'Name' does not help I suppose the problem may be that your Bash task is running within AzureCLI#2 task.
Maybe as workaround you could add new Bash task right after AzureCLI#2 and try to set there output variable for next job?

Unexpected value 'steps' in azure-pipelines.yml

Pipeline validation fails with Unexpected value 'steps' in acr-login.yaml. Even I tribble-checked the docs and stackoverflows, I can't find the issue in my pipeline:
pipeline.yaml
trigger: none
pool:
name: MyPool
variables:
- template: vars/global.yaml
- template: vars/stage.yaml
stages:
- stage: Import
jobs:
- template: steps/acr-login.yaml
parameters:
registry_name: ${{variables.registry_name}}
acr-login.yaml
parameters:
- name: registry_name
type: string
steps:
- bash: |
az login --identity --output none
az acr login --name ${{ parameters.registry_name }} --output none
Your acr-login.yaml must contain (one or multible) jobs since you are using it as job template:
parameters:
- name: registry_name
type: string
jobs:
- job: myStep
steps:
- bash: |
az login --identity --output none
az acr login --name ${{ parameters.registry_name }} --output none

azure devops yaml 'if' not evaluating as expected

Oh man, I'm having a really time with the yaml learning curve. Something that seems so simple, I just can't get to work.
I have this nested parameter group (not sure if that's the correct term) -
build: {
configuration: '',
nugetSource: '',
dotnetVersion: '',
projectsToPublish: '',
releaseCandidate: 'False',
tag: ''
}
I'm trying to then use an 'if' to set a variable for pool, dependent on the parameters.build.dotnetVersion passed in.
- job: build
displayName: Build & test
variables:
- ${{ if eq(parameters.build.dotnetVersion,6.0.200) }}:
- name: buildpool
value: build-dotnet6
- ${{ if ne(parameters.build.dotnetVersion,6.0.200) }}:
- name: buildpool
value: build-default
pool: $(buildpool)
So even though I have confirmed with an echo command that the value of parameters.build.dotnetVersion is 6.0.200, buildpool is always getting set to build-default.
I also tried wrapping 6.0.200 in single quotes, but didn't work as expected.

Ansible write on file from jinja file

I want to lanch a application :
command=/usr/bin/toto --config /var/toto/conf1.json /var/toto/conf2.json /var/toto/conf3.json
the config file are on /var/toto directory
task.yml
- name: Ansible find file
find:
paths: "/var/toto"
register: found_files
- name: print files
debug:
msg: "{{ found_files['files'] | map(attribute='path') | map('regex_replace','^.*/(.*)$','\\1') | list }}"
register: file_name
- name: Create the Jinja2 based template
template:
src: "etc/control/config.conf.j2"
dest: "/etc/control/config.conf"
with_dict: "{{ file_name }}"
config.conf.j2
command=/usr/bin/toto --config {% for item in file_name %} /var/toto/{{ item }} {% endfor %}
but, I have get this on my file
/etc/control/config.conf
command=/usr/bin/toto --config /var/opt/msg /var/opt/failed /var/opt/changed
varfile_name :
"msg": [ "conf1.json", "conf2.json", "conf3.json"
]
You're iterating over the dictionary in file_name, which is a task result. If you were to print that out, you would find that it contains something like:
TASK [debug] *********************************************************************************************************************************************************************************
ok: [localhost] => {
"file_name": {
"changed": false,
"failed": false,
"msg": [
"file1",
"file2",
"file3"
]
}
}
So when you iterate over it using for item in file_name, you're iterating over the top level keys (changed, failed, msg).
But this is all the wrong way to do it. You never use the debug module to create variables; that's what set_fact is for. You want something like:
- name: build list of files
set_fact:
file_name: "{{ found_files['files'] | map(attribute='path') | map('regex_replace','^.*/(.*)$','\\1') | list }}"
- name: Create the Jinja2 based template
template:
src: "config.conf.j2"
dest: "config.conf"
After the set_fact task, the variable file_name will contain a
list of file names.
It looks like you're using that regular expression to get the basename
of the files found in your find task. There's a basename filter
that can do that with less complexity:
- name: print files
set_fact:
file_name: "{{ found_files['files'] | map(attribute='path') | map('basename') | list }}"
- name: Create the Jinja2 based template
template:
src: "config.conf.j2"
dest: "config.conf"
Here's the playbook I'm using to test this locally:
- hosts: localhost
gather_facts: true
tasks:
- name: find files
find:
paths: /tmp/example
register: found_files
- name: print files
set_fact:
file_name: "{{ found_files['files'] | map(attribute='path') | map('basename') | list }}"
- name: Create the Jinja2 based template
template:
src: "config.conf.j2"
dest: "config.conf"
Before running this, I run:
mkdir /tmp/example
touch /tmp/exampe/file{1,2,3}
This produces a config.conf file that looks like:
command=/usr/bin/toto --config /var/toto/file3 /var/toto/file2 /var/toto/file1

Using variables from one yml file in another playbook

I am new to ansible and am trying to use variables from a vars.yml file in a playbook.yml file.
vars.yml
---
- firstvar:
id: 1
name: One
- secondvar:
id: 2
name: two
playbook.yml
---
- hosts: localhost
tasks:
- name: Import vars
include_vars:
file: ./vars.yml
name: vardata
- name: Use FirstVar
iso_vlan:
vlan_id: "{{ vardata.firstvar.id }}"
name: "{{ vardata.firstvar.name }}"
state: present
- name: Use Secondvar
iso_vlan:
vlan_id: "{{ vardata.secondvar.id }}"
name: "{{ vardata.secondvar.name }}"
state: present
So you can see here I am treating the imported variable data, which is stored in vardata, as object and trying to call each of them in other tasks. I am pretty sure these imported vars at the first task are only available in that very task. How can I use that in other tasks? It would output as variables undefined for each tasks. Any input is appreciated.
Your vars.yml file isn't formatted correctly.
Try this:
---
firstvar:
id: 1
name: One
secondvar:
id: 2
name: two
I used this to test it:
---
- hosts: localhost
tasks:
- name: Import vars
include_vars:
file: ./vars.yml
name: vardata
- name: debug
debug:
msg: "{{ vardata.firstvar.name }}"
- name: more debug
debug:
msg: "{{ vardata.secondvar.id }}"
On top of the error you made when declaring the variables (syntax is very important), you can also define include_vars: ./vars.yml such that you can just call {{ firstvar.name }}, {{ firstvar.id }} immediately. Much more leaner/shorter.