Based on Nested variables in dbt_project.yml file of dbt
vars:
env: env_var('DBT_ENVIRONMENT')
bucket:
dev: "dev-bucket"
uat: "uat-bucket"
ppd: "ppd-bucket"
prd: "prd-bucket"
I would like to access to the bucket name based on the value of the env var env.
For example I would reference the dev bucket like:
{{ var('bucket')['dev'] }}
But I want instead to call it parametric based on the var env with something similar:
{{ var('bucket')[ {{ var('env') }} ] }}
It's just that with this I get the following error message:
Compilation Error Could not render {{ var('bucket')[ {{ var('env') }} ] }}
and I don't know if this is possible to reference with DBT syntax.
Any suggestions on how to achieve this?
A possible solution can be
{{ var('bucket')[ env_var('DBT_ENVIRONMENT') ] }}
but still I'd prefer to reference directly var('env') from the dbt_project.yml, so open to better solutions.
Related
Oh man, I'm having a really time with the yaml learning curve. Something that seems so simple, I just can't get to work.
I have this nested parameter group (not sure if that's the correct term) -
build: {
configuration: '',
nugetSource: '',
dotnetVersion: '',
projectsToPublish: '',
releaseCandidate: 'False',
tag: ''
}
I'm trying to then use an 'if' to set a variable for pool, dependent on the parameters.build.dotnetVersion passed in.
- job: build
displayName: Build & test
variables:
- ${{ if eq(parameters.build.dotnetVersion,6.0.200) }}:
- name: buildpool
value: build-dotnet6
- ${{ if ne(parameters.build.dotnetVersion,6.0.200) }}:
- name: buildpool
value: build-default
pool: $(buildpool)
So even though I have confirmed with an echo command that the value of parameters.build.dotnetVersion is 6.0.200, buildpool is always getting set to build-default.
I also tried wrapping 6.0.200 in single quotes, but didn't work as expected.
I use schema registry 5.4 - https://hub.docker.com/r/confluentinc/cp-schema-registry/, and use helm chart https://github.com/helm/charts/tree/master/incubator/schema-registry to deploy.
To set my logger, I create a configmap and mount it to the schema registry container that uses the image -confluentinc/cp-schema-registry:5.4.0
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "schema-registry.fullname" . }}-log4j-configmap
labels:
app: {{ template "schema-registry.name" . }}
chart: {{ template "schema-registry.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
data:
log4j.properties: |+
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.encoding=UTF-8
But I don't know what should be changed in the command of the container, or there is a variable pointing to log4j.properties to set it?
for example I plan to change the command like the following:
command:
- java
- -Dlog4j.configuration=file:/etc/schema-registry-log4j/log4j.properties
- -jar
- schema-registry.jar
Option 1:
One way I found is to build a new image using FROM and put my custom log4j properties there.
https://docs.confluent.io/current/installation/docker/development.html#log-to-external-volumes
Option 2:
The other way is to set an environment variable
SCHEMA_REGISTRY_LOG4J_LOGGERS="org.apache.kafka=ERROR,io.confluent.rest.exceptions=FATAL"
I have a set of variables and a task as follows. My intent is to dynamically do a healthcheck based on the URL the user chose.
vars:
current_hostname: "{{ ansible_hostname }}"
hc_url1: "https://blah1.com/healthcheck"
hc_url2: "https://blah2.com/healthcheck"
tasks:
- name: Notification Msg For Healthcheck
shell: "echo 'Performing healthcheck at the URL {{ lookup('vars', component) }} on host {{ current_hostname }}'"
Run playbook in Ansible 2.3
ansible-playbook ansible_playbook.yml -i inventory -k -v --extra-vars "component=hc_url1"
Error
fatal: [hostname]: FAILED! => {"failed": true, "msg": "lookup plugin (vars) not found"}
I know this happens because lookup plugin "var" was introduced in Ansible v2.5. Is there a way to do this in Ansible 2.3? I want get the value of {{ component }}, and then the value of {{ hc_url1 }}
PS - upgrading to 2.5 is not an option because of org restrictions
Alternatively, maybe you can do this using a dictionary.
For example,
vars:
current_hostname: "{{ ansible_hostname }}"
urls:
hc_url1: "https://blah1.com/healthcheck"
hc_url2: "https://blah2.com/healthcheck"
tasks:
- name: Notification Msg For Healthcheck
shell: "echo 'Performing healthcheck at the URL {{ urls[component] }} on host {{ current_hostname }}'"
That way, the user provided value of component will just be looked up as a key in the dictionary.
I am trying to create local facts from variables.
My fact is:
datadog_http_checks:
- name : {{ env }} ResourceManager
url : http://{{ inventory_hostname }}:
threshold : 5
window : 5
timeout : 10
My task is:
- include_vars: clouderamanager.yml
- lineinfile: dest=/etc/ansible/facts.d/datadog_http_checks.fact line={{ datadog_http_checks }} create=yes
Which doesn't create the local facts, it fails with following error
TASK [hadoop : lineinfile] *****************************************************
fatal: [hmn001.dev.abc.cc]: FAILED! => {"changed": false, "failed": true, "module_stderr": "", "module_stdout": "Traceback (most recent call last):\r\n File \"/home/xyz/.ansible/tmp/ansible-tmp-1460581285.88-252089462921262/lineinfile\", line 2540, in <module>\r\n main()\r\n File \"/home/abc/.ansible/tmp/ansible-tmp-1460581285.88-252089462921262/lineinfile\", line 371, in main\r\n ins_aft, ins_bef, create, backup, backrefs)\r\n File \"/home/abc/.ansible/tmp/ansible-tmp-1460581285.88-252089462921262/lineinfile\", line 266, in present\r\n lines.append(line + os.linesep)\r\nTypeError: can only concatenate list (not \"str\") to list\r\n", "msg": "MODULE FAILURE", "parsed": false}
Lineinfile does exactly what it says it does: it changes a single line in a file.
If you want to create a local fact that looks like:
datadog_http_checks:
- name : {{ env }} ResourceManager
url : http://{{ inventory_hostname }}:
threshold : 5
window : 5
timeout : 10
Then you need to create a file that looks like this:
[datadog_http_checks]
name={{ env }} ResourceManager
url=http://{{ inventory_hostname }}:
threshold=5
window=5
timeout=10
You can do this with the template module if, like in your example, you have variables in it that you want to dynamically build.
In this exact scenario though I'm confused as to what the benefit of this is rather than having a variable set up as you have in your question and using that rather than templating out a local fact file and then re-reading it later.
I have a task:
- name: Copy celeryconfig.py to "proj_dir/monitor/"
copy:
src="templates/repo/celeryconfig.py.j2"
dest={{proj_dir}}/monitor/celeryconfig.py
run_once: true
Variables that is stored it vars\mail.yml. Inside this file I have rabbitmq_app_user, rabbitmq_app_pass, rabbitmq_app_vhost are defined.
And template file:
BROKER_URL = "apmq://{{rabbitmq_app_user}}:{{rabbitmq_app_pass}}#IP/{{rabbitmq_app_vhost}}"
But when I run the playbook, result looks exactly same as what in inside the template file. Seems the way I try to access to variables that are defined in /vars/main.yml is incorrect. What's the proper way of accessing to variables in my case?
If you want to use jinja template in Ansible you have to use the template module as well. Try something like this:
- name: Copy celeryconfig.py to "proj_dir/monitor/"
template:
src: "repo/celeryconfig.py.j2"
dest: "{{ proj_dir }}/monitor/celeryconfig.py"
run_once: true