I want to concatenate a few columns using column1 ^^ column2 ^^ ... syntax in DBT for Redshift. If there are NULL values in the columns ## should be used, resulting in f.e. ## ^^ ##. I have found the following macro for concatenation:
{% macro safe_concat(field_list) %}
{# Takes an input list and generates a concat() statement with each argument in the list safe_casted to a string and wrapped in an ifnull() #}
concat({% for f in field_list %}
ifnull(safe_cast({{ f }} as string), '##')
{% if not loop.last %}, {% endif %}
{% endfor %})
{% endmacro %}
When I use it in my select statement:
select
{{ safe_concat([street, city]) }} as address_key
from source
I get the following error. Is this related to the code I am using?
Database Error in model address (models/address.sql)
syntax error at or near "as"
LINE 32: ifnull(safe_cast( as string), '##')
Try wrapping your column names in quotes when you call them in the macro - I think it’s trying to pass in the variables street and city (because you’re already inside of curly braces), which don’t exist so are evaluating to None
you can try pushing every loop into an array and then you can use evaluated strings.and also for concat func. you can use '~' this.
{% set query_results = [] %}
{% for f in field_list %}
{% set x = ifnull(safe_cast({{ f }} as string), '##') ~ '^^' %}
{% if not loop.last %}, {% endif %}
{% set query_results = query_results.append(x) %}
{% endfor %}
...
return{{query_results }}
Related
Why does casting
select cast(st_makepoint(-90.345929, 37.278424) as geography)
raise the following error:
SQL compilation error: invalid type [CAST(ST_MAKEPOINT(TO_DOUBLE(-90.345929), TO_DOUBLE(37.278424)) AS GEOGRAPHY)] for parameter 'TO_GEOGRAPHY'
While a seemingly more direct pass of the st_makepoint result to to_geography does not?
select to_geography(st_makepoint(-90.345929, 37.278424))
I'm fairly sure I'm stuck with the casting behavior in the dbt tool I'm using. Basically I'm trying to union a bunch of tables with this geography field, and in the compiled SQL this casting logic appears as a function of dbt's union_relations macro, and I don't seem to be able to control whether the casting occurs.
The source for union_relations is here.
You can copy this macro into your own project (under the macros directory) and patch the source, and then call it with union_relations instead of dbt_utils.union_relations.
The offending lines are 106-113. Something like this should work fine:
{% for col_name in ordered_column_names -%}
{%- set col = column_superset[col_name] %}
{%- set col_type = column_override.get(col.column, col.data_type) %}
{%- set col_name = adapter.quote(col_name) if col_name in relation_columns[relation] else 'null' %}
{% if col_type == 'geography' %}
to_geography({{ col_name }}) as {{ col.quoted }}
{% else %}
cast({{ col_name }} as {{ col_type }}) as {{ col.quoted }}
{% endif %}
{%- if not loop.last %},{% endif -%}
{%- endfor %}
Because CAST doesn't support that particular combination of source and target datatypes
I am getting a TemplateSyntaxError: "could not parse remainder % 2 from num%2":
{% if num%2 ==0 %}
{{"Even"}}
{% else %}
{{"Odd"}}
{% endif %}
You can't use arbitrary Python expressions in Django templates. You should create a custom filter for them.
However, for your expression there is a built-in tag divisibleby. From its example:
{{ value|divisibleby:"2" }}
If value is 4, the output would be True. So the final answer looks like (untested):
{% if num|divisibleby:"2" %}
Even
{% else %}
Odd
{% endif %}
Problem
Currently in my CI process, I am surfacing specific models built to multiple schemas. This is generally my current process.
macros/surface_models.sql
{% set model_views = [] %}
{% for node in graph.nodes.values() %}
{% if some type of filtering criteria %}
{%- do model_tables.append( graph.node.alias ) -%}
{% endif %}
{% endfor %}
{% for view in model_views %}
{% set query %}
'create view my_other_schema.' ~ table ~ 'as (select * from initial_schema.' ~ table ~ ');'
{% endset %}
{{ run_query(query) }}
{% endfor %}
while this works, if the underlying table/view's definition changes, the view created from the above macro will return an error like: QUERY EXPECTED X COLUMNS BUT GOT Y
I could fix this by writing each query with each query's explicit names:
select id, updated_at from table
not
select * from table
Question
Is there a way to utilize the above macro concept but using {{ dbt_utils.star() }} instead of *?
i am trying to split the string in template using custom template filter. But i got an error
TemplateSyntaxError at /job/16/
'for' statements should use the format 'for x in y': for skill in form.instance.skills | split : ","
Here it is my filter
#register.filter(name='split')
def split(value, key):
"""
Returns the value turned into a list.
"""
return value.split(key)
this is my template
<h4>Skills</h4>
{% for skill in form.instance.skills | split : "," %}
{{ skill }}
{% endfor %}
Thanks
Split is a custom filter, don't forget to create your filter, and to load it in your HTML page.
Documentation for Django 4.0: https://docs.djangoproject.com/en/4.0/howto/custom-template-tags/
<h4>Skills</h4>
{% with form.instance.skills|split:"," as skills %}
{% for skill in skills %}
{{ skill }}<br>
{% endfor %}
{% endwith %}
For extract character string, use filter cut:
Phone
this removes the scripts from the string.
The direct for loop works too, you just have to remove the spaces in the syntax:
<h4>Skills</h4>
{% for skill in form.instance.skills|split:"," %}
{{ skill }}
{% endfor %}
I'm using Shopify and want to hook into customer tags, however they are case sensitive. So {% if customer.tags contains "wholesale" %} is not the same as {% if customer.tags contains "Wholesale" %}. My client may or may not stick to one case when applying tags so I want to guard against that in the future.
I would like to take an array, customer.tags, and convert all of the values to lowercase. I'm trying to work out the logic but am having trouble.
I want to put customer.tags into a new array which isn't working.
{% assign newArray = customer.tags %}
{{ newArray }}
What am I doing wrong?
You could use the downcase filter for this:
{% assign contains_wholesale = false %}
{% for tag in customer.tags %}
{% assign lowercase_tag = tag | downcase %}
{% if lowercase_tag == 'wholesale' %}
{% assign contains_wholesale = true %}
{% endif %}
{% endfor %}
Note: downcase just works for ASCII characters. If you need to search for strings with accented letters or other Unicode characters then this won't be enough.
if you would like to keep customer.tags as an array so you can keep using contains in a simple if statement (like your example). You could also chain a couple of liquid filters together to turn all of the strings within the array to lowercase.
Example:
{% assign lowercaseTags = customer.tags | join: ',' | downcase | split: ',' %}
{% assign randomString = 'WholeSale' | downcase %}
{% if lowerCaseTags contains randomString %}
{% comment %}
Will now match regardless of case sensitivity
{% endcomment %}
{% endif %
Explanation:
Join filter: Turn the array into a string seperated by ,
Downcase filter: make the whole string lowercase
Split filter: opposite of join, recreates the array from the string based on the character used in join ,
Another solution as you use the "contains" operator would be to skip the "w".
Something like {% if customer.tags contains 'holesale' %} should work.