How can I generate different SQL for dbt models during train vs test runs? - dbt

Question

This question came up on dbt Slack (I've paraphrased here):
I want my dbt models to use different logic depending on whether I'm training or testing a data set. Here is my current code, which uses SQL variables:
SELECT
*
FROM tracks tr
WHERE
event_timestamp >= (CURRENT_DATE - INTERVAL %(from_inclusive)s) -
INTERVAL '60 days'
from_inclusive is a variable taken from this json:
{
"train": {
"from_inclusive": "15 days"
},
"train-extra": {
"from_inclusive": "1 days"
},
"eval-test": {
"from_inclusive": "1 days"
}
}
How can I write a dbt model so that from_inclusive compiles different SQL depending on a variable setting?

Answer 1

I think you can write a macro to do this.
Off the top of my head, the macro could accept one argument (e.g., "train" or "test"). The macro could then contain an if else statement and return the appropriate number of days depending on the argument. That macro could be used throughout your project and would result in appropriately compiled SQL.
Assuming you have some way of the model knowing whether it's a train or test (or other) run, you could feed the macro different arguments based on the schema and/or envi, like this:
select
*
from tracks tr
where event_timestamp >= (current_date - interval
{% if target.name == 'train' %}
{{ your_macro('train') }}
{%- else -%}
{{ your_macro('test') }}
{% endif %}
) - interval '60 days'
I used target as an example, but you could do the same thing with other states or configs to determine the argument that goes into the macro.
Your macro (your_macro) would then be stored in the macros subdirectory and would look like this:
{% macro your_macro(state) %}
{% if state == 'train' %}
"15 days"
{% elif state == 'test' %}
"1 day"
{% else %}
"0 days"
{% endif %}
{% endmacro %}