I have a generic test and need it to be always saved under a particular name for the given table it is running on, e.g. on table report_revenue the generated generic test name will always be diff_check_report_revenue. Right now the default dbt naming behavior for generic tests is kinda messy (it sets the test name based on the test config, which is a great idea for most cases, but unfortunately not for my particular one).
According to the docs it looks like the [alias]https://docs.getdbt.com/reference/resource-configs/alias is exactly what I need. However, I also need to set the name dynamically based on the table that is tested. I know it can be set in the yml config by setting the field alias, but I hope there might be a more elegant solution.
When I try the following:
{{
config({
"severity": 'warn',
"tags": ["diff_check"],
"alias": 'diff_check_' + {{ model | replace("XXXXXXX") | trim }}
})
}}
It just doesn't work and dbt completely ignores the alias property. model is the relation on which the test is running. It's probably just my own wrong syntax, but I'm stuck and humbly asking for advice. Thanks a lot in advance!
The docs are super confusing on test config, since they group together generic tests and singular tests, and the behavior is different.
You can use a config() block inside the definition for a generic test to configure it, and some keys (e.g., severity) work fine, but alias is not one of them.
I think alias is meant for singular tests only. To give generic tests a unique identifier (only possible since v1.1), you are supposed to use the name property (not config). Docs. Does this make sense? No. Does it make it easy for you to do what you want to do? Also no.
I'll point out that the default naming convention for a generic test includes the name of the test followed by the name of the model, but assuming that isn't good enough, your only option will be to add a name property to every test, where you define the test in the properties (fka schema.yml) file. And it looks like the name property doesn't jinja-template its value (so you can't use jinja to populate the test name). So you'll have to do a lot of this:
models:
- name: my_model
tests:
- diff_check:
name: diff_check_my_model
You could fork dbt-core. The relevant code is here.
Related
I'm trying to have a structure to run a dbt project, where I have multiple entities (bank, names, cars). I'm going to have exactly the same code for them all.
Based on that, I'm trying to have several folder with the same code, where I can define inside the dbt_project.yamlfile. The idea is something like this:
vars:
db_name: 'db_official'
staging:
bank: 'variable_bank'
car: 'variable_car'
name: 'variable_name'
The variable "db_name" works. So, my the two problems I'm having are:
How to have this structure inside the the yaml file?
How to reference this structure inside each file?
(extra) Any other ideas how to handle this?
Thanks!
vars are basically globals. They can be scoped to your whole project, or to a package within your project, but not more specifically than that (they share a flat namespace). See the docs.
I would pull out the common code into a macro, then call that macro from each model file, passing in the unique values as string literals in the model file:
-- models/staging/bank.sql
{{ my_model_template('variable_bank') }}
-- models/staging/car.sql
{{ my_model_template('variable_car') }}
I'd like to enable a test if a certain tag is "included", i.e. passed with option --include-tag of the ConsoleLauncher or useJUnitPlatform.includeTags property in Gradle. Is there any API to retrieve the value of this option in the context of test class or method?
I tried the script-based condition #EnabledIf like this:
#EnabledIf("'true' == systemProperty.get('itest.backendSystemPresent') || junitTags.contains('BackendSystemIT') == true")
But junitTags contains the #Tag annotations of the element in question, not the tags included at runtime.
Reading your question again, my answer is "No". You can't use junitTags to achieve your goals. And no, there's no such API at the moment. You would need something like:
#EnabledIf("'true' == evaluateTagExpression('BackendSystemIT') || ...)
Because you need to take care of tag expression here as well: https://junit.org/junit5/docs/current/user-guide/#running-tests-tag-expressions
But, tags are evaluated earlier in the process. Your condition will not get a chance to be executed when the test was already excluded by tag evaluation. So, I guess, you'll have to stick with the single system property switch to control the enabled state of the test method.
Btw. we are improving the tag expression language with any() and none() tokens, soon. https://github.com/junit-team/junit5/issues/1679
Possible solution:
Annotate your test with #Tag("BackendSystemIT")
Before running your tests, check for itest.backendSystemPresent system property and if it is set, pass a --include-tag "BackendSystemIT" to the test run.
Let Jupiter do the job of evaluating tag expressions
Is there any API to retrieve the value (of this option) of all tags that are attached directly or inherited in the context of test class or method?
Yes. Declare and use a org.junit.jupiter.api.TestInfo parameter in your test method.
#Test
#DisplayName("TEST 1")
#Tag("my-tag")
void test1(TestInfo testInfo) {
assertEquals("TEST 1", testInfo.getDisplayName());
assertTrue(testInfo.getTags().contains("my-tag"));
}
For details see https://junit.org/junit5/docs/current/user-guide/#writing-tests-dependency-injection
But junitTags contains the #Tag annotations of the element in question, not the tags included at runtime.
This is the expected behaviour -- the platform (here: console launcher) already applied the filter passed via --include-tag and other configuration parameters. In short: there's no need to manually check for tags in standard Jupiter tests. If there's problem with the built-in filtering, please create an issue here: https://github.com/junit-team/junit5/issues/new/choose
I faced with the problem of writing my vcs current branch name each time I have written 'todo' comment.
Recently I learned about Intellij's 'Live Templates' which is quite comfortable to use. I tried to apply it to my problem but there's no templates to take out a branch name.
So the question is could I actually take out the name of my branch to code comments somehow?
It is possible to use the groovyScript predefined function and a script to extract the branch name. For example create the following live template:
$COMMENT$ todo [$BRANCH$]: $END$
with abbreviation "todo" and description "Inserts todo comment with branch name". Click Edit variables and give the variables the following definitions:
COMMENT:
lineCommentStart()
BRANCH (updated for 2020.2 and newer)
groovyScript("com.intellij.dvcs.repo.VcsRepositoryManager.getInstance(_editor.project).getRepositoryForFileQuick(com.intellij.openapi.fileEditor.FileDocumentManager.getInstance().getFile(_editor.document)).getCurrentBranchName()")
Skip if defined checked for both variables. The Groovy script is (unfortunately) all one line. Set applicable contexts to Everywhere.
With this live template it is now possible to type todoTab somewhere in a source file and a line comment with the branch name will be inserted. This will insert the proper line comment depending on the language of the file, or nothing in case of languages without a line comment like HTML. And should extract the branch name no matter the type of version control used (I tested with Git).
For live templates you can use predefined functions. Unfortunately there is no function to detect the current VCS branch.
But you can create a template to make work a little easier:
// TODO [$branch_name$]: $comment$
With this template, you still have to fill branch name, but you should not type symbols like [ and caret will be placed automatically.
You can also create a feature request for a new predefined function.
In my project Mockito.times(1) is often used when verifying mocks:
verify(mock, times(1)).call();
This is redundant since Mockito uses implicit times(1) for verify(Object), thus the following code does exactly what the code above does:
verify(mock).call();
So I'm going to write an a structural search drive inspection to report such cases (let's say, named something like Mockito.times(1) is redundant). As I'm not an expert in IntelliJ IDEA structural search, my first attempt was:
Mockito.times(1)
Obviously, this is not a good seach template because it ignores the call-site. Let's say, I find it useful for the following code and I would not like the inspection to trigger:
VerificationMode times = Mockito.times(1);
// ^ unwanted "Mockito.times(1) is redundant"
So now I would like to define the context where I would like the inspection to trigger. Now the inspection search template becomes:
Mockito.verify($mock$, Mockito.times(1))
Great! Now code like verify(mock, times(1)).call() is reported fine (if times was statically imported from org.mockito.Mockito). But there is also one thing. Mockito.times actually comes from its VerificationModeFactory class where such verification modes are grouped, so the following line is ignored by the inspection:
verify(mockSupplier, VerificationModeFactory.times(1)).get();
My another attempt to fix this one was something like:
Mockito.verify($mock$, $times$(1))
where:
$mock$ is still a default template variable;
$times$ is a variable with Text/regexp set to times, Whole words only and Value is read are set to true, and Expression type (regexp) is set to (Times|VerificationMode) -- at least this is the way I believed it should work.
Can't make it work. Why is Times also included to the regexp? This is the real implementation of *.times(int), so, ideally, the following line should be reported too:
verify(mockSupplier, new Times(1)).get();
Of course, I could create all three inspection templates, but is it possible to create such a template using single search template and what am I missing when configuring the $times$ variable?
(I'm using IntelliJ IDEA Community Edition 2016.1.1)
Try the following search query:
Mockito.verify($mock$, $Qualifier$.times(1))
With $Qualifier$ text/regexp VerificationModeFactory|Mockito and occurrences count 0,1 (to find it when statically imported also).
To also match new Times(1) you can use the following query:
Mockito.verify($mock$, $times$)
With $times$ text/regexp .*times\s*\(\s*1\s*\) and uncheck the Case sensitive checkbox.
I can't figure out how to get variables into Sphinx documents via a template. I am certainly making an elementary mistake, but while there is lots of documentation for using Jinja/Flask templates for web service and some documentation for Sphinx using it, I am having trouble doing the following. Maybe it's not possible and I'm barking up the wrong tree, but then this is fairly different from how variables work in general in web (as opposed to doc) templates?
I am working within a much larger project. Suppose in my project's conf.py I make a variable, say
LANGS = ['en', 'de', 'cn']
I know that this works because if I do the docbuild (we have a custom docbuild but I don't think it does anything really crazy other than a customized logger and eating a bunch of 'chatter') with the following line in conf.py
print len(LANGS)
it shows up during the docbuild.
But now of course I want to access this variable in my template. As far as I can tell, we override index.rst with templates/index.html, which inherits from the basic layout.html for Sphinx. And if I do
<p>We have {{ LANGS|len }} languages</p>
I get
We have 0 languages
Now, this is weird, because sometimes I can cause an error in the build by referring to variables not defined (though not consistently), so that somehow it 'knows' that the variable is defined but thinks it has length zero. Or does a "null" variable have length zero automatically?
How do I get this variable defined - or is it not possible?
What I want to do is then do something for each language in the list (make an outside link, in particular), but I figure there is no point in trying {% for %}/{% endfor %} or whatever if I can't get this working. Maybe Sphinx implements only a subset of Jinja?
Anyway, please help!
There are at least two ways to pass variables to a template:
Via html_context:
A dictionary of values to pass into the template engine’s context for all pages. Single values can also be put in this dictionary using the -A command-line option of sphinx-build.
Example:
# conf.py:
html_context = {'num_langs': len(LANGS)}
<!-- template: -->
<p>We have {{ num_langs }} languages</p>
Via the html_theme_options. This requires adding an option to theme.conf (you can create a theme by inheriting from a standard one):
[options]
num_langs = 1
Then you can set num_langs in conf.py via html_theme_options:
html_theme_options = {'num_langs': len(LANGS)}
and use it in a template:
<p>We have {{ theme_num_langs }} languages</p>