I know there is way to declare rules that need to be executed on the local machine using localrules as:
localrules: all, foo
Is there a similar option to declare rules that need to be executed on the cluster? Perhaps a clusterrules option?
I have a bunch of rules in my pipeline that don't need to be executed on the cluster and while I can list them all with localrules, it will be easier to just enter the one or two rules that need to be executed on the cluster.
An alternative option is the use of rule groups that will execute all rules from a group on a single node instead of using multiple nodes in the cluster.
I do not think this is supported unfortunately. I checked the snakemake source code, and one way to hack this is do to something like this:
all_rules = [rule for rule in dir(rules) if not rule.startswith("__")]
cluster_rules = ["my_cluster_rule1", "my_cluster_rule2"]
workflow._localrules = set(rule for rule in all_rules if rule not in cluster_rules)
I haven't tested it but I think this should work. This way we just overwrite what Snakemake parsed from the document. The problem with doing something like this is that it might not be stable between different snakemake versions.
Related
I use SQLFluff to ensure a uniform syntax in the company and reduce error warnings before running the models in dbt. Since our syntax does not completely match the syntax of SQLFluff I would like to make some changes.
The Rules References provided by SQLFluff helped me to set up Inline Ignoring Errors, as displayed in the code below (last line of code).
So I have two questions, which I was not able to answer also with the help of the Rules References of SQLFluff.
I would like to set Rule L032 as default 'false' without typing it manually every time in my SQL.
How do I change the maximal length of a line regarding Rule L016? I would like to set the default value e.g. 150.
SELECT
country.country_name,
country.population,
currency.currency_name,
currency.currency_id,
currency.strange_long_variable_name_which_is_too_long as not_so_long_variable_name
FROM country
LEFT JOIN currency
USING (country) -- noqa: L032
I tried to figure it out with the Rules References but could not figure it out. Help is very much appreciated!
Try looking into .sqlfluff config file
https://docs.sqlfluff.com/en/stable/configuration.html#
With the help of #suhprano's answer, I was able to find the proper solution for my issue. For this reason, I will post an answer to my own question. I do this intending to provide others assistant with similar issues.
I created the .sqlfluff file in my user profile folder. In this file I have then included the following:
[sqlfluff]
exclude_rules = L032
[sqlfluff:rules]
max_line_length = 150
In this case, SQLFluff will load the configuration from any .sql file found at the path specified on this variable.
Just an addition to the answer:
The default config of the rules can be found inside the package in file core\default_config.cfg
See also:
https://github.com/sqlfluff/sqlfluff/blob/main/src/sqlfluff/core/default_config.cfg
https://docs.sqlfluff.com/en/stable/configuration.html#defaultconfig
As already mentioned by #Albin the easiest way to override the config is to add a .sqlfluff file in the user profile folder.
See also:
https://docs.sqlfluff.com/en/stable/configuration.html#rule-configuration
Here's a problem.
I'm building a dashboard in Apache Zeppelin using org.postgresql.Driver to connect to Greenplum database. Usually I use something like:
where country_title like '${country=UK,UK|USA|Canada|%}'
to pass parameters into my query. But I have a long list of other parameters (like "towns") which I also need to pass to my query. And they are different for different country_title values. I can use something like:
select towns from towns_by_country where country_title like '${country=UK,UK|USA|Canada|%}'
to get a list of towns. How can I use it as a parameter in drop-down menu (WITH '%', of course).
I can use z.angularBind or z.put & z.get , of course. But they work in %spark interpreters, not in custom interpreters.
I would be very grateful for an answer or any other constructive feedback.
I have a content directory called foo and I want all files under that directory to have an extra metadata item foovar: default, unless explicitly overridden in the file header. I think I'm supposed to do this with EXTRA_PATH_METADATA, but I can't figure out what incantation it wants.
(for my current use case I'm trying to apply template: sometemplate within this dir, but I'm interested in solving the general case as it would make several related headaches go away)
I think what you're looking for is actually DEFAULT_METADATA. Check out this portion of the documentation:
DEFAULT_METADATA = {}
The default metadata you want to use for all articles and pages.
So, in your case it might look something like this in your config file:
DEFAULT_METADATA = {'foovar': 'default'}
Then to assign your custom template(s), see this portion of the documentation.
This wasn't possible at the time I asked. I've since sent the devs a PR adding support, and it's been merged to master. Presumably it will go out in the next release. It makes EXTRA_PATH_METADATA recursive, so you can apply settings to a subdir like this:
EXTRA_PATH_METADATA = {'dirname/subdir': {'status': 'hidden'}}
I need a rule to redirect only certain terms.
'<view:(about)>'=>'site/page/view/<view>',
'<view:(faq)>'=>'site/page/view/<view>',
'<view:(terms)>'=>'site/page/view/<view>',
I cannot use
<view:\w+>'=>'site/page/view/<view>
because i am using another rule for all terms other than this.
So is there any way to write an expression to short the 3 line of code to single one like this.
<view:(about),(faq),(terms)>'=>'site/page/view/<view>
Have you tried this?
<view:(about|faq|terms)>
I've got a website with a large directory tree. At some levels there are numerically-named directories, such as 123,45, 67,8 and 67,21. I'd like to sort these such that 123,45 comes after 67,*. Ideally, 67,8 would come before 67,21 as well:
67,8
67,21
123,45
(Note: ',' could as easily be '.', '-', etc.)
Is there a simple way to accomplish this within Apache (either directly or by creating a small plugin for it), or am I going to have to turn the site into some kind of CGI-based thing to make it sort numerically?
Just need add V=1 parameter, then directory will be displayed in version sorting mode.
example:
http://www.yoursite.com/?V=1
There are more detail information about parameters of apache autoindex module in http://httpd.apache.org/docs/2.2/mod/mod_autoindex.html#page-header.