How to add keywords like “lateral view explode” in Babel parser - sql

I want to parse a SQL statements (ANSI SQL or HiveQL) into equivalent AST. When I try to parse statements with “lateral view explode” keywords in it, which is a valid HiveQL syntax, Babel fails with ParseException. Adding these as keywords to the default list of keywords for Babel also does not help. Can someone point me to an example where something similar has been done.

Calcite does support the lateral keyword, but it does not support the "view explode" keywords.:
https://github.com/apache/calcite/blob/master/core/src/main/codegen/templates/Parser.jj#L2083
You could extend the parser, and might be able to use the free-marker support to skip the unsupported keywords ( I haven't tried it myself ):
https://calcite.apache.org/docs/adapter.html#extending-the-parser
However, if you need to access it through the corresponding SqlNode implementation, then it will require contribution given you would need to modify the core module.
more about parser.jj:
https://stackoverflow.com/a/44467850/1332098

Related

Using tags to exclude group of features [duplicate]

This question already has an answer here:
Can you use wildcard characters with tags to get all matching tags
(1 answer)
Closed 1 year ago.
We are using tags to be able to group features and senarios. For example, we have something like:
#jira=123
Scenario: test scenario 1
...
#jira=456
Scenario: test scenario 2, known failure
...
Scenario: test scenario 3, new feature
Now, we are hoping to run test that are not tagged with #jira=123 or #jira=456. Because we have many features and scenarios tagged with the #jira=somevalue, it is impractical to add them all. So I am looking for a way to be able to exclude anything tagged with #jira. I tried ~#jira and "~#jira=" but no luck.
Looking at the following junit case:
TagTest.java#testToString()
Which is using "#foo=" as a tag, but was not able to find an example. Is there a way to exclude a group of scenarios tagged by #jira, regardless of the tag value ?
The tag value is the whole string, even if it contains a = and you may assume there is key and a value.
But you could consider to use multiple tags, they are allowed.
So, in your case, I would use something like:
#jira=123
#jira
Scenario: test scenario 1
...
#jira=456
#jira
Scenario: test scenario 2, known failure
And the you can use the ~#jira to exclude all the #jira scenarios.
This will allow you to still reference the single #jira=123 when needed.
Yes, we haven't documented this well but this question here can be a start. Karate actually supports a mini expression language for tags.
Have a look at this test for some options: TagsTest.java
And this should work for your requirement, do confirm in the comments ! Yes just use the string below where you would normally put #jira etc.
!valuesFor('#jira').isPresent
One more important point. When you use the special expression language, any AND or OR complexity has to be managed within the single expression that you pass into the tags option. Only one expression is needed and the use of comma-separated values or multiple values for the tag parameter is not applicable.
For example:
To select scenarios that have values for either the #fail tag or the #bad tag (note the use of the JS || (OR) operator):
valuesFor('#fail').isPresent || valuesFor('#bad').isPresent
And to select any scenario that has values for the #fail tag and where the #smoke tag is present (without values, just the plain tag and no = part):
valuesFor('#fail').isPresent && anyOf('#smoke')
And yes, you can use the "expression language" on the command-line i.e. within the karate.options or as the --tags or -t option to the stand-alone JAR: https://stackoverflow.com/a/72054253/143475

Apache Zeppelin: passing parameters between different interpreters

Here's a problem.
I'm building a dashboard in Apache Zeppelin using org.postgresql.Driver to connect to Greenplum database. Usually I use something like:
where country_title like '${country=UK,UK|USA|Canada|%}'
to pass parameters into my query. But I have a long list of other parameters (like "towns") which I also need to pass to my query. And they are different for different country_title values. I can use something like:
select towns from towns_by_country where country_title like '${country=UK,UK|USA|Canada|%}'
to get a list of towns. How can I use it as a parameter in drop-down menu (WITH '%', of course).
I can use z.angularBind or z.put & z.get , of course. But they work in %spark interpreters, not in custom interpreters.
I would be very grateful for an answer or any other constructive feedback.

How do I write a robust structural search template to report Mockito times(1)/Times(1) passed to verify in IntelliJ IDEA?

In my project Mockito.times(1) is often used when verifying mocks:
verify(mock, times(1)).call();
This is redundant since Mockito uses implicit times(1) for verify(Object), thus the following code does exactly what the code above does:
verify(mock).call();
So I'm going to write an a structural search drive inspection to report such cases (let's say, named something like Mockito.times(1) is redundant). As I'm not an expert in IntelliJ IDEA structural search, my first attempt was:
Mockito.times(1)
Obviously, this is not a good seach template because it ignores the call-site. Let's say, I find it useful for the following code and I would not like the inspection to trigger:
VerificationMode times = Mockito.times(1);
// ^ unwanted "Mockito.times(1) is redundant"
So now I would like to define the context where I would like the inspection to trigger. Now the inspection search template becomes:
Mockito.verify($mock$, Mockito.times(1))
Great! Now code like verify(mock, times(1)).call() is reported fine (if times was statically imported from org.mockito.Mockito). But there is also one thing. Mockito.times actually comes from its VerificationModeFactory class where such verification modes are grouped, so the following line is ignored by the inspection:
verify(mockSupplier, VerificationModeFactory.times(1)).get();
My another attempt to fix this one was something like:
Mockito.verify($mock$, $times$(1))
where:
$mock$ is still a default template variable;
$times$ is a variable with Text/regexp set to times, Whole words only and Value is read are set to true, and Expression type (regexp) is set to (Times|VerificationMode) -- at least this is the way I believed it should work.
Can't make it work. Why is Times also included to the regexp? This is the real implementation of *.times(int), so, ideally, the following line should be reported too:
verify(mockSupplier, new Times(1)).get();
Of course, I could create all three inspection templates, but is it possible to create such a template using single search template and what am I missing when configuring the $times$ variable?
(I'm using IntelliJ IDEA Community Edition 2016.1.1)
Try the following search query:
Mockito.verify($mock$, $Qualifier$.times(1))
With $Qualifier$ text/regexp VerificationModeFactory|Mockito and occurrences count 0,1 (to find it when statically imported also).
To also match new Times(1) you can use the following query:
Mockito.verify($mock$, $times$)
With $times$ text/regexp .*times\s*\(\s*1\s*\) and uncheck the Case sensitive checkbox.

Avoid duplicates in the destination schema

I have a little problem. I want to map every detail line to one OrderInfo. The destination schema can not have any duplicate OrderInfo. All the detail lines should be in the destination orderInfo, but the SuppliersOrderNo and BuyersOrderNo should not be twice.
Any ideas how to do this, is it possible to use XSL or inline script?
<inv:OrderInfo>
<inv:SuppliersOrderNo>123456</inv:SuppliersOrderNo>
<inv:BuyersOrderNo>6789</inv:BuyersOrderNo>
<inv:DetailLines>
<inv:DetailLine>
<inv:InvoiceDetailLineNo>1</inv:InvoiceDetailLineNo>
<inv:Item>
<inv:SuppliersArticleNo>article2</inv:SuppliersArticleNo>
<inv:SuppliersDescription>BestArticle</inv:SuppliersDescription>
</inv:Item>
</inv:DetailLine>
<inv:DetailLine>
<inv:InvoiceDetailLineNo>2</inv:InvoiceDetailLineNo>
<inv:Item>
<inv:SuppliersArticleNo>article3</inv:SuppliersArticleNo>
<inv:SuppliersDescription>AlmostBestArticle</inv:SuppliersDescription>
</inv:Item>
</inv:DetailLine>
</inv:DetailLines>
</inv:OrderInfo>
<inv:OrderInfo>
<inv:SuppliersOrderNo>123456</inv:SuppliersOrderNo>
<inv:BuyersOrderNo>6789</inv:BuyersOrderNo>
<inv:DetailLines>
<inv:DetailLine>
<inv:InvoiceDetailLineNo>1</inv:InvoiceDetailLineNo>
<inv:Item>
<inv:SuppliersArticleNo>article1337</inv:SuppliersArticleNo>
<inv:SuppliersDescription>WOW</inv:SuppliersDescription>
</inv:Item>
</inv:DetailLine>
</inv:DetailLines>
</inv:OrderInfo>
If you want to do this purely in XSLT, you'll have to use Muenchian gruoping. I wrote a blog that links to some other blogs on how to do this in BizTalk a little while back: https://blog.tallan.com/2014/12/09/muenchian-grouping-in-biztalk-while-keeping-mapper-functionality/
To summarize the blog: if you pursue this, you'll need a map that's completely custom XSLT somewhere, but you could put it into a custom pipeline component if you still want to be able to use "regular" maps functionality without any other caveats (in my blog I describe a method of doing that in a pipeline component so that a "regular" BizTalk map can still be used on the preprocessed output). There are lots of resources on Muenchian grouping out there (including on StackOverflow), so I'm not rehashing all of that in this answer.
You could also try to serialize the message in a C# component and use some LINQ methods to group/sort/order/etc, or if you're inserting the content into SQL at some point you could do it in SQL (which would be able to handle this kind of task more naturally).

Can i use xpath-like expression in the attributevalue in a xacml plicy

I'd like to declare some policies likes:
some one can visit anything under the img path, but img folders are scattered everwhere, so the attributevalue in the xacml policy may seem like this: "/rootpath/**XPATH_PART**/img/*".
how to write policy of this kind.
I looked through the "XACML3.0 core spec", "Multiple Profile", they says
Each Individual Decision Request SHALL be identical to the original request context with two exceptions: the “multiple:content-selector” attribute SHALL NOT be present and an added “content-selector” attribute value SHALL be an XPath expression that evaluates to a single node in the <Content> element
I think this means that in the policy file, i cann't use XPath in the AttributeValue to refer to multiple resources like i said in the first place, right? because the request is resolved to individual request each asking for a resource with a specified attribute id.
Is there something in the specification i missed out or misunderstood? or can anyone suggest a better way to do what i want?
Now i'm wondering if using regular expression in the resource can do that. The corresponding function is
urn:oasis:names:tc:xacml:1.0:function:string-regexp-match.
P.S.: I'm trying to setup a authorization server for my company, XACML seems a good place to start with. But nobody around me knows about it. I would be appreciated if any one can give me any suggestion about setting up the access control system.
I chatted with some of my colleagues at Axiomatics and the conclusion is that you do not need XPath but rather regular expressions. XACML provides a regular expression function that works on URI data types. It is called anyURIRegexpMatch and it takes in a string (the regular expression) and the XACML attribute to which to apply the regular expression. It returns either true or false.
Your rule target would look as follows in ALFA (Axiomatics Language for Authorization):
policy matchResources{
apply firstApplicable
rule allow{
target clause anyURIRegexpMatch("^https?://(?:[a-z\\-]+\\.)+[a-z]{2,6}(?:/[^/#?]+)+\\.(?:jpg|gif|png)$", resourceId)
permit
}
}
See also this other example (XACML 2.0): How do I apply XACML rules to every child URI?