I am creating some tool to dynamically generating MDX queries. In a part of the query I am generating I need to check whether an expression is a member expression or tuple expression and apply different logic on it. Does anyone have any clue about how I can check the MDX expression type at run time by just using MDX?
To know the exact type of an MDX expression, you would have to write an MDX parser (at least for the expressions that can appear).
There are some rules: something like (x, y) is probably a tuple; and the result of all methods that return a tuple (like StrToTuple, Root, or Item, the latter only if applied to a set) is a tuple, and the result of all methods that return a member (like Ancestor or DefaultMember, but also Item if applied to a tuple) is a member. See http://msdn.microsoft.com/en-us/library/ms145970.aspx for a list of functions classified by type. But you see already the difficulty of the Item method which can either deliver a tuple, or a member, depending on context.
And I would not think that you can easily write an MDX statement that tests the type, as Analysis Services uses automatic type casting which converts a member to a tuple whenever the context needs one.
The best approach from my point of view would be to use syntax that allows both a member and a tuple being used, and to avoid having to know the type.
One approach that could work as well without need to explicitly checking for data type, but just checking if some construct is valid or not would be using the VBA function IsError as follows:
IIf(IsError(x.Level, <something avoiding the Level function>, <use x.Level>)
Related
I'm trying to use Microsoft.SqlServer.TransactSql.ScriptDom to check that an expression is a scalar constant.
Here is such an expression:
DATEADD(YEAR, -21, CURRENT_TIMESTAMP)
Here is not such an expression:
DATEADD(YEAR, -21, DateOfBirth)
It is not a constant because it references the column DateOfBirth.
How can I determine this?
What I didn't expect -- and why I've run into trouble -- is that Microsoft.SqlServer.TransactSql.ScriptDom thinks that YEAR is a ColumnReferenceExpression.
(too long for comment)
ScriptDom does not compile, just parses and treats all "strange names" as possible column names, e.g. in IF (MAGICNAME = 0) will be detected a "column" named MAGICNAME. If you want more, you have to add more intelligence to this process by yourself.
This can be done by making additional visitor classes to be used as nested parsers. And by storing lists of "known magic words relevant to specific cases". Which in given case may lead to code which:
catches udf
checks if it is a one of well known functions
invokes nested visitor class which understands more about this specific function
In this approach a specific visitor for DATEADD (or all the date handling functions) might have the list of words YEAR, MONTH and so on to change the understanding of first argument from "possible column" to "known static magic word".
Given task can hardly be accomplished in general, for any possible case, however it looks like many cases can be handled correctly. An idea is to implement "duck typing" approach:
detect expressions which can possibly be scalar and "constant" and take a deeper look on them only
in deeper look recursively apply this approach to all expression arguments
if none of them violates your understanding of "scalar constant expression" - then it is one
We have a table with 2 top level columns of type 'struct' - one is a 'before', and an 'after' image. The struct schemas are non trivial - nested, with arrays to a variable depth. The are sent to us from replication, so the schemas are always the same (but the schemas of course can be updated at some point, but always together)
Objective is for the two input structs, to return 2 struct 'diffs' of the before and after with only fields that have changed - essentially the 'delta' diff of the changes produce by the replication source. We know something has changed, but not 'what' since we get the full before and after image. this raw data lands in BQ and is then processed from there but need to determine the more granular change for high order BQ processing.
The table schema is very wide (1000's of leaf fields), and the data populated fairly spare (so alot of nulls will be present on both sides of the snapshot) - so would need to be performant as best as possible when executing over 10s of millions of rows.
All things are nullable for maximum flexibility.
So change could look like:
null -> value
value -> null
valueA -> valueB
Arrays:
recursive use of above for arrays of structs, ordering could be relaxed if that makes it easier?
It might not be possible.
Ive not attempted this yet as it seems really difficult so am looking to the community boffins for some support for this. I feel the arrays could be difficult part. There is probably an easy way perhaps in Python I dont or even doing some JSON conversion and comparison using JOSN tools? It feels like it would be a super cool feature built in to BQ as well, so if can get this to work, will add a feature request for it.
Id like to have a SQL UDF for reuse (we have SQL skills not python, although if easier in python then thats ok), and now with the new feature of persistent SQL UDFs, this seems the right time to ask and test the feature out!
sql
def struct_diff(before Struct, after Struct)
(beforeChange, afterChange) - type of signature but open to suggestions?
It appears to be really difficult to get a piece of reusable code. Since currently there is no support for recursive functions for SQL UDF, you cannot use a recursive approach for the nested structs.
Although, you might be able to get some specific SQL UDF functions depending on your array and structs structures. You can use an approach like this one to compare the structs.
CREATE TEMP FUNCTION final_compare(s1 ANY TYPE, s2 ANY TYPE) AS (
STRUCT(s1 as prev, s2 as cur)
);
CREATE TEMP FUNCTION compare(s1 ANY TYPE, s2 ANY TYPE) AS (
STRUCT(final_compare(s1.structA, s2.structA))
);
You can use UNNEST to work with arrays, and the final SQL UDF would really depend on your data.
As #rtenha suggested, Python could be a lot easier to handle this problem.
Finally, I did some tests using JavaScript UDF, and it was basically the same result, if not worst than SQL UDF.
The console allows a recursive definition of the function, however it will fail during execution. Also, javascript doesn't allow the ANY TYPE data type on the signature, so you would have to define the whole STRUCT definition or use a workaround like applying TO_JSON_STRING to your struct in order to pass it as a string.
I'm trying to run the following MDX query (I'm newbie in the matter):
WITH MEMBER [Measures].[Not Null SIGNEDDATA] AS IIF( IsEmpty( [Measures].[SIGNEDDATA] ), 0, [Measures].[SIGNEDDATA] )
SELECT
{[Measures].[Not Null SIGNEDDATA]} ON COLUMNS,
{[Cuenta].[818000_001],[Cuenta].[818000_G02]} ON ROWS
FROM [Notas_SIC]
WHERE ([Auditoria].[AUD_NA],[Concepto].[CONCEPTO_NA],[Entidad].[CCB],
[Indicador].[INDICADOR_NA],[Interco].[I_NONE],[Moneda].[COP],
[Tiempo].[2010.01],[Version].[VERSION_NA])
Where 818000_001 is a base member of my 'Cuenta' dimension, and 818000_G02 is a node or aggregation. I receive the following message:
"Two sets specified in the function have different dimensionality"
What am I doing wrong? If I put the query with only base members (many) or only differents aggregations, the result is ok as expected.
Thanks in advance!
By using commas in your WHERE clause, you are trying to create a set out of members from different dimensions. You should use asterisks to make a CROSSJOIN instead.
So replace this:
WHERE ([Auditoria].[AUD_NA],[Concepto].[CONCEPTO_NA],[Entidad].[CCB],
[Indicador].[INDICADOR_NA],[Interco].[I_NONE],[Moneda].[COP],
[Tiempo].[2010.01],[Version].[VERSION_NA])
With this:
WHERE ({[Auditoria].[AUD_NA],[Concepto].[CONCEPTO_NA],[Entidad].[CCB]} *
{[Indicador].[INDICADOR_NA],[Interco].[I_NONE],[Moneda].[COP]} *
{[Tiempo].[2010.01],[Version].[VERSION_NA]})
I added curly braces though I'm not sure if they're absolutely required.
This is maybe your problem:
{[Cuenta].[818000_001],[Cuenta].[818000_G02]} ON ROWS
You have put them as a set but you can only make a set out of members with the same dimensionality. From your description it sounds like [Cuenta].[818000_G02] is being classed as a different hierarchy which is unexpected.
Is [Cuenta].[818000_G02] created using the Aggregate function? Can you swap to using the Sum function? Does the script then work?
(not a great answer - just more questions?)
{[Cuenta].[818000_001],[Cuenta].[818000_G02]}
Your problem probably lies in the above statement. As is clear, they could be from different hierarchies(which is not clear from your example and I come to that below).
Try replacing the above with
{[Cuenta].[Hierarchy1].[818000_001]} * {[Cuenta].[Hierarchy2].[818000_G02]}
where Hierarchy1 and Hierarchy2 are the hierarchies to which these members belong. (If you are not sure, just try {[Cuenta].[818000_001]} * {[Cuenta].[818000_G02]}). I am assuming that the second aggregate members may be built on a different hierarchy and thus the engine is throwing an error.
It is generally a good habit to always use the fully qualified member name because then the SSAS engine has to spend less time in searching for the member.
I am writing a CLR userdefinedAggregate function to implement median. While I understand all the other function which I have to implement. I can not understand, what is the use of the merge function.
I am getting a vague idea that if aggregated function is partially evaluated ( i.e. evaluated for some rows with one group and the remaining in other ) then the values needs to be aggregated. If its the case is there a way to test this ?
Please let me know if any of the above is not clear or if you need any further information.
Your vague idea is correct.
From Requirements for CLR User-Defined Aggregates
This method can be used to merge another instance of this aggregate
class with the current instance. The query processor uses this method
to merge multiple partial computations of an aggregation.
The parameter to merge is another instance of your aggregate and you should merge the aggregated data in that instance to your current instance.
You can have a look at the sample string concatenate aggregate. The merge method add the concatenated strings from the parameter to the current instance of the aggregate class.
In a custom ContentProvider I need to filter out some columns specified in the inputs. Given the text-oriented Android interfaces this is giving me a hard time.
For example the input on MyContentProvider.query() would effectively ask something like:
SELECT column_a, column_b FROM my_table WHERE column_a=1 AND column_b=red;
The problem is that at this particular MyContentProvider _column_b_ might not make any sense and would not be present in the table. Filtering the projection so that only relevant columns remain can be easily done since it's a String[]. However, filtering the String "where" (selection) and "selectionArgs" inputs for these columns is not trivial. If done properly it would become:
SELECT column_a FROM my_table WHERE column_a=1;
Otherwise one would get a SQLiteException "no such column".
So, is there any easy way to ignore or filter columns from such an sql statement or do I need to go and write some smart albeit very limited regexp parsing code for the selection part?
The reason I'm not getting the right inputs is because I maintain a custom ContentProvider as an interface to address, but I talk to multiple custom ContentProviders herein (in the background). One way or another, I would need to filter the selection somewhere.
Please note that I am not asking simply how to do a query or use the SELECT ... WHERE statement. However it concerns my implementation of the query() function.
Since you are extending your MyContentProvider with ContentProvider why don't you just overload the query() method?
Look at ContentProvider - Sharing Content using the ContentProvider for someone elses example on how to create a custom ContentProvider. You should have full control over what data you fetch from your SQLiteDatabase.
More importantly, look at the arguments provided to query(), as they contain the information you need to you in a way where you can dynamically build the query from what is passed into the method call.
Depending on if you can find a good query builder, you have an opportunity to build a small but powerful abstraction layer to build your queries, so that you minimize the amount of actual SQL that you write yourself.
Also, always remember to sanitize your inputs!