Has anybody ever ported a complex ssas multidmensional cube to iccube? and have any advice to offer on lessons learnt/gaps between the two tools etc?
The major one i can see is scope(). What's the equivalent in iccube? nested if/case statements?
I have a list here. Anything else?
function | SSAS | iccube
------------------------|------------------------|------------------------
multi threaded calcs | no | yes
------------------------|------------------------|------------------------
fix/scope block code | SCOPE() | ??
------------------------|------------------------|------------------------
custom functions | clr.net but it's slow | mdx+
------------------------|------------------------|------------------------
generic date utility | third-party code that | ??
dimensions (eg generic | uses scope() eg |
prior period/prior | datetool |
corresponding period) | |
------------------------|------------------------|------------------------
We have a very mdx script calculation heavy cube, and the single threaded nature of the SSAS calculation engine is a real bottleneck. None of the other olap tools we've looked at have been fast enough or had a rich enough language
We use disconnected utility dimensions to drive the functionality, and need to have a date utility dimension (we use a version of this http://sqlmag.com/sql-server-analysis-services/optimizing-time-based-calculations-ssas), use AXIS() extensively, and have recursive sum product across descendents of a hierarchy for a non-additive measure.
Our cube isn't a self service reporting cube. It's a multidimensional calculation engine for our app with a fixed generic schema
Update 1: A simpler example of how we use scope. ic, You mention 'robust' workarounds exist. What would they be for code like this?
// Assumes the date utility dim has been setup as with the priorperiod function as [Dim Date Calculations].[Date Calculations].[Prior Period]
// DimBenchmark is a single attribute disconnected utility dimension. The initial/default value is DimBenchmark.Benchmark.None ie do nothing. The remainder are dynamically set based on code. hardcoded below for simplicity
Scope(DimBenchmark.BenchMark.PriorPeriod);
THIS = [Dim Date Calculations].[Date Calculations].[Prior Period]; // assign the value of some physical and utility dim members to new benchmark attributes. Allows us to only refer to dimbenchmark in subsequent code, irrespective of number of benchmarks or the src dimension.attribute
END SCOPE;
SCOPE(DimBenchmark.BenchMark.Budget);
THIS = DimScenario.Scenario.Budget; //we also have a budget
END SCOPE;
.... // any number of other benchmarks
Create measure currentcube.measures.ComplexCalc as NULL;
SCOPE (measures.ComplexCalc); // this code will only change how complex calc behaves
SCOPE (DimBenchmark.Benchmark.All - DimBenchmark.Benchmark.None); // this will only change the ComplexCalc when the active benchmark selection is not "none"
this= (some measure,Complex subcube etc);
End Scope;
End Scope;
the benefit of this is that complexcalc is null by default. it only gets a value when it meets specific conditions. the main reason to use scope is for speed. Much faster than if/case blocks (and simpler understand)
i dont need to explicitly define which benchmarks are valid, just which benchmark isn't.
and below is how we've implemented the date utility dimension. It allows us to do something like
(measure,[Dim Date Calculations].[Date Calculations].[Prior Period]) and it gives use the prior period of measure for the current member of dim date (month goes back 1 month, quarter goes back 3 months, semester goes back 6 mo, year goes back 12 mo). It's very clean, accurate and pretty fast.
-- Fiscal Month
Scope( [Dim Date].[Month Key].[Month Key].members);
-- Prior Period
Scope([Dim Date Calculations].[Date Calculations].[Prior Period]);
this =
( [Dim Date].[Month Key].CurrentMember.PrevMember
,[Dim Date Calculations].[Date Calculations].[Current]
);
END scope;
End Scope;
-- Fiscal Quarter
Scope( [Dim Date].[Fiscal Quarter].[Fiscal Quarter].members);
-- Prior Period
SCOPE( [Dim Date Calculations].[Date Calculations].[Prior Period]);
THIS = ( [Dim Date].[Fiscal Quarter].CurrentMember.PrevMember
,[Dim Date Calculations].[Date Calculations].[Current]
);
END SCOPE;
END SCOPE;
-- Fiscal Semester
Scope( [Dim Date].[Fiscal Semester].[Fiscal Semester].members);
-- Prior Period
SCOPE( [Dim Date Calculations].[Date Calculations].[Prior Period]);
THIS = ( [Dim Date].[Fiscal Semester].CurrentMember.PrevMember
,[Dim Date Calculations].[Date Calculations].[Current]
);
END SCOPE;
End Scope;
-- Fiscal Year
Scope( [Dim Date].[Fiscal Year].[Fiscal Year].members);
-- Prior Period
SCOPE( [Dim Date Calculations].[Date Calculations].[Prior Period]);
THIS =
( [Dim Date].[Fiscal Year].CurrentMember.PrevMember
,[Dim Date Calculations].[Date Calculations].[Current]
);
END SCOPE;
End Scope;
[disclaimer I'm working for icCube]
Scope is not part of icCube, not yet planned. It's a tricky feature that didn't naturally fit into icCube's architecture (see discussion below, later ... ). The strength of icCube is also the agility of it's R&D team, do not hesitate to contact them directly they will me more than happy to improve and add features.
In icCube there are some functionalities that are different from classical MDX server that might be useful, it's Categories, SubCubes and the eval function.
Categories. Allows for defining a new member that behaves like classical members. This new member can be defined as a set of members or as a subcube. For example here we can define a [Top10] category member as our 10 most important customers :
CATEGORY MEMBER [Top10] as TopCount( [Customers], 10, ([Measures].[Sales],[2015]) )
-> so later on ( [Top10], [Sales] ) will return the sales of this top10 customers
SubCubes, allows defining richer logical relations on members as a set of tuples. icCube implements SubCubeComplement, SubCubeIntersect,SubCubeOthers, SubCubeSymDifference, SubCubeUnion and SubCubeMinus. So calculating all without France (it's trivial here, but think on hierachies with many-to-many relations)
SubCubeMinus([Geography].[Geo].[All], [Geography].[Geo].[France] )
The Eval function, allows for evaluating an expression on a subCube. Here is a trivial example doing the sum using the union :
MEMBER [US+CH] AS Eval( SubCubeUnion( [Switzerland], [United States]) , [Amount])
Last but not least, for the dates function you can define Function in icCube that you can reuse in your MDX, no need to copy&paste everywhere :
CREATE FUNCTION square( Value val ) AS val * val
and combine with CompactSet for a faster evaluation on dates periods (if there are no m2m relations on this dimension) or call some Java functions (you've to activate this module that is off by default).
--------------------- Scope ---------------------------
Warning : Comments might be obsolete as my understanding of scope is the one of a few years ago. Let's go :
Scope is a nice functionality but there are some drawbacks that you can check in Chris Webb's presentation ( link ), check from 47:30 for about 5 minutes.
The issues where/are :
Somehow a scope allows to define a new value for a subcube (remember a subcube might be a single indivisible cell as well as thousands of them)
1) Scopes allows for defining the value of a subcube but what do you do if you want a 'part' of this subcube ?
2) What happens if two scopes collide (intersection is not empty) ?
And all this mixed with security, many-to-many relations, subqueries and set where clauses.
We're not SSAS specialist and it's possible that with a better understanding we try again to implement a clean solution but for know we believe there are other ways solving the problems (for example using calc. members or writebacks).
hope it helps.
Related
A measure 'X' gets its value from different fact tables.Lets consider Time (Fiscal Week, Month)dimension and Channel dimension. For different combination of attributes in these two dimensions X will get its value from different tables as follows:
Week + Channel - gets from table FactTrafficByWeekChannel
Week - gets from table FactTrafficByWeek
Month + Channel - gets from table FactTrafficByMonthChannel
Month - gets from table FactTrafficByMonth
To achieve this I added these fact to cube and created a calculated measure and scope scripts to overwrite the scope. Following is the scope script statement:`
CALCULATE;
CREATE MEMBER CURRENTCUBE.[Measures].[Y]
AS (0),
FORMAT_STRING = "Standard",
VISIBLE = 1;
Scope
([Measures].[Y],[Dim Time].[Fiscal Week].[Fiscal Week].Members
) ;
This = [Measures].[X - Vw Fact Total Weekly Traffic];
End Scope ;
Scope
([Measures].[Y],[Dim Time].[Fiscal Week].[Fiscal Week].Members,
[Dim Campaign].[Channel].[Channel].Members
) ;
This = [Measures].[X - Vw Fact Total Weekly Traffic By Channel];
End Scope ;
Scope
([Measures].[Y],[Dim Time].[Fiscal Month].[Fiscal Month].Members
) ;
This = [Measures].[X - Vw Fact Monthly Traffic];
End Scope ;
Scope
([Measures].[Y],[Dim Time].[Fiscal Month].[Fiscal Month].Members,
[Dim Channel].[Channel].[Channel].Members
) ;
This = [Measures].[X - Vw Fact Monthly Traffic By Channel];
End Scope ;
`
Above code works fine when corresponding dimension attributes are dragged in browsing pane but do not work when added to filter pane.
Fiscal Week dimension dragged to browsing pane this works. But
Fiscal Week dimension dragged to filter pane does not work.
This is because attributes added to filter pane are added as sub-cube statements.
Is there a way to achieve this when attributes are dragged to filter pane as well?
Time dimension Attribute relationship (as asked by Greg)
Any help is highly appreciated. Thanks in advance
One way is using dynamic named set as described in this post in the "Using dynamic sets to detect subselects" section.
Add this to the bottom of your existing MDX script:
Create dynamic named set CurrentCube.[SelectedWeeks] as [Dim Time].[Fiscal Week].[Fiscal Week].Members;
Scope
([Measures].[Y],[Dim Time].[Fiscal Month].[All],
[Dim Time].[Fiscal Week].[All],
[Dim Channel].[Channel].[All]
) ;
This = iif(SelectedWeeks.Count<[Dim Time].[Fiscal Week].[Fiscal Week].Members.Count,[Measures].[X - Vw Fact Total Weekly Traffic],0);
End Scope ;
But the problem with that approach is that it is going to get totally out of control checking all the permutations of single select on channel vs multi select on week, etc. And it will likely be slow.
Instead I would recommend removing the Create Member statement for Y from your MDX script and adding a new measure to your "by channel by week" measure group. Call the new physical measure Y and make it a Sum and connect it to a new SQL column that returns 0 on every row. This is just a placeholder measure you will do scoped assignments against. Why physical? When you make a scoped assignment on a physical measure at the channel level for example that assignment aggregates up. (Assignments to calc measures don't aggregate up.) This helps solve the multiselect issue. You assign at the channel level for example and then the All channel number is correct even under multiselect.
That being said, I am not optimistic you will be able to figure out the correct order of scoped assignments so that all 4 of your measures show up where you want with proper multiselect handling. But feel free to try.
One tip is to use Freeze([Measures].[Y], [Dim Time].[Fiscal Month].[All]) after one scoped assignment to months if you are happy with that portion of the cube but find subsequent scoped assignments are changing parts you were happy with.
If you need further help include a screenshot of the Attribute Relationships tab for Dim Time.
The following script gives exactly the result I want.
It feels like a hack as I've added the custom members VALUE and VALUE_MTD onto the hierarchy [Customer].[Country]. I've chosen this hierarchy arbitrarily - just not used [Measures] or [Date].[Calendar] as they are already in use.
Is there a more standard approach to returning exactly the same set of cells?
WITH
MEMBER [Customer].[Country].[VALUE] AS
Aggregate([Customer].[Country].[(All)].MEMBERS)
MEMBER [Customer].[Country].[VALUE_MTD] AS
Aggregate
(
PeriodsToDate
(
[Date].[Calendar].[Month]
,[Date].[Calendar].CurrentMember
)
,[Customer].[Country].[VALUE]
)
SELECT
{
[Customer].[Country].[VALUE]
,[Customer].[Country].[VALUE_MTD]
} ON 0
,NON EMPTY
{
[Measures].[Internet Sales Amount]
,[Measures].[Internet Order Quantity]
}
*
Descendants
(
{
[Date].[Calendar].[Month].&[2007]&[12]
:
[Date].[Calendar].[Month].&[2008]&[01]
}
,[Date].[Calendar].[Date]
) ON 1
FROM [Adventure Works];
The standard approach is called utility dimension. If you Google this term, you will find several descriptions of this approach. A "utility dimension" is one which does not reference any data, but is just added to the cube for the purpose of being able to cross join them with all other dimensions for calculations. You can have one or more of them.
Thus, in most cases, physically there is nothing in the dimension. It is just used for calculated members. (Depending on the implementation, you may have the attribute members defined physically, if you want to have some properties for them. But then, only the default member is referenced in the star schema from the fact tables. The attribute member values are then overwritten in the calculation script.)
Typical applications for this are time calculations like YTD, MTD, MAT (Moving Annual Total, i. e. a full year of data ending in the selected date), or comparisons like growth vs. a previous period.
I am attempting to return a measure two periods from the current member on the time dimension however I need to only include periods that match a certain criteria (Is_Business_Day = true)
I currently have:
(
[Date].[Calendar].CURRENTMEMBER.NEXTMEMBER.NEXTMEMBER,
[Measures].[SOME MEASURE]
)
Which accurately returns the value of the measure two members in the future, but now I need to additionally apply the filter, but can't quite figure out how to do so.
EDIT:
My thinking is that I would have to do something similar to the following
(
Head(
exists(
[Date].[Calendar].CurrentMember.NextMember:[Date].[Calendar].CurrentMember.Lead(6),
[Date].[Is Business Day].&[True]
),
2
).item(1),
[Measures].[SOME MEASURE]
)
As both hierarchies, [Date].[Calendar] and [Date].[Is Business Day] are in teh same dimension, you can rely on SSAS "Autoexists" which is usually faster than Exists. Hence,
((([Date].[Calendar].[Date].&[20050718].nextmember : null )
*
{ [Date]..Item(0) }
).Item(0)
,[Measures].[SOME MEASURE]
)
The : null construct builds a set to the end of the Date level, i. e. to the last day contained in the attribute.
Cross joining with [Date].[Is Business Day].&[True] automatically restricts the set to those members that co-exist in the dimension (the magic of autoexists).
And .Item(0) extracts the first tuple. In case you need a tuple not of the date, the .Item(0), and the measure member, but just of the date and the measure in your context, apply another Item(0) after the first. This would extract the first member from the tuple.
I have created new measure that counts distinct policies (this measure is called FK Policy Distinct Count).
Then I created new calculated member called CountPolicyEndorsesNull which counts all policies from FK Policy Distinct Count using a filter:
(([Policy].[Endorses].&[0],[FK Policy Distinct Count]).
Than I did new calculated member called CountPolicy:
SUM(EXCEPT([Policy].[Policy Status].[Policy Status],[Policy].[Policy Status].&[Void]), [Measures].[CountPolicyEndorsesNull])
Next, I created a new member CountNewBound
SUM(
{
[Submission].[Tran Type].&[New], [Submission].[Tran Type].&[Developed]
},
[Measures].[CountPolicy]
)
And finally, YTDCountNewBound
SUM(YTD([Invoice Date].[Date Hierarchy].CurrentMember), [Measures].[CountNewBound])
Obviously, SUM function doesn't work in this case. Any idea how to make proper YTD count for calculated member?
Distinct count is a special measure which should be managed with a little more care. The rational behind this is that the when evaluating the measure a set of previous values is kept in memory. In order to improve performance, this structure is not passed and it's quickly converted to a scalar value.
Going back to your problem :
Distinct count can be evaluated over a tuple without problem, but you'll get in problems once you try to evaluate over a set of tuples. A possible, but costly and not always possible, is to create a hierarchy of values so you can convert your set in a member of a dimension.
In your case instead of using YTD([Invoice Date].[Date Hierarchy].CurrentMember) function using another hierarchy -> [Invoice Date].[YTD Date Hierarchy].
All this depends on the specific OLAP implementation you're using, but I guess holds true for mainly all OLAP vendors.
Following on from this question regarding calculating a member I have 2 calculated members defined as:
MEMBER [Asset].[Class].[Fixed Income Derivatives]
AS
AGGREGATE(
{
[Asset].[Class].&[Fixed Income],
[Asset].[Sub Class].&[Derivatives]
},
[Measures].CurrentMember
)
MEMBER [Asset].[Class].[Fixed Income Non-derivatives]
AS
AGGREGATE(
{
[Asset].[Class].&[Fixed Income],
EXCEPT([Asset].[Sub Class].[Sub Class],[Asset].[Sub Class].&[Derivatives])
},
[Measures].CurrentMember
)
I can use these in an MDX query as follows:
SELECT
{
[Measures].[Market Value]
} ON 0,
NON EMPTY
{
[Asset].[Class].[Fixed Income Derivatives],
[Asset].[Class].[Fixed Income Non-derivatives]
[Asset].[Class].[Class]
} ON 1
FROM [Asset]
And this gives me output as follows:
Class-----------------------|-MarketValue
============================|=============
Fixed Income Derivatives | 12345
Fixed Income Non-derivatives| 54321
Fixed Income | 66666
Property | 123
Equity | 987
Note that the first 2 rows are actually constituant parts of Row 3. Now, I can do some magic with the client code which reads this data to turn that table into a hierarchy, but - and here's the question - can I do it using MDX? Or am I just complicating things? Im not adverse to making changes in the cube if necessary, or if I could define this hierarchy there.
AFAIK, you can't use MDX to get a result that is supposed as hierarchy at the client side without some magic with the client code or changing the cube structure. Even though you can find a trick to fake the client the result as an hierarchy i don't suggest you to go that way.
Here is my suggestion.
If i understand correctly, you want to get a result that is kind of an unbalanced hierarchy tree that looks like below.
[Asset].[Class]
[Fixed Income]
[Asset].[Class].[Fixed Income Derivatives]
[Asset].[Class].[Fixed Income Non-Derivatives]
Property
Equity
This kind of unbalanced trees can only be implemented by parent-child hierarchies.
You can either try to balance the hierarchy by adding an intermediate level (or leaf levels) that includes Property and Equity members or create a parent-child hierarchy to achieve even deeper level unbalanced tree.
Edit
Here is an article about parent-child dimensions.
I ended up just returning both Class and Sub Class, and building the hierarchy in client code.
Perhaps adding the calculated member this way:
WITH MEMBER [Asset].[Class].[Fixed Income].[Derivatives] AS ...