My database connection is MySQL through JDBC, standard innodb.
From this, I created a very simple data source with two dimensions and one measure.
The two dimensions are:
location (string)
ddate (date/time, at least in Mysql)
The measure is the same "location" element, set to aggregate on count
I am trying to get a fairly simple count of items by location, by day of the week (either mon-sun or 1-7 is fine)
I have tried dozens of variations on the following, including "with member...as..." for extracting the day of week, all met with failure.
SELECT
NON EMPTY Hierarchize({ datepart("d",cdate([diDate.hDate].[mDate]) }) ON COLUMNS,
NON EMPTY {Hierarchize({[diLocation.hLocation].[mLocation].Members})} ON ROWS
FROM [k_olap]
Any help would be greatly appreciated. I've been banging my head against this for hours & hours, and it seems like it should be simple & straightforward for just a single portion of the date without needing to build a full time dimension in the schema, which Pentaho BA doesn't facilitate in the web server.
Here's the XML schema generated by Pentaho BA from the MySQL data source:
<Schema name="k_olap">
<Dimension name="diDate">
<Hierarchy name="hDate" hasAll="true" primaryKey="ID">
<Table name="post" schema="kn"/>
<Level name="mDate" uniqueMembers="false" column="ddate">
</Level>
</Hierarchy>
</Dimension>
<Dimension name="diLocation">
<Hierarchy name="hLocation" hasAll="true" primaryKey="ID">
<Table name="post" schema="kn"/>
<Level name="mLocation" uniqueMembers="false" column="location" type="String">
</Level>
</Hierarchy>
</Dimension>
<Cube name="k_olap">
<Table name="post" schema="kn"/>
<DimensionUsage name="diDate" source="diDate" foreignKey="ID"/>
<DimensionUsage name="diLocation" source="diLocation" foreignKey="ID"/>
<Measure name="mesLocation" column="location" aggregator="count" formatString="Standard"/>
</Cube>
</Schema>
Create a proper date dimension and hierarchy which has the day of week as a column in the table, then use that.
See here:
http://type-exit.org/adventures-with-open-source-bi/2010/07/a-simple-date-dimension-for-mondrian-cubes/
Related
I have a table dimDate built correctly. I am using a surrogate key DC_tk('20211202') in my fact table to map to the primary key in dimDate CCYYMMDD('20211202') which correlates to date_tk in dimDate. The issue is that when the previous week is selected in the analyzer filter the month that is showing up in the Pentaho Analyzer is November and not December, in other words only the beginning days of the week. I suspect this is because week 49 overlaps Nov and Dec. If the Select from list filter value is set to week 49 then two lines of 49 are available to filter on.
The code is as follows:
<Dimension type="TimeDimension" visible="true" highCardinality="false" name="EXCREATEDDATE">
<Hierarchy name="Created Date" visible="true" hasAll="true" primaryKey="date_tk">
<Table name="DimDate" schema="dbo">
</Table>
<Level name="Created Year" visible="true" column="YEAR" type="String" uniqueMembers="true" levelType="TimeYears" hideMemberIf="Never">
<Annotations>
<Annotation name="AnalyzerDateFormat">
<![CDATA[[yyyy]]]>
</Annotation>
</Annotations>
</Level>
<Level name="Created Month" visible="true" column="MONTH" type="String" uniqueMembers="false" levelType="TimeMonths" hideMemberIf="Never" captionColumn="month_short_desc">
<Annotations>
<Annotation name="AnalyzerDateFormat">
<![CDATA[[yyyy].[M]]]>
</Annotation>
</Annotations>
</Level>
<Level name="Created Week" visible="true" column="week_of_year" type="String" uniqueMembers="false" levelType="TimeWeeks" hideMemberIf="Never">
<Annotations>
<Annotation name="AnalyzerDateFormat">
<![CDATA[[yyyy].[M].[w]]]>
</Annotation>
</Annotations>
</Level>
<Level name="Created Day" visible="true" column="day_of_month" type="String" uniqueMembers="false" levelType="TimeDays" hideMemberIf="Never">
<Annotations>
<Annotation name="AnalyzerDateFormat">
<![CDATA[[yyyy].[M].[w].[d]]]>
</Annotation>
</Annotations>
</Level>
<Level name="Created Date" visible="true" column="DateCCYYMMDD" type="String" uniqueMembers="false" levelType="TimeDays" hideMemberIf="Never">
<Annotations>
<Annotation name="AnalyzerDateFormat">
<![CDATA[[yyyy].[M].[w].[d].[yyyyMMdd]]]>
</Annotation>
</Annotations>
</Level>
</Hierarchy>
<DimensionUsage source="EXCREATEDDATE" name="EXCREATEDDATE" visible="true" foreignKey="DC_tk" highCardinality="false"></DimensionUsage>
How can I prevent this from happening while using Pentaho's built in relative date filtering?
Any help on this issue is appreciated.
I realized that since there are two values for month and this would skew the hierarchy.
The recognized hierarchy is:
[yyyy] (Year)
[yyyy].[q] (Quarter)
[yyyy].[q].[M] (Month)
[yyyy].[q].[M].[w] (Week)
[yyyy].[q].[M].[w].[yyyy-MM-dd] (Day)
But this fails if a week is extended over two months.
Modifying the hierarchy by switching the order of Month and Week to read as below will work:
[yyyy] (Year)
[yyyy].[q] (Quarter)
[yyyy].[q].[w] (Week)
[yyyy].[q].[w].[M] (Month)
[yyyy].[q].[w].[M].[yyyy-MM-dd] (Day)
By design, if Previous Week is Week 49 and it spans a month (or year), when filtering on Previous Week, the MDX that Analyzer generates should be able to figure out there are two members and include both Week 49 members (i.e. the one from November and the one from December).
If you are able to upgrade to 9.x, then there are a bunch of analyzer.properties that you can configure so that Analyzer can generate the correct week number and week year. As long as Analyzer can figure out the correct current week member based on the current date, then it can look back to the prior member to see if it is also the same week number but possibly different month or year.
These new properties are as follows:
filter.relative.dates.week.firstDayOfWeek
filter.relative.dates.week.minimalDaysInFirstWeek
filter.relative.dates.week.firstWeekOnJanuary1st
filter.relative.dates.week.firstWeekOnJanuary1st.fromOne
See https://help.hitachivantara.com/Documentation/Pentaho/9.0/Products/Adapt_Mondrian_schemas_to_work_with_Analyzer#Relative_date_filters_for_weeks
BTW, there's a link in Admin->MDX->Check AnalyzerDateFormat that will print out the current through previous 100 time period members for each date level so that you can verify that the AnalyzerDateFormat annotation and the above properties are setup correctly for relative date filtering on your date dimension.
Hard to believe, but I can't seem to find a straight answer for this: How can I get the SQL statement including the parameter values when the statement generates an exception and only when it generates an exception. I know how to log the statement+parameters for every SQL generated, but that's way too much. When there's a System.Data.SqlClient.SqlException, though, it only provides the SQL, not the parameter values. How can I catch that at a point where I have access to the that data so that I can log it?
Based on a number of responses to various questions (not just mine), I've cobbled something together that does the trick. I think it could be useful to others as well, so I'm including a good deal of it here:
The basic idea is to
Have NH log all queries, pretty-printed and with the parameter values in situ
Throw all those logs out except the one just prior to the exception.
I use Log4Net, and the setup is like this:
<?xml version="1.0"?>
<log4net>
<appender name="RockAndRoll" type="Util.PrettySqlRollingFileAppender, Util">
<file type="log4net.Util.PatternString" >
<conversionPattern value="%env{Temp}\\%property{LogDir}\\MyApp.log" />
</file>
<DatePattern value="MM-dd-yyyy" />
<appendToFile value="true" />
<immediateFlush value="true" />
<rollingStyle value="Composite" />
<maxSizeRollBackups value="10" />
<maximumFileSize value="100MB" />
<staticLogFileName value="true" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date %-5level %logger - %message%newline" />
</layout>
</appender>
<appender name="ErrorBufferingAppender" type="log4net.Appender.BufferingForwardingAppender">
<bufferSize value="2" />
<lossy value="true" />
<evaluator type="log4net.Core.LevelEvaluator">
<threshold value="ERROR" />
</evaluator>
<appender-ref ref="RockAndRoll" />
<Fix value="0" />
</appender>
<logger name="NHibernate.SQL">
<additivity>false</additivity>
<appender-ref ref="ErrorBufferingAppender" />
<level value="debug" />
</logger>
<logger name="error-buffer">
<additivity>false</additivity>
<appender-ref ref="ErrorBufferingAppender" />
<level value="debug" />
</logger>
<root>
<level value="info" />
<appender-ref ref="RockAndRoll" />
</root>
</log4net>
The NHibernate.SQL logger logs all queries to the ErrorBufferingAppender, which keeps throwing them out and saves only the last one in its buffer. When I catch an exception I log one line at ERROR level to logger error-buffer, which passes it to ErrorBufferingAppender which -- because it's at ERROR level -- pushes it, along with the last query, out to RockAndRoll, my RollingFileAppender.
I implemented a subclass of RollingFileAppender called PrettySqlRollingFileAppender (which I'm happy to provide if anyone's interested) that takes the parameters from the end of the query and substitutes them inside the query itself, making it much more readable.
If you are using nhibernate for querying the db (as the tag presence on your question suggests), and your SQL dialect/driver relies on ADO, you should get a GenericADOException from the failing query.
Its Message property normally already include parameters values.
For example, executing the following failing query (provided you have at least one row in DB):
var result = session.Query<Entity>()
.Where(e => e.Name.Length / 0 == 1);
Yields a GenericADOException with message:
could not execute query
[ select entity0_.Id as Id1_0_, entity0_.Name as Name2_0_ from Entity entity0_ where len(entity0_.Name)/#p0=#p1 ]
Name:p1 - Value:0 Name:p2 - Value:1
The two literals, 0 and 1, of the query have been parameterized and their values are included in the message (with an index base mismatch: on hibernate queries, they are 1 based, while on the SQL query with my setup, they end up 0 based).
So there is nothing special to do to have them. Just log the exception message.
Have you just missed it, or were you asking something else indeed?
Your question was not explicit enough in my opinion. You should include a MVCE. It would have show me more precisely in which case you were not able of getting those parameters values.
I am currently doing a PoC and facing a problem with closure table. I am using Saiku CE and database is postgres. Everything works until I add a closure table.
If I remove closure table hierarchy, I don't get any error. If keep it I get the error. I have created my demo schema using Foodmart.xml which I downloaded from Saiku itself.
Some forums suggested that it's an open bug with Mondrian but if it is then why same syntax works with foodmart? Is it a problem with Saiku CE? If I use Saiku EE (Trial version) for my PoC then will it work?
11:54:17,900 WARN [RolapUtil] Mondrian: Warning: JDBC driver sun.jdbc.odbc.JdbcOdbcDriver not found
11:54:17,902 WARN [RolapUtil] Mondrian: Warning: JDBC driver oracle.jdbc.OracleDriver not found
11:54:18,728 ERROR [SecurityAwareConnectionManager] Error connecting: ersdemods
java.lang.NullPointerException
<Dimension name="Organisation" key="Org Id">
<Attributes>
<Attribute name="Par Org" table="org_organisation" keyColumn="parent_id" />
<Attribute name="Org Id" table="org_organisation" keyColumn="id" nameColumn="name" />
<Attribute name='Country Name' table='org_organisation' keyColumn='country' hasHierarchy='false' />
<Attribute name='County Name' table='org_organisation' hasHierarchy='false'>
<Key>
<Column name='country' />
<Column name='county' />
</Key>
<Name>
<Column name='county' />
</Name>
</Attribute>
<Attribute name='City Name' table='org_organisation' keyColumn='city' hasHierarchy='false' />
</Attributes>
<Hierarchies>
<Hierarchy name="Organisations" allMemberName="All Organisations">
<Level attribute="Org Id" parentAttribute="Par Org" nullParentValue="NULL">
<Closure table='organisation_closure' parentColumn="closure_parent_org_id" childColumn="org_id" />
</Level>
</Hierarchy>
<Hierarchy name='Oragnisation Location' allMemberName='All Org Location'>
<Level attribute='Country Name' />
<Level attribute='County Name' />
<Level attribute='City Name' />
</Hierarchy>
</Hierarchies>
</Dimension>
Regards,
Puneet Tayal
Managed to fix this issue. Dimension definition was correct however dimension with closure table should be declared within the cube.
If you declare them outside of the cube you would get this idiotic error.
Looks like a bug with Mondrian 4.
Regrads,
Puneet Tayal
I have a problem with filters in saiku 3.7 in Pentaho biserver CE,
I want filters to be joined, what i mean is for example I have a sales table with data from Jan 1 2014 until Nov 25 2015... When user filter by year only 2015, I need in months only to show current data in this case it should show months only until November.
Is this possible?
THis is part of my schema xml
<Level name="Anio" visible="true" column="anio" type="Numeric" uniqueMembers="false" levelType="TimeYears" hideMemberIf="Never" description="Año Factura">
</Level>
</Hierarchy>
<Hierarchy name="Trimestre" visible="true" hasAll="true" primaryKey="id">
<Level name="Trimestre" visible="true" column="trimestre" type="String" uniqueMembers="false" levelType="TimeQuarters" hideMemberIf="Never">
</Level>
</Hierarchy>
<Hierarchy name="Mes" visible="true" hasAll="true" primaryKey="id">
<Level name="Mes" visible="true" column="mesno" type="Numeric" uniqueMembers="false" levelType="TimeMonths" hideMemberIf="Never" captionColumn="mes">
</Level>
</Hierarchy>
</Dimension>
<Dimension type="StandardDimension" visible="true" highCardinality="false" name="Bodega">
<Hierarchy name="Bodega" visible="true" hasAll="true" allMemberName="Bodegas" primaryKey="id">
<Level name="Bodega" visible="true" column="Bodega" type="String" uniqueMembers="false" levelType="Regular" hideMemberIf="Never">
</Level>
</Hierarchy>
</Dimension>
Thanks for your help
When you create a date dimension if your hierarchy is designed properly you will only get data returned for the rows that have data as Mondrian will do an inner join on your date dimension to your fact table.
So if you have a date dimension that is something like:
<Dimension name="my_date">
<Hierarchy name="my_date_hierarchy">
<Level name="years">
<Level name="months">
<Level name="days">
</Hierarchy>
</Dimension>
Then join that to your fact table. If you have data for 2014 and 2015 and drop years onto an axis, you'll get data for 2014 and 2015 even if there are date dimension records for future dates. If you then drop months onto the same axis, you'll then see all the monthly data for 2014, 2015. If you then just wanted to see 2015 you could click on the years level on the rows axis, and select 2015 from the selections
I have this MDX query:
WITH
MEMBER [CLIENT].[WITHOUT CLIENT X] AS
'[CLIENT].[All CLIENTs] - [CLIENT].[CLIENT X]'
SELECT
[Measures].[Sales] ON COLUMNS
,{[STORE].[All STOREs].Children} ON ROWS
FROM [Sales]
WHERE
CrossJoin
(
{[YEAR].[2015]}
,{[CLIENT].[WITHOUT CLIENT X]}
);
This query return corect result (All stores sales without [CLIENT].[CLIENT X]).
Now, I want to use this calculated member ([CLIENT].[WITHOUT CLIENT X]) in Saiku, OpenI, BTable or Pivot4J as a normal dimension or something like this.
For this i added in schema.xml:
<CalculatedMember name="WITHOUT CLIENT X"
formula="([CLIENT].[All CLIENTs] - [CLIENT].[CLIENT X])" hierarchy="CLIENT" parent="CLIENT" visible="true"></CalculatedMember>
If i run:
Select [Measures].[Sales] on COLUMNS,
{[STORE].[All STOREs].Children} ON ROWS
from [Sales]
WHERE CrossJoin({[YEAR].[2015]}, {[CLIENT].[WITHOUT CLIENT X]})
the result returned is corect, but i can't see this dimension [CLIENT].[WITHOUT CLIENT X].
In Saiku, OpenI or Pivot4J i can't find this dimension.
BTable get this dimension as MEASURES because of CDA (/api/olap/getCubeStructure) and result is this:
{
"type": "measure",
"name": "WITHOUT CLIENT X",
"caption": "WITHOUT CLIENT X]",
"qualifiedName": "[CLIENT].[CLIENT X]",
"memberType": "FORMULA"
}
but is useless because I can't use in filter.
Do you have same answers?
Posible questions:
Why u use member and not named set?
Because dimension CLIENT it's huge (more than 50000) query with member run in 1 sec and with set in more than 5 minutes
why don't use EXCEPT?
Same as 1.
why not use just mdx query?
Because final report is for non-tehnic and they want to change this filter.
UPDATE INFO
I renamed [CLIENT].[WITHOUT_this_CLIENT] with [CLIENT].[WITHOUT CLIENT X] and [CLIENT].[My Special Client] with *[CLIENT].[CLIENT X]*to avoid some confusion.
I want to filter a level in this way: add all members (CLIENT) except one (CLIENT X).
The result returned is corect when i use mdx query, because GUI OLAP clients cant read my xml schema.
I don't know where is my error.
Thank you,
Geo
SOLVED
For what I want (filtering one element from a big list) the answer is this:
I created another dimension, which is used just for filter.
<Dimension type="StandardDimension" visible="true" highCardinality="false" name="CLIENT X">
<Hierarchy visible="true" hasAll="true">
<Table name="SALES" schema="SALES" />
<Level name="CLIENT X" visible="true" column="CLIENT" type="String" uniqueMembers="false" levelType="Regular" hideMemberIf="Never">
<KeyExpression>
<SQL dialect="oracle">
<![CDATA[CASE WHEN CLIENT = 'CLIENT X' THEN 'CLIENT X' ELSE 'WITHOUT CLIENT X' END]]>
</SQL>
</KeyExpression>
</Level>
</Hierarchy>
</Dimension>
You're not defining your member correctly. ie. The parent member must be the fully qualified name of the parent of the newly created member, not the level's name. Read this carefully and you'll figure it out.