I am new to SSAS and I'm facing a confusing problem.
I have a regular process for updating dimensions (with a ProcessUpdate).
<Process xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ddl2="http://schemas.microsoft.com/analysisservices/2003/engine/2" xmlns:ddl2_2="http://schemas.microsoft.com/analysisservices/2003/engine/2/2" xmlns:ddl100_100="http://schemas.microsoft.com/analysisservices/2008/engine/100/100" xmlns:ddl200="http://schemas.microsoft.com/analysisservices/2010/engine/200" xmlns:ddl200_200="http://schemas.microsoft.com/analysisservices/2010/engine/200/200"> <Object> <DatabaseID>Central</DatabaseID> <DimensionID>Prestatarios</DimensionID> </Object> <Type>ProcessUpdate</Type> <WriteBackTableCreation>UseExisting</WriteBackTableCreation> </Process>
It has been working fine but in the last run I got the following error:
<root xmlns="urn:schemas-microsoft-com:xml-analysis:empty">
<Exception xmlns="urn:schemas-microsoft-com:xml-analysis:exception" />
<Messages xmlns="urn:schemas-microsoft-com:xml-analysis:exception">
<Error ErrorCode="3238002695" Description="Internal error: The operation terminated unsuccessfully." Source="Microsoft Analysis Services" HelpFile="" />
<Error ErrorCode="3240034307" Description="Errors in the OLAP storage engine: Rigid relationships between attributes cannot be changed during incremental processing of a dimension. The error occurred when processing attribute 'Sub Grupo'. Table: 'dbo_Prestatarios', Column: 'SubGrupo', Value: 'A00377'. Source attribute: 'Prestatario'. Key column value(s) of the source attribute: '7384538'." Source="Microsoft Analysis Services" HelpFile="" />
<Error ErrorCode="3240034317" Description="Errors in the OLAP storage engine: An error occurred while the 'Prestatario' attribute of the 'Prestatarios' dimension from the 'Central' database was being processed." Source="Microsoft Analysis Services" HelpFile="" />
<Error ErrorCode="3239837702" Description="Server: The current operation was cancelled because another operation in the transaction failed." Source="Microsoft Analysis Services" HelpFile="" />
</Messages>
I googled this and the cause seems to be that in the source data some of the of the attributes has been changed. However, I reviewed it and the offending record has not been updated at all:
Source data before T-SQL processing:
Key
Grupo
GrupoCubo
Correlativo
IDCubo
7384538
ARIV
A00377
2971
A003772971
Source data after T-SQL processing:
Key
Grupo
GrupoCubo
Correlativo
IDCubo
7384538
ARIV
A00377
2971
A003772971
So I'm not sure why is failling. I restored backups, reprocessed, and get the same results.
I would apprreciate very much any suggestion or advice.
Thanks for reading
If anyone has a similar problem, this is how I solved.
I restored the last known "good" backup for both the transactional and OLAP databases related to the process, and reran all the processes in sequence during all the periods needed -- in this case there were only 2 periods to reprocess.
All the processes reexecuted this way ran smoothly and the error didn't appear again.
I'm assuming that the first time there were an execution error at some point (not sure exactly when) that corrupted the OLAP database and we couldn't fix it. Any attempt we did for that (reprocess dimensions / partitions, etc) only generated more errors.
This incident illustrate the importance of taking backups at key points in time as a part of the processes. Fortunately, we have that policy so we had the proper backups to recover.
Thanks for reading.
Related
I am working on a .Net example where I define my own data type using RTI Connext DDS.
Instead of creating the application from the beginning, I got help from the source code of the hello_world_xml_dynamic example in rti_workspace directory. I have made several changes to the USER_QOS_PROFILES.xml file to create my own data type and changes its name to MY_PROFILES.xml
But when I compile the application and run it from the command line, I get the following error:
DDS_DomainParticipantFactory_create_participant_from_config_w_paramsI:ERROR: Profile library 'MyParticipantLibrary::PublicationParticipant' not found
! Unable to create DDS domain participant
The line of code that catching the error:
if (this.participant == null)
{
this.participant = DDS.DomainParticipantFactory.get_instance().
create_participant_from_config(
"MyParticipantLibrary::PublicationParticipant");
if (this.participant == null)
{
Console.Error.WriteLine("! Unable to create DDS domain participant");
return;
}
}
this is the configuration file MY_PROFILES.xml :
<!--
RTI Data Distribution Service Deployment
-->
<dds xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://community.rti.com/schema/6.0.1/rti_dds_profiles.xsd">
<!-- Qos Library -->
<qos_library name="qosLibrary">
<qos_profile name="DefaultProfile">
</qos_profile>
</qos_library>
<!-- types -->
<types>
<struct name="FlightData">
<member name="Latitude" type="double"/>
<member name="Longitude" type="double"/>
<member name="Altitude" type="double"/>
</struct>
</types>
<!-- Domain Library -->
<domain_library name="MyDomainLibrary" >
<domain name="FlightDataDomain" domain_id="0">
<register_type name="FlightDataType"
type_ref="FlightData" />
<topic name="FlightDataTopic"
register_type_ref="FlightDataType">
<topic_qos name="FlightData_qos"
base_name="qosLibrary::DefaultProfile"/>
</topic>
</domain>
</domain_library>
<!-- Participant library -->
<domain_participant_library name="MyParticipantLibrary">
<domain_participant name="PublicationParticipant"
domain_ref="MyDomainLibrary::FlightDataDomain">
<publisher name="MyPublisher">
<data_writer name="FlightDataWriter"
topic_ref="FlightDataTopic"/>
</publisher>
</domain_participant>
<domain_participant name="SubscriptionParticipant"
domain_ref="MyDomainLibrary::FlightDataDomain">
<subscriber name="MySubscriber">
<data_reader name="FlightDataReader"
topic_ref="FlightDataTopic">
<datareader_qos name="FlightData_reader_qos"
base_name="qosLibrary::DefaultProfile"/>
</data_reader>
</subscriber>
</domain_participant>
</domain_participant_library>
</dds>
where am i making a mistake?
Your XML file looks correct. From the 'not found' error message, it seems that you may not have taken the right steps to instruct your application to load that profiles-file MY_PROFILES.xml to actually learn about your desired Participant. You can easily verify that this is the case by introducing an error in your XML file (for example by incorrectly renaming one tag) and rerun your application. If it does not complain about the syntax or schema of the XML, then your file did not get loaded and this hypothesis is correct.
If that turns out to be your problem indeed, then you have several options to fix that. They are listed in the User's Manual section 18.5 How to Load XML-Specified QoS Settings.
I am looking for a method to check if a cube is accessable i.e. it is processed and not broken.
Example: I got a working cube and i full process a shared dimension so that the cube gets broken.
Is there any mdx or xmla method of finding out what cubes are accessable / processed?
There is an XMLA command DISCOVER_XML_METADATA that can return the state of the database (processes/unprocessed) among other properties. I don't have the best handle on XMLA, so I don't know how to get just the part you need, but this query will return results in the form of XML, and you can parse it from there.
<Discover xmlns="urn:schemas-microsoft-com:xml-analysis">
<RequestType>DISCOVER_XML_METADATA</RequestType>
<Restrictions>
<RestrictionList>
<DatabaseID>AdventureWorks2012MD</DatabaseID>
</RestrictionList>
</Restrictions>
<Properties>
<PropertyList>
</PropertyList>
</Properties>
</Discover>
This requests gets the properties from the objects related to the SSAS database called AdventureWorks2012M. In the results you will see the following:
<Database>
<Name>AdventureWorks2012MD</Name>
<ID>AdventureWorks2012MD</ID>
<CreatedTimestamp>2013-08-01T01:41:10.926667</CreatedTimestamp>
<LastSchemaUpdate>2013-08-01T01:45:05.91</LastSchemaUpdate>
<Description />
<LastProcessed>2013-08-01T01:46:39.713333</LastProcessed>
<State>Processed</State>
<LastUpdate>2014-01-07T19:41:45.146667</LastUpdate>
<AggregationPrefix />
<Language>1033</Language>
<Collation>Latin1_General_CI_AS</Collation>
<Visible>true</Visible>
...
You care about <State>Processed</State>for that database. You can also get the state for each of the dimensions and measure groups as well by adding MeasureGroupID or DimensionID to the restrictions list.
I'm using Apache Camel 2.13.1 to poll a database table which will have upwards of 300k rows in it. I'm looking to use the Idempotent Consumer EIP to filter rows that have already been processed.
I'm wondering though, whether the implementation is really scalable or not. My camel context is:-
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route id="main">
<from
uri="sql:select * from transactions?dataSource=myDataSource&consumer.delay=10000&consumer.useIterator=true" />
<transacted ref="PROPAGATION_REQUIRED" />
<enrich uri="direct:invokeIdempotentTransactions" />
<!-- Any processors here will be executed on all messages -->
</route>
<route id="idempotentTransactions">
<from uri="direct:invokeIdempotentTransactions" />
<idempotentConsumer
messageIdRepositoryRef="jdbcIdempotentRepository">
<ognl>#{request.body.ID}</ognl>
<!-- Anything here will only be executed for non-duplicates -->
<log message="non-duplicate" />
<to uri="stream:out" />
</idempotentConsumer>
</route>
</camelContext>
It would seem that the full 300k rows are going to be processed every 10 seconds (via consumer.delay parameter) which seems very inefficient. I would expect some sort of feedback loop as part of the pattern so that the query that feeds the filter could take advantage of the set of rows already processed.
However, the messageid column in the CAMEL_MESSAGEPROCESSED table has the pattern of
{1908988=null}
where 1908988 is the request.body.ID I've set the EIP to key on so this doesn't make it easy to incorporate into my query.
Is there a better way of using the CAMEL_MESSAGEPROCESSED table as a feedback loop into my select statement so that the SQL server is performing most of the load?
Update:
So, I've since found out that it was my ognl code that was causing the odd message id column value. Changing it to
<el>${in.body.ID}</el>
has fixed it. So, now that I have a usable messageId column, I can now change my 'from' SQL query to
select * from transactions tr where tr.ID IN (select cmp.messageid from CAMEL_MESSAGEPROCESSED cmp where cmp.processor = 'transactionProcessor')
but I still think I'm corrupting the Idempotent Consumer EIP.
Does anyone else do this? Any reason not to?
Yes, it is. But you need to use scalable storage for holding sets of already processed messages. You can use either Hazelcast - http://camel.apache.org/hazelcast-idempotent-repository-tutorial.html or Infinispan - http://java.dzone.com/articles/clustered-idempotent-consumer - depending on which solution is already in your stack. Of course, JDBC repository would work, but only if it meets performance criteria selected.
I'm trying to set up custom aggregated tables in our mondrian data warehouse.
I defined the custom aggregated table like this:
<Table>
<AggName name="..." ...>
<AggFactCount ...></AggFactCount>
<AggMeasure name="[Measures].[Example]" column="..."> </AggMeasureName>
</AggName>
</Table>
As long as [Measures].[Example] points to a <Measure name="Example" column="..." .../> item it works fine.
When it points to a <CalculatedMember name="Example" formula="..." ... /> then I get the following error message:
"Failed to find measure 'Example' for Cube 'ExampleCube'"
As far as I checked it online, this thing should be working for recent versions. We're using mondrian 2.4.2 which supports only Measures in AggregatedTables.
Is there any workaround, sg. like defining a cache-table?
Maybe is it enough to provide a configuration item?
Thanks,
Tamas
I have a dimension Entity with just Key and Value attributes. We need to add a new attribute, IsSpecial with a default member of [False].
When I try to run the xmla, I get the following error:
Errors in the metadata manager. The
cube with the ID of 'X', Name
of 'Y' was invalidated by
operations in the transaction.
Here's a fragment of the XMLA used to alter the dimension
<Dimension>
<ID>Dim Entity</ID>
<Name>Entity</Name>
<UnknownMemberName>Unknown</UnknownMemberName>
<Attributes>
<Attribute>
<ID>Dim Entity</ID>
<Name>Entity</Name>
<Usage>Key</Usage>
<EstimatedCount>119</EstimatedCount>
<KeyColumns>
<KeyColumn>
<DataType>Integer</DataType>
<Source xsi:type="ColumnBinding">
<TableID>shared_DimEntity</TableID>
<ColumnID>EntityKey</ColumnID>
</Source>
</KeyColumn>
</KeyColumns>
<NameColumn>
<DataType>WChar</DataType>
<DataSize>32</DataSize>
<Source xsi:type="ColumnBinding">
<TableID>shared_DimEntity</TableID>
<ColumnID>EntityValue</ColumnID>
</Source>
</NameColumn>
<AttributeRelationships>
<AttributeRelationship>
<AttributeID>IsSpecial</AttributeID>
<Name>IsSpecial</Name>
</AttributeRelationship>
</AttributeRelationships>
</Attribute>
<Attribute>
<ID>IsSpecial</ID>
<Name>IsSpecial</Name>
<KeyColumns>
<KeyColumn>
<DataType>Boolean</DataType>
<Source xsi:type="ColumnBinding">
<TableID>shared_DimEntity</TableID>
<ColumnID>IsShadowTracking</ColumnID>
</Source>
</KeyColumn>
</KeyColumns>
<NameColumn>
<DataType>WChar</DataType>
<Source xsi:type="ColumnBinding">
<TableID>shared_DimEntity</TableID>
<ColumnID>IsShadowTracking</ColumnID>
</Source>
</NameColumn>
<DefaultMember>[Entity].[IsSpecial].[False]</DefaultMember>
</Attribute>
</Attributes>
<Hierarchies>
<Hierarchy>
<ID>Hierarchy</ID>
<Name>Hierarchy</Name>
<Levels>
<Level>
<ID>IsSpecial</ID>
<Name>IsSpecial</Name>
<SourceAttributeID>IsSpecial</SourceAttributeID>
</Level>
</Levels>
</Hierarchy>
</Hierarchies>
</Dimension>
Any suggestions?
I doubt there is a way of doing this.
When changing dimension model you also modify cubes which use that dimension. Without redeploying the whole cube, metadata manager doesn't know about changes in cube.
The same situation occurs when changing dimensions in BIDS on-line - appropriate cubes are affected so they need to be deployed again.
"Recall that ROLAP partition-mode storage means that source data is not
copied to the SSAS destination. Another characteristic of ROLAP partition storage is that
aggregations are written back to relational tables in the source schema."
"To set a dimension as a ROLAP dimension, open the Dimension editor in BIDS, and in the Properties window for that dimension change the StorageMode property from the default MOLAP to ROLAP"
"though this requires Enterprise Ed of SSAS"