We are having a very frustrating problem with SSAS. we are trying to create a fairly simple Tabular model have had no problem with a few dimensions and a single fact table. when we recently added a new dimension table, and a second fact table we can not get it to stop telling us that
The relationship cannot be created because each column contains duplicate values.
we have deleted and reimported these tables many times. at times we can attach the Dim table to an existing fact without problem, which tells me that the key is unique, but then it errors when attaching to a second fact. at other times it just errors right away.
The key and Foreign Key values are Int in the database. and we have examined the data many times, creating stripped down tables with just a few columns, validating that a
Select Distinct ID from Table
type query returns the same number of rows as a wide open one.
At this point it looks like we simply have a bug in SSAS Tab. I cant see a good reason for this to not work.
any ideas would be greatly appreciated
I have had the same problem, the way I solved is to create the relationship in reverse order meaning: from the 1 to many. This will solve the issue, and the relationship's direction will indeed be from the many to the 1
I do not have any explanation for this...
sometimes SSAS cannot detect that your column is unique so you should do that. you can click on your primary key column and get properties and choose true on the unique property and after that create your relationship
Related
I'm working on a data warehouse project using BigQuery. We're loading daily files exported from various mainframe systems. Most tables have unique keys which we can use to create the type 2 history, but some tables, e.g. a ledger/positions table, can have duplicate rows. These files contain the full data extract from the source system every day.
We're currently able to maintain a type 2 history for most tables without knowing the primary keys, as long as all rows in a load are unique, but we have a challenge with tables where this is not the case.
One person on the project has suggested that the way to handle it is to "compare duplicates", meaning that if the DWH table has 5 identical rows and the staging tables has 6 identical rows, then we just insert one more, and if it is the other way around, we just close one of the records in the DWH table (by setting the end date to now). This could be implemented by adding and extra "sub row" key to the dataset like this:
Row_number() over(partition by “all data columns” order by SystemTime) as data_row_nr
I've tried to find out if this is good practice or not, but without any luck. Something about it just seems wrong to me, and I can't see what unforeseen consequences can arise from doing it like this.
Can anybody tell me what the best way to go is when dealing with full loads of ledger data on a daily basis, for which we want to maintain some kind of history in the DWH?
No, I do not think this would be a good idea to introduce an artificial primary key based on all columns plus the index of the duplicated row.
You will solve the technical problem, but I doubt there will be some business value.
First of all you should distinct – the tables you get with primary key are dimensions and you can recognise changes and build history.
But the table without PK are most probably fact tables (i.e. transaction records) that are typically not full loaded but loaded based on some DELTA criterion.
Anyway you will never be able to recognise an update in those records, only possible change is insert (deletes are typically not relevant as data warehouse keeps longer history that the source system).
So my todo list
Check if the dup are intended or illegal
Try to find a delta criterion to load the fact tables
If everything fails, make the primary key of all columns with a single attribute of the number of duplicates and build the history.
I have a product database where I am trying to replicate a particular product's data and relationships to a new product, a clone. I am puzzled however on how to replicate several many to many relationships. For example, consider a product with two parts, and for each part, their are several colors available. I have a Product table, a product Areas table, and a Colors table. The product id is a foreign key in the area table, one to many. The Area table has an area id (pk) along with other descriptive fields, and the Colors have color ids (pk) along with palette information. A fourth table serves as the many to many look up table, it's primary key being the part id and the color id combined. This is a pretty straight forward configuration as far as it goes.
I can't think of a way to clone this structure, however, despite many approaches which would be way too much to elaborate upon here. I can easily enough replicate the left hand, product-area relationship, generating new AreaIDs (A,B,C). But in a next step, I then want to replicate the many-to-many relationship using the new area ids. However, now I don't know which original ID (H,L,W) to associate with which new ID.
For example, does the new id A get mapped to the set of colors from the old ID H, L, or W? I have only id's to work with. I can select both parts and part-color pairs from the source in one select statement, but I can't insert into two tables with one statement.
In other words, how do I replicate many to many relationships if I want to supply a new ID for half of it? Do I have to resort to cursors? I can if I need to, but I'm imagining there an elegant way to accomplish this that I just can't figure out. Maybe using a temp table or some sort of table valued function? I've tried to search for answers, but I all I can find is advice on setting up many-to-many relationships.
Thanks for you experts who have the patience to read through this question.
SymmetricDS replicates tables that have many into many relationships using change data capture. The key is to perform an initial load to get the databases in sync initially so that if a child record is updated the change data capture will also work. In the latest versions of SymmetricDS (3.10 and higher) it will also auto resolve foreign key errors if the databases are not in sync. If a child row is being loaded to a target without the parent it will callback to the source to load the missing parent as well so that you do not need to intervene.
My problem is simple:
I have a column referencing people, and among all those persons one and only one has a specific status. What is the best way to represent this in a PostgreSQL database?
My first idea was to create a column of booleans all equal to false but for the specific person. It means I somehow also need to check that there is only one true in the entire column. However, it does not seem optimal as it means having one more bit per column, and as there might be quite a number of lines, it will waste data.
Second solution is to create a second table to reference the person. However it means creating a table just with one line...
Do you have any other idea of how to solve this problem?
Thank you!
PostgreSQL has an inheritance system that may work well for a problem like this.
CREATE TABLE people (...);
CREATE TABLE special_person ( ) INHERITS (people);
You can select * from people to pull all of the records, while still adding exclusive columns to the special_person.
Creating a table with one entry, with a foreign key to the main table seems reasonable to me. Having to maintain a status when you only care about one value seems wasteful and comparatively difficult to maintain.
I have Table
Field1 PK int not null
Field2 PK int not null like this
when i want to map this table I get this error
Error 3 Error 3034: Problem in mapping fragments starting at lines 2212, 2218:
Two entities with possibly different keys are mapped to the same row.
Ensure these two mapping fragments map both ends of the AssociationSet to the corresponding columns.
And I tried to delete and re-create that table inside model. When i add table it gives another error but at the end i always get this error
How can i handle this problem please help me...
When i investigate the problem, i realised something becuse i read an article before for this data model situation and i understand the why the problem occur. (Problem in mapping fragments in Entity Framework)
Actually problem comes from table mapping because i said that table has many to many relationship so That article says if you put that table in the model design it always gives us this error and finally when we delete that table on design side and add table silently in data model so program can be build. I ‘m telling you this because maybe you can make an idea for this because i learned this and when i add new view in model and vs doesnt give me any error.
This is the old model picture when the delete that table inside red box The program doesn't give any error and continiue to use bottom table
If nothing else works, try deleting and recreating the whole .edmx-file.
I got this problem when I added a couple of many-to-many tables to an existing Entity Framework 6.2 project.
I tried removing and regenerating all tables in the model. Nothing seemed to work. I believe there was some "junk" somewhere in the EF project causing the problem.
When regenerating the edmx, I added all tables in the project, including the many-to-many tables.
solution for error:3034
PK UserId;
PK RoleId
two primary key cannot allow edmx file.so create sno column in that table and make that as primary key . remove the pk of UserId and RoleId.
now in that two column there is no primary key available.
like
Pk sno;
FK UserId;
FK RoleId
In my searching, I have seen this error text been asked in different contexts, but specifically, I want to know if I am missing something here.
In a basic setup, I have 1 measure table and 1 dimension table in my DSV. In my test, the measure table has about half of the sold products, but the dimension table has info on the entire product catalog.
The relationship has the measure table as the foreign key and the dimension as the primary. Case is not an issue as both are integers.
What am I doing wrong? I have tried reversing the relationship and I always process the dimension in full before processing the cube in full whenever I make a change.
To avoid the errors, I can set the ignore key errors thing or ensure that the dimension only contains the products that the measure contains. I can do the latter, but I feel that using an unfiltered reference table is cleaner and more efficient - it's on that note that I say "am I missing something?"
Apologies for wasting everyone's time on this one. I was going to delete the question but thought I would share my self-rectified findings:
In my above example, I failed to mention that the dimension table also had links to other tables. Missing (null) references between those tables caused the dimension member to be missing when processed hence the 'attribute key could not be found' when the cube was processed!