I am new to Oracle BI. I have been reading documents, tutorials, oracle by example guides and watching some videos.
But I cannot answer some questions which are confusing my mind.
1-) Why we have 3 repository layer? In the tutorials, they are dragging objects from physical layer to business model and mapping layer. In this layer they are renaming table names. Then they are dragging objects from Business Model to Presentation Layer. In this layer they are renaming table names again and they are deleting some columns. I think that we can use just physical layer and presentation layer. In presentation layer we can rename table names and remove some columns.
2-) In physical layer, we are importing objects from data sources. We can import tables, views and foreign keys. What is the best practice for designing business model?
I created a test repository. I imported tables, views and foreign keys from database. But when I try to check consistency ( afte preparing presentation layer), I took error messages about self joins in the physical layer. Can I solve self join problem?
3-) Should I import only tables and views from database in physical layer? I think that if I do not ceate joins manually after import operations, Oracle BI Server may not prepare correct sql statements.
We have a big database (maybe 500 tables), so if I dont import foreign keys, manually creating foreign keys will be a massive manual process. I also do not know which foreign keys are mandatory for a well designed business model?
4-) When database tables changed( for. ex. added new column), are these changes automatically updated to the physical layer?
Thank you..
1-
Physical Layer = Physical Data Model (Database Implementation)
Business Model and Mapping Layer (BMM) = Logical Data Model (Declarative way to say which table is a fact, dimension, ...)
Presentation Layer + What the end-user, client see
The most important to see is that the definition of the BMM is the most important as it permits the generation of the physical query (query against the database).
More check this link:
http://gerardnico.com/wiki/dat/obiee/logical_business_model#centralized_and_consistent_conceptual_data_model
2-
To correct the self-join, you have to create an alias of your table in the physical layer
http://gerardnico.com/wiki/dat/obiee/alias#physical_table_alias
3-
You can import all. also foreign key ;-) the best practice is to import only what you need.
4-
No but you have an utility to do the job. You can find it in the utility box.
And the best OBIEE forum is here: forums.oracle.com/forums/forum.jspa?forumID=378
Success
Nico
Related
I am trying to develop a database for my homework. I designed a Logical Model in SQL data modeler. I tried to convert it to Relational model but Relations were created as Tables. not like relations. I watched some videos on youtube and tried to do same tables. I got same problems again. Where is my mistake and how can I fix it? Thank you so much...
Logical Model
Relational Model
There are different terms used in data modelling, and several types of model provided by different tools and named differently, so exactly what a Relational Model is (I'd never heard of that one) will depend on the tool you are using. From the screenshot, it appears to be a physical model, which means tables, columns, foreign keys, indexes, storage details etc. If you wanted something different then please describe what it is.
Traditionally there is
a logical model, also known as an entity model or entity relationship diagram (ERD), which is database agnostic and merely models business entities, their attributes, unique keys and relationships.
a physical model, defining a schema ready for implementing in a specific vendor's RDBMS. This has tables with columns, primary keys, check constraints, indexes etc.
In my experience, tools often blur the line between entities and tables.
The way I learned it, an entity is a conceptual item of interest to the business, like "Doctor" or "Patient", because the purpose of the model is to show what a Doctor is and what a Patient is and how they are related, and that is why they are named in the singular. A database table, however, (in the physical model) will usually contain rows representing many Doctors and Patients and so the plural is appropriate. Opinions differ, though.
I am creating a dataset In Power BI Desktop using data in a SQL Server database. I have read the sqlbi article that recommends using database views but I would like to know: how do I structure them?
There is already a view of the database that contains most of the information I need. This view consists of 7 or 8 other, more basic views (mainly 2 column tables with keys and values), combined using left joins. Do I import the larger view as a flat table, or each of the smaller views and create a relationships etc, ideally in a star schema, in Power BI?
I guess conceptually I am asking: where does the SQL stop and Power BI start when it comes to creating and importing views?
where does the SQL stop and Power BI start when it comes to creating and importing views?
Great question. No simple answer. Typically modeling in Power BI is quicker and easier than modeling in the database. But modeling in the database enables DirectQuery, and is more useful outside of Power BI.
Sometimes it boils down to who is creating the model. The "data warehouse" team will tend to create models in the database first, either with views or tables. Analysts and end-users tend to create models directly in Power BI.
Sometimes it boils down to whether the model is intended to be used in multiple reports or not.
There is no one-size-fits-all approach here.
If your larger view already has what you need and you need it for just one-off report then you can modify it to add additional fields(data points) considering the trade off for effort needed to create a schema.
The decision weather you should import smaller views and connect them as Star schema ( considering that they have a fact table surrounded by dimension tables) depends on if you are going to use that in lot of other reports where the data is connected i.e. giving you same level of information in every report.
Creating views also depends on lot of other factors, are you querying a reporting snapshot(or read-replicas) of your prod database or you are querying the actual production database. This might restrict you or impact the choice for Views and Materialized Views.
I have a requirement for a graph database that needs to be backed-up and potentially accessed at a lower level of abstraction. It is also must be distributed for the sake of load balancing, (single master replication will do).
I know that it is possible to implement a graph database using a self-referencing key-value store. The Git object database is an example of this pattern. One of the frustrating things I find about most graph databases is that they do not "disclose" their underlying persistence layer in public api.
Do any replicated graph databases exist that allow for an underlying key-value stores to be "plugged-in" or accessed directly?
I addition to Gremlin/Tinkerpop, mentionned by #amirouche above, I'm aware of two solutions:
Redis, completed by its Graph Module, matches your description.
Cayley could also be a solution as it provides graph features over various SQL and NoSQL backends, some of them supporting distributed mode (Postgresql, MySQL, MongoDB, CockroachDB)
I am developing an application that uses a neural network. Currently I am looking at either trying to put it into a relational database based on SQL (probably SQL server) or a graph database.
From a performance viewpoint, the neural net will be very large.
My questions:
Do relational databases suffer a performance hit when dealing with a neural net in comparison to graph databases?
What graph-database technology would be best suited to dealing with a large neural net?
Can a geospatial database such as PostGIS be used to represent a neural net efficiently?
That depends on the intent of progress on the model.
Do you have a fixated idea on an immutable structure of the network? Like a Kohonnen map. Or an off-the-shelf model.
Do you have several relationship structures you need to test out, so that you wish to be able flip a switch to alternate between various structures.
Does your model treat the nodes as fluid automatons, free to seek their own neighbours? Where each automaton develops unique characteristic values of a common set of parameters, and you need to analyse how those values affect their "choice" of neighbours.
Do you have a fixed set of parameters for a fixed number of types/classes of nodes? Or is a node expected to develop a unique range of attributes and relationships?
Do you have frequent need to access each node, especially those embedded deep in the network layers, to analyse and correlate them?
Is your network perceivable as, or quantizable into, set of state-machines?
Disclaimer
First of all, I need to disclaim that I am familiar only with Kohonnen maps. (So, I admit having been derided for Kohonnen as being only entry-level of anything barely neural-network.) The above questions are the consequence of personal mental exploits I've had over the years fantasizing after random and lowly-educated reading of various neural shemes.
Category vs Parameter vs Attribute
Can we class vehicles by the number of wheels or tonnage? Should wheel-quantity or tonnage be attributes, parameters or category-characteristics.
Understanding this debate is a crucial step in structuring your repository. This debate is especially relevant to disease and patient vectors. I have seen patient information relational schemata, designed by medical experts but obviously without much training in information science, that presume a common set of parameters for every patient. With thousands of columns, mostly unused, for each patient record. And when they exceed column limits for a table, they create a new table with yet thousands more of sparsely used columns.
Type 1: All nodes have a common set of parameters and hence a node can be modeled into a table with a known number of columns.
Type 2: There are various classes of nodes. There is a fixed number of classes of nodes. Each class has a fixed set of parameters. Therefore, there is a characteristic table for each class of node.
Type 3: There is no intent to pigeon-hole the nodes. Each node is free to develop and acquire its own unique set of attributes.
Type 4: There are fixed number of classes of nodes. Each node within a class is free to develop and acquire its own unique set of attributes. Each class has a restricted set of attributes a node is allowed to acquire.
Read on EAV model to understand the issue of parameters vs attributes. In an EAV table, a node needs only three characterising columns:
node id
attribute name
attribute value
However, under constraints of technology, an attribute could be number, string, enumerable or category. Therefore, there would be four more attribute tables, one for each value type, plus the node table:
node id
attriute type
attribute name
attribute value
Sequential/linked access versus hashed/direct-address access
Do you have to access individual nodes directly rather than traversing the structural tree to get to a node quickly?
Do you need to find a list of nodes that have acquired a particular trait (set of attributes) regardless of where they sit topologically on the network? Do you need to perform classification (aka principal component analysis) on the nodes of your network?
State-machine
Do you wish to perceive the regions of your network as a collection of state-machines?
State machines are very useful quantization entities. State-machine quatization helps you to form empirical entities over a range of nodes based on neighbourhood similarities and relationships.
Instead of trying to understand and track individual behaviour of millions of nodes, why not clump them into regions of similarity. And track the state-machine flow of those regions.
Conclusion
This is my recommendation. You should start initially using a totally relational database. The reason is that relational database and the associated SQL provides information with a very liberal view of relationship. With SQL on a relational model, you could inquire or correlate relationships that you did not know exist.
As your experiments progress and you might find certain relationship modeling more suitable to a network-graph repository, you should then move those parts of the schema to such suitable repository.
In the final state of affairs. I would maintain a dual mode information repo. You maintain a relational repo to keep track of nodes and their attributes. So you store the dynamically mutating structure in a network-graph repository but each node refers to a node id in a relational database. Where the relational database allows you to query nodes based on attributes and their values. For example,
SELECT id FROM Nodes a, NumericAttributes b
WHERE a.attributeName = $name
AND b.value WItHIN $range
AND a.id = b.id
I am thinking, perhaps, hadoop could be used instead of a traditional network-graph database. But, I don't know how well hadoop adapts to dynamically changing relationships. My understanding is that hadoop is good for write-once read-by-many. However, a dynamic neural network may not perform well in frequent relationship changes. Whereas, a relational table modeling network relationships is not efficient.
Still, I believe I have only exposed questions you need to consider rather than providing you with a definite answer, especially with a rusty knowledge on many concepts.
Trees can be stored in a table by using self-referencing foreign keys. I'm assuming the only two things that need to be stored are topology and the weights; both of these can be stored in a flattened tree structure. Of course, this can require a lot of recursive selects, which depending on your RDBMS may be a pain to implement natively (thus requiring many SQL queries to achieve). I cannot comment on the comparison, but hopefully that helps with the relational point of view :)
I have 5 tables all unnormalised and I need to create an ER model, a logical model, normalise the data and also a bunch of queries.
Where would you begin? Would you normalise the data first? Create the ER model and the relationships?
There are two ways to start data modelling: top-down and bottom-up.
The top-down approach is to ask what things (tangible and intangible) are important to your system. These things become your entities. You then go through the process of figuring out how your entities are related to each other (your relationships) and then flesh out the entities with attributes. The result is your conceptual or logical model. This can be represented in ERD form if you like.
Either along the way or after your entities, relationships and attributes are defined, you go through the normalization process and make other implementation decisions to arrive at your physical model - which can also be represented as an ERD.
The bottom-up approach is to take your existing relations - i.e. whatever screens, reports, datastores, or whatever existing data representations you have and then perform a canonical synthesis to reduce the entire set of data representation into a single, coherent, normalized model. This is done by normalizing each of your views of data and looking for commonalities that let you bring items together into a single model.
Which approach you use depends a little bit on personal choice, and quite a bit on whether you have existing data views to start from.
I think you should first prepare the list of entities and attributes. so that you will be able to get the complete details of the data information.
Once you are clear with the data information. You can start creating the master table and Normalize then.
Then after the complete database is design with normalization, You can create the ER diagram very easily.
I would start by evaluating and then preparing the list of entites and attributes within your data.
I would do it in this order.
Relationships
Create the ER model.
Normalise the data.
I know many others will have a different opinion but this is the way I would go ahead with it :)