Consider the client application that should store its data on remote server. We do not want it to access this data "on-fly", but rather want it to have a copy of this data in local database. So we do not need connection with remote server to use application. Eventually we want to sync local database with remote server. The good example of what I am talking about is Evernote service. This type of applications is very common, for instance, in mobile development, where user is not guaranteed to have permanent Internet connection, bandwidth is limited and traffic can be expensive.
ORM (object relational mapping) solutions generally allows developer to define some intermediate "model" for his business logic data. And then work with it as some object hierarchy in his programming language, having the ability to store it relational database.
Why not to add to an ORM system a feature that will allow automatic synchronization of two databases (client and server), that share the same data model? This would simplify developing the applications of the type I described above. Is there any systems for any platform or language that have this or similar feature implemented?
this links may provide some useful information
http://gwtsandbox.com/?q=node/34
http://www.urielkatz.com/archive/detail/google-gears-orm-v01/
http://www.urielkatz.com/archive/detail/google-gears-orm/
AFAIK, there are no such ORM tools.
It was one of original goals of our team (I'm one of DataObjects.Net developers), but this feature is still not implemented. Likely, we'll start working on database sync this spring, but since almost nothing is done yet, there is no exact deadline for this.
There is at least one Open Source ORM matching your needs, but it is a Delphi ORM.
It is called mORMot, and use JSON in a stateless/RESTless architecture to communicate over GDI messages, named pipes or HTTP/1.1. It is able to connect to any database engine, and embed an optimized SQlite3 engine.
It is a true Client-Server ORM. That is, ORM is not used only for data persistence of objects (like in other implementations), but as part of a global n-Tier, Service Oriented Architecture. This really makes the difference.
Related
Say I have a python/Django website fully built. I now want to re-create that website with Ruby on Rails or some other language. Is this possible to keep the same database? Or would I have to transfer data between the two databases?
Yes, it is possible to keep the same database in the new application that you used from the old. You can even have multiple applications use the same database as the same time.
However, you should not just port everything over query for query. There are likely subtleties in the old application that will be easy to miss... places where logic one would normally expect to live in the database instead lives in the application. There are also likely decisions regarding database structure that were made to accommodate quirks or abilities of the old environment that no longer make sense for the new.
The result is this a common point where you might also create a service layer. With a service layer, neither application talks to the database directly. Instead, they both talk to a service application that mediates access to the DB.
This new service layer helps make sure business logic is consistent across applications, without drifting between the two platforms. It helps avoid duplicating work. It helps with performance by creating an obvious place for things like a heroku caching layer, and by making it easier to scale data access across multiple servers.
Hi everyone I’m new to server-side technologies so maybe this is a bit of a dumb question but after reading dozens of articles and viewing dozens of videos I’m still very confused. This has to do with arquitecture principles of modern apps.
Relational model:
I know that a few years ago the model was to have a database (mostly relational) and a DBMS that enabled the connection between an app and the database.
Question 1: Since we are talking about a relational model some examples of DBMS’s are MySQL or PostgreSQL?
Question 2: What is the process of information exchange? The client-side uses a language like PHP to make a request to the server and then the DBMS transforms the request into SQL and accesses the database? Is the conversion of the PHP into SQL part of DBMS function or another server-side software is needed?
(If someone could provide me summary detailed explanation I would be very thankful)
Non-Relational Models:
Question 2: Nowadays with the rise of NoSQL models does the same concept of DBMS apply? Since these systems allow other querying language other than SQL there should be some piece of software that has this function?
Service Oriented Arquitecture:
Almost every app uses this type of arquitecture. I understand the concept of avoid the creation of too tight software relation between client and server side allowing for multiple use across several platforms. What I don’t understand is what parts constitute a system that is build this way.
Question 3: Does the DBMS provides the API’s that constitute the web services made available?
Web Frameworks:
Last but not least, where do frameworks like Django or Ruby on Rails land on?
Question 4: These are supposed to provide tools to develop everything between the front-end and the database of a SOA system right?
Question 5: I’ve seen a lot of buzz about REST arquitecture. Can you explain me of the querying process happens and what are the software entities involved.
Thank you in advance for any explanation that helps me understating these questions. Please provide some links or any diagrams that you find useful.
EDIT:
I'll tackle your questions individually:
Question 1: Since we are talking about a relational model some examples of DBMS’s are MySQL or PostgreSQL?
Correct. The Database Management System is the suite of software that lets you interact with a particular database technology. The examples you give are correct.
Question 2: What is the process of information exchange? The client-side uses a language like PHP to make a request to the server and then the DBMS transforms the request into SQL and accesses the database? Is the conversion of the PHP into SQL part of DBMS function or another server-side software is needed?
There are many different avenues for this. Typically the API for accessing a database is done via ODBC (Open Database Connectivity). ODBC drivers are available for most (if not all) Relational DB vendors and are all very similar.
A language like PHP could connect to the database via an ODBC connection library (eg http://php.net/manual/en/intro.uodbc.php) which would allow you to send CRUD operations to the DBMS to be executed on the database.
Since most DBMS's use a subset or superset of a SQL standard for querying the database you can either pass this code directly via ODBC or you may use another level of abstraction. A common method is called an ORM (Object Relational Mapper). ORMs (eg SQLAlcmehy for Python: http://www.sqlalchemy.org/) provides an abstraction layer so you're not reliant on writing SQL but instead write queries and database commands in a format more common of your language of choice.
Question 2: Nowadays with the rise of NoSQL models does the same concept of DBMS apply? Since these systems allow other querying language other than SQL there should be some piece of software that has this function?
Same general concept (in that there is a DB Driver that exposes an API that languages can hook into) but there are generally more and differing ways of interacting with DB's now since they have many differing structures. Most NoSQL DBs still have ODBC connectors (eg MongoDB and Hadoop) so the general programming practices still apply whilst connecting with them, but the things you expect the database to do (and their natural query languages) will differ.
This continues to be an evolving space as these technologies evolve.
Question 3: Does the DBMS provides the API’s that constitute the web services made available?
Not sure I'm understanding this question. ODBC and webservices are different. Often webservices sit on top of ODBC if you want to query a database via a web API, but it's one layer of abstraction more than connecting to the DB directly via ODBC.
Last but not least, where do frameworks like Django or Ruby on Rails land on?
Web Frameworks are a way of quickening the development of web applications by trying to stop some of the "reinvent the wheel" things that you commonly do with every web application. They give you the basics, and have a lot of extensions that allow you to implement other common elements of web apps (like a subscription/login system, session management, admin system etc).
These are supposed to provide tools to develop everything between the front-end and the database of a SOA system right?
Both Django and RoR aim to be end-to-end frameworks. They include all the common elements you'll need including an Object Relational Mapper. They don't prescribe which DBMS you have to use, their ORMs can interface with many so that choice is still up to you.
Yes, they are aimed to cover everything from the front end to the DB, including the interaction with and initialization of the structure of the database.
Question 5: I’ve seen a lot of buzz about REST arquitecture. Can you explain me of the querying process happens and what are the software entities involved.
REST stands for Representational State Transfer (quick Wikipedia article: https://en.wikipedia.org/wiki/Representational_state_transfer). In a nutshell the creation of a "RESTful" (REST compliant) web API would mean that you have GET, PUT, POST and DELETE methods to accomplish all your services. REST aligns closely with the HTTP protocol which is why it's been very appropriate for the ever growing concept of Web Apps, it's helped transform the thinking from a Web Page (or set of web pages) into Web Apps.
It's hard to sum up better than the Wikipedia article does, I'd suggest you dive into it.
Hope that's cleared up a few of the questions!
The OP in this question asks about using an WCF/OData as an internal data access layer.
Arguments of using WCF/OData as access layer instead of EF/L2S/nHibernate directly
The resounding reply seems to be don't do it. I'm in similar position to the OP, but have a concern not raised in the original question. I'm trying to develop (natively) for a lot of different platforms but want to keep as much of the data and business logic server side as possible. So I'll have iOS/Android/Web (MVC)/Desktop applications. Currently, I have a single WinForms applications with an ORM data access layer (LLBLGen Pro).
I'm envisioning moving most of my business / data access logic (possibly still with LLBLGen or other ORM) behind a WCF / OData interface. Then making all my different clients on the different platforms very thin (basically UI and WCF calls).
Is this also overengineered? Am I missing a simpler solution?
I cannot see any problem in your architecture or consider it overengeenered as a OData is a standard protocol and your concept conforms the DRY principle as well.
I change the question: Why would you implement the same business logic in each client to introduce more possible bugs and loose the possibility to fix the errors at one single and centralized place. Your idea makes you able to implement the security layer only once.
OData is a cross-platform standard and you can find a OData libraries for each development platform (MSDN, OData.org, JayData for JavaScript). Furthermore, you can use OData FunctionImports/Service methods and entity-level methods, which will simplify your queries.
If you are running multiplatform development, then you may find more practical to choose platform-agnostic communication protocol, such as HTTP, rather than bringing multiple drivers and ORMs to access your data Sources directly. In addition since OData is a REST protocol, you don't need much on the Client side: anything that can format OData HTTP requests and parse HTTP responses. There are however a few aspects to be aware of:
OData server is not a replacement for an SQL database. It supports batches but they are not the same as DB transactions (although in many cases can be used to model transactional operations). It supports parent-child relations but it does not support JOINs in classic SQL sense. So you have to plan what you expose as OData entity. It's too easy to build an OData server using WCF Data Services wrapping EF model. Too easy because People often expose low Level database content instead of building high level domain types.
As for today an OData multiplatorm clients are still under development, but they are coming. If I may suggest something I am personally working on, have a look at Simple.Data OData adapter (https://github.com/simplefx/Simple.OData, look at its Wiki pages for examples) - it has a NuGet package. While this a Client Library that only supports .NET 4.0, part of it is being extracted to be published as a portable class Library Simple.OData.Client to support .NET 4.x, Windows Store, Silverlight 5, Windows Phone 8, Android and iOS. In fact, if you check winrt branch of the Git repository, you will find a multiplatform PCL already, it's just not published on NuGet yet.
I want to know if using Self Tacking Entities (in Entity Framework) is recommended with WCF services? If yes, then can you guide me to a tutorial which may guide how to do that?
Actually, I am going to develop a WPF application using Prism with MEF and MVVM. I have decided to use Entity Framework. I want suggestions and advices regarding this approach.
Any help will be appreciated.
I want to know if using Self Tacking Entities (in Entity Framework) is
recommended with WCF services?
It depends who you ask. If you ask MS they will tell you Yes because they simply don't have anything better to offer. STEs were response to this very old MS Connect suggestion. The problem is that EF itself has terrible bad support for merging changes between two entity graphs (you must do it completely yourselves) and developers working on MS platform (sometimes including me) share some common behaviors:
They are lazy to develop their own solution to problem and they expect some magic directly in APIs provided by MS.
Most of the time they are not trained / skilled / competent in the technology they have to use, because they have to move to a new one too often.
The only APIs they know are part of .NET Framework. They don't look for other options neither they compare features.
First two points are result of MS strategy where RAD become synonym for designer (or newly also T4 templates).
I share #Richard opinion about STEs. I would add one additional drawback of STEs - they move large datasets between participants. If you decide to get an entity graph from the server, change a single entity in the graph and push data back they will transfer again the whole graph. Transferring only changed entities results in fighting with STE's core logic. I'm also afraid that they track changes completely on per entity level instead of per property level. In case of modification to entities with large binary or string data it can result in transferring too much unneeded data between the service and the database and between the service and the client.
Anyway for a simple application with low data traffic and small entities they can do a good job and allow you building your application quickly but without strict separation of concerns. You will get entities from service and bind them directly to WPF UI and they will be able to track changes for you. Later you will push entities back to service and they will be able to persist changes. Your client and service will be tightly coupled but in some scenarios it can be good enough.
I would avoid self tracking entities in general - I blogged about it here.
Create your own DTOs and use them to manage the transfer of data - then biuold your POCO objects in the service and use them with entity framework for persistence
If you want self tracking then there is a slightly cleaner approach here
In my SOA architecture, I have several WCF services.
All of my services need to access the database.
Should I create a specialized WCF service in charge of all the database access ?
Or is it ok if each of my services have their own database access ?
In one version, I have just one Entity layer instanced in one service, and all the other services depend on this service.
In the other one the Entity layer is duplicated in each of my services.
The main drawback of the first version is the coupling induced.
The drawback of the other version is the layer duplication, and maybe SOA bad practice ?
So, what do so think good people of Stack Overflow ?
Just my personal opinion, if you create a service for all database access then multiple services depend on ONE service which sort of defeats the point of SOA (i.e. Services are autonomous), as you have articulated. When you talk of layer duplication, if each service has its own data to deal with, is it really duplication. I realize that you probably have the same means of interacting with your relational databases or back from the OOA days you had a common class library that encapsulated data access for you. This is one of those things I struggle with myself, but I see no problem in each service having its own data layer. In fact, in Michele Bustamante's book (Chapter 1 - Page 8) - she actually depicts this and adds "Services encapsulate business components and data access". If you notice each service has a separate DALC layer. This is a good question.
It sounds as if you have several services but a single database.
If this is correct you do not really have a pure SOA architecture since the services are not independant. (There is nothing wrong with not having a pure SOA architecture, it can often be the correct choice)
Adding an extra WCF layer would just complicate and slow down your solution.
I would recommend that you create a single data access dll which contains all data access and is referenced by each WCF service. That way you do not have any duplication of code. Since you have a single database, any change in the database/datalayer would require a redeployment of all services in any case.
Why not just use a dependency injection framework, and, if they are currently using the same database, then just allow them to share the same code, and if these were in the same project then they would all use the same dll.
That way, later, if you need to put in some code that you don't want the others to share, you can make changes and just create a new DAO layer.
If there is a certain singleton that all will use, then you can just inject that in when you inject in the dao layer.
But, this will require that they use the same DI framework controller.
The real win that SOA brings is that it reduces the number of linkages between applications.
In the past I've worked with organizations who have done it a many different ways. Some data layers are integrated, and some are abstracted.
The way I've seen it most successfully done is when you create generic data-layer services for each app/database and you create the higher level services based on your newly created data layer.