I'm working with my team on the "core" domain of our job, we have several DBs, Webservices, importer, etc... We expose the domain services to other teams through a backend WCF webservice. This webservice is cut in several api (or areas).
We deploy this on several production server (~ 80) (1 server <-> 1 customer).
Problem is: Some api goes public and we must maintain the same contract throught many releases. Others team want more flexibility (3 internal api contract version) as they can't follow our delivery rate (longer cycle)
Actually we are only serving this as a whole, and we are using a classic three branch branching (DEV/MAIN/RELEASE'n')
I'm trying to upgrade my TFS Team collection organization to handle a defined version range of each API version:
API Foo v2 (hxxp://-host-/api/foo/v2/...)
API Foo v3 (hxxp://-host-/api/foo/v3/...)
API Bar v1 (hxxp://-host-/api/bar/v1/...)
API Bar v2 (hxxp://-host-/api/bar/v2/...)
Currently we have the following branching schema:
$\MyTeamCollection
|___\DEV {Branch}
|___\Databases
|___\MyDBProj (SSDT)
|___\Documents
|___\MyWebService\MyWebSvc.Usage.docx
|___\Projects
|___\MyWebService.Core
|___\MyWebService.API.Bar
|___\MyWebService.API.Foo
|___\Shared
|___\MyDB\MyDB.DacPac, etc...
|___\MyWebService\MyWebsvc.Bar.Contract.Dll <-- WCF DataContract/ServiceContract
|___\MyWebService\MyWebsvc.Foo.Contract.Dll
|___\MAIN {Branch} <-- This one is what is on the integration server
|___\R1 {Branch} <-- This one went on a production server and is kept aside
|___\R2 {Branch}
|___\Archive
|___\R0 {Branch} <-- This one is obsolete but is archived here
Actually it handles a group of single api versioning. => Each release ship with a set of latest api version (Foo, Bar)
My question is how can I evolve this to handle multiple Api Version in each release (Foo-v1, Foo-v2, bar-v3, bar-v4)
Should i create branch inside branch? Won't this be a nightmare?
Any actual working teamcollection organization on your side?
Thanks,
-Jeremy.
Some additions:
All my clients are .Net, we provide them the assembly named MyWebsvc.Foo.Contract.Dll. This assembly contains:
ServiceContract interface which is implemented by a proxy pattern
[ServiceContract]
public interface IFooService {
Foo Get(int Id);
}
And all data contract needed (DTO + Serialization)
[DataContract]
public class Foo {
// ...
}
On the web service side there is the actual implementation
public class FooServiceImpl : FooApi.IFooService { ... }
For the purpose of this answer, I imagine that you have thoroughly reviewed the abundance of version control branching and merging guidance available online. As such, I suspect that the answer may not be found in “more” version control, but rather in a modest shift in the way features are enabled and available for customer use. Therefore, you may find that implementing a feature toggle solution will reduce the number of release versions that you need to maintain. The following article by Martin Fowler provides a good overview of feature toggles.
http://martinfowler.com/bliki/FeatureToggle.html
Our development team has utilized the concept of feature toggles, in our self-hosted WCF Windows Services, to provide a fully configurable set of endpoints that can be enabled/disabled (and adjusted) via configuration. In such scenarios, we typically utilize the customers’ data repository (database, ldap, etc) to store the configuration values and toggle switches, then when the service runs, it searches the repository for the configuration and then sets up and enables the interfaces in the manner the customer needs/wants.
Hope this helps.
Regards.
Jeremy,
there are a few questions you need to consider.
a) are your clients .Net as well or can they be in any technology? WCF has a really nice "compatible" versioning mechanism, but unfortunately it's non standard and only works with .Net clients
b) do you have the ability to give some guidance as to how the clients consume the services?
The only way to keep this problem under control is to adopt a "compatible" versioning strategy. As you pointed out, you want to have several (max 3) major releases in production and each time you evolve one or more major release you want to create a compatible (minor) version such that there is always one and only one release for each major version of the service running.
From a TFS perspective, I would consider each major version independent from each other, as in essence, they are somewhat independent services. I understand that creates a bit more work since you could have some update that applies to more than one major release, but IMHO, it's not worth the trouble to try to optimize that, unless you are actively working on all major versions simultaneously.
Achieving "compatibility" from one version to the next is not that hard, there are number of guidelines you want to follow both on the service side and the client side. I co-authored this article way back in 2008. http://www.infoq.com/articles/contract-versioning-comp2
I also wrote that blog post recently specifically for HTTP based APIs: http://www.ebpml.org/blog2/index.php/2013/04/13/formalizing-the-api-style
I want to emphasize that it's generally a good idea to use a minor version that is provided by the client to indicate which version of the service was in effect when it was built. The reason for that is that you can handle small breaking changes in a compatible way by knowing which minor version of the service it is compatible with. You still have a single release per major version and the variants are directly handled by the latest (minor) version of the service.
Versioning is key to a healthy API that allows clients to upgrade on their own terms (risk, budget, resources,...) and to the best of my knowledge, a "compatible versioning" strategy is the one that works best.
Related
He are having a debate about a proper way to support versions in SOA.
One option: is to implement all endpoints (Version/Contracts) in the same application in a strict manner. Thus supporting backward compatibility in the same application.
Second option: to create branches with single endpoint. Deploy each endpoint in a different application.
Each option has its pro/cons. Most of the sites I've notice support strategies in the same application (code base) strict/agile/flexible.
But there are also benefits to the second options. By breaking the endpoint to branches and different deployments each endpoint become more stable and detached from the others. Allowing developers to not consider backward compatibility when they write new functionality. I guess it also makes the 3rd party integration harder (changes to DB scheme etc.).
What do you think?
Definitely option 2!
To solve the DB issues you can use a repository for each micro service, and to avoid multiplicity, handle the generated ORM entities in a shred kernel.as i see it, the idea of a micro service is that it is a stand alone process, while if you go with the 1st approach you are coupling between services that are not related(NOT IN THE SAME DOMAIN => DDD) + the backward compatability issue, every time you want a fix you have to: or update all micro services, or ask the clients to update their side.
sincerely YBM
Recently I've been working with MVC4 and have grown quite comfortable with the View > View Model > Controller > Service > Repository stack with IoC and all. I like this. It works well. However, we're moving towards company wide application platform that will serve the majority of all business applications needs within the company.
Basic architecture goals:
Client facing MVC site
Internal Admin web site
Plethora of scheduled jobs importing/exporting data/etc to third parties
Service Bus sitting in the middle to expose business events
Public API for customer consumption
My initial thoughts are to introduce an "enterprise service layer" by applying my service interfaces to WCF contracts and registering the WCF proxy classes in my IoC. This would allow me to reuse the same pattern I'm currently using, butt I've not found a lot of examples of this in practice. Except this guy.
Admittedly though, I'm unsure what the best solution is for a project this scale.
1) What are the considerations when centralizing business services?
2) How does this affect cross cutting concerns like validation, authorization, etc? I thought I had that figured out already, but putting DTOs between the layers changes all this.
3) I'm experienced with WCF, but I hear Service Stack is all the rage...Should SS be a consideration with its RESTful goodness?
This guy here. I am not an expert in this area by any means, but hopefully I can provide a bit more context around things.
The main problem with using IoC to resolve WCF ChanelFactory dependencies as per my post is that the client also needs to have access to the service contracts. This is fine for a View > View Model > Controller > Service > Repository type architecture but is not likely to be possible (or desirable) for a shared public API.
In an attempt to cover your other questions:
1) Some of the concerns are already mentioned in your second question. Add to that things like security, discoverability, payload type (XML, JSON etc), versioning, ... The list goes on. As soon as you centralize you suddenly gain a lot more management overhead. You cannot just change a contract without understanding the consequences.
2) All the cross cutting stuff needs to be catered for in your services. You cannot trust anything that comes in from clients, especially if they are public. Clients can add some validation for themselves but you have to ensure that your services are locked down correctly.
3) WCF is an option, especially if your organisation has a lot of existing WCF. It is particularly useful in that it supports a lot of different binding types and so means you can migrate to a new architecture over time by changing the binding types of the contracts.
It is quite 'enterprisey' and has a bewildering set of features that might be overkill for what you need.
ReST is certainly popular at the moment. I have not used Service Stack but have had good results with Asp.Net Web Api. As an aside, WCF can also do ReST.
I've previously provided a detailed explanation of the technical and philosophical differences between ServiceStack and WCF on InfoQ. In terms of API Design, this earlier answer shows the differences between ServiceStack Message-based approach and WCF / WebApi remote method approach.
SOAP Support
ServiceStack also has Soap Support but you really shouldn't be using SOAP for greenfield web services today.
HTML, Razor, Markdown and MVC
ServiceStack also has a great HTML Story which can run on stand-alone own with Razor support as seen in razor.servicestack.net or Markdown Razor support as seen in servicestack.net/docs/.
ServiceStack also integrates well with ASP.NET MVC as seen in Social Bootstrap Api, which is also able to take advantage of ServiceStack's quality alternative components.
Setting: I'm developing an intranet tool set for my department, the main point of which is to centrally manage data quality and accessibility, but also to automate and scale some partial-processes.
Problem: I currently have my business logic in a CLR assembly, which is available on my SQL-Server for other CLR assemblies that run automated ETL directly on the SQL-Server. I am also developing an intranet site, which also needs the code information in that business logic assembly, but referencing the CLR assembly code has been working out sub-optimally, in terms of deployment and code maintenance. Also another department has voiced interest in using the code-base and data for their own intranet site.
Question(s): I've read quite a few Q&A(1,2,3,4,...) on SO to this topic, but I find it a very encompassing, so I'll try to ask questions for a more specific case(i.e. a single BL and Data Access code base)
Is a WCF service the solution I want? All my potential service clients run on the same server, is there maybe another way to reference the same code base both in CLR assembly and website projects? I don't need support for different platforms(ex. Java) - everything is .NET(yay for in-house progr!) - is WCF overkill?
Can code from a WCF service be used like a class library, or do I need to program a new way for accessing classes/methods from the service?
Separation of Development, Test and Productive instances?
Can a WCF service be updated while clients are accessing it, or do I need to schedule maintenance windows? When I update the service, do I need to update the client as well in some way?
Can I dynamically set the service reference, like I currently am dynamically setting the database connection string, depending on if StageConfig = dev, test, or prod?
My CLR assemblies are written for .Net 3.5, but the websites for .NET 4.0, will that pose a problem?
What minimum set of .NET service architecture programming do I need to know to accomplish this? I'll learn more about WCF with time, but I need to evaluate architecting effort and weigh it against getting things done(feature requests). Does the MS tutorial get me the desired skill?
I appreciate answers to only single questions, if you feel you know something, I'll +1 whatever helps me get closer to a complete answer.
OK, so you want to make your code enterprise-wide. There are two fundamental problems to talk about when you want to do this, so I'll structure the answer that way:
You have to understand what WCF is all about.
You have to manage your dependencies correctly.
What WCF is about
WCF is a way of doing RPC/RMI (Remote procedure call/remote method invocation) which means that some client code can call code that is located somewhere else through the network.
A callable WCF service is determined by the ABC triplet:
The service specification is implemented as a .NET interface with a "ServiceContract" attribute. This is the Contract ("C")
The "location" of the service is determined by a pair : Address ("A") and Binding ("B"). The Binding determines the protocol suite to be used for communication between client and server (NetPipe, TCP, HTTP, ...). The Address is a URI following the scheme determined by the Binding ("net.pipe", "net.tcp", "http", ...)
When the client code calls a WCF service at a specific Address, with a specfic Binding, and a specific Contract (which must match what the server at the specific Address and the specific Binding is delivering), WCF generates a proxy object implementing the interface of the contract.
The program delivering the service is any .NET executable. It has to generate one or many WCF Hosts, that will register objects or classes that implement the service contract, and asociate each delivered service to a specific Address and Binding. (possibly many thereof)
The configuration can be through the app .config file, in which you will be specifying ABC triplets and assotiate these triplets with a name that you will use in your application. You can also do it programmatically, which is very easy.
WCF does not address your problem of deploying your application, or the configuration of addresses and binding. It just addresses the problem of letting two executables communicate with each other with strongly-typed objects (through a specific interface). Sharing the service configuration is up to you. You may use a shared .config file on a Windows share, or even set up a LDAP server that will deliver all the data you need to find your service (namely A and B).
Managing your dependencies correctly
In your scenario, there are three actors that want to use your WCF infrastructure:
Your SQLCLR assembly, which will be a client.
The intranet site, which will be another client.
The service host, which will be a server.
The bare minimum number of assemblies will be 4. One for each of the aforementioned actors, and one specifying the contract, which will be used by all three actors. It should contain the following things:
The interface specifying the contract.
All types needed by the interface, which will of course be sent through the network, and therefore must be serializable.
There should be nothing more in it, or else, it will be a maintenance nightmare.
Answer to your questions
I hope that my answer is clear. Let's sum up the answers to your questions.
Is a WCF service the solution I want? All my potential service clients
run on the same server, is there maybe another way to reference the
same code base both in CLR assembly and website projects? I don't need
support for different platforms(ex. Java) - everything is .NET(yay for
in-house progr!) - is WCF overkill?
Everything is overkill. WCF is rather easy to use and scales down very well.
Can code from a WCF service be used like a class library, or do I need
to program a new way for accessing classes/methods from the service?
Setting up a WCF on existing code requires only the implementation of an additional class, and some code creating the Hosts which will serve the aforementioned class.
Calling a WCF service requires the creation of a Channel, which is a .NET (proxy) object implementing the interface.
So basically, your business code remains in the same state.
Separation of Development, Test and Productive instances?
WCF does not take care of that. Different environments, different service addresses. You have to take care of this yourself.
Can a WCF service be updated while clients are accessing it, or do I need to schedule maintenance windows?
It depends on your maintenance policy. Kill the serving process and launch the new version is the basic upgrade mechanism.
When I update the service, do I need to update the client as well in some way?
Provided that you manage your dependencies correctly like I sketched in the previous section, you need to update the clients only if the service specification (the interface) changes.
Can I dynamically set the service reference, like I currently am dynamically setting the database connection string, depending on if StageConfig = dev, test, or prod?
You have to manage that, probably by etting Address and Binding for a service programmatically.
My CLR assemblies are written for .Net 3.5, but the websites for .NET 4.0, will that pose a problem?
Provided that you manage your dependencies correctly like I sketched in the previous section, the only constraint will be the minimum CLR version required by the "contract" assembly.
What minimum set of .NET service architecture programming do I need to know to accomplish this? I'll learn more about WCF with time, but I need to evaluate architecting effort and weigh it against getting things done(feature requests). Does the MS tutorial get me the desired skill?
You'll need the result of these exercises:
Make two executables, a client and a server, that will communicate
through a WCF contract located in a separate DLL. The configuration
should be located in the app .config file.
Make two executables, a client and a server, that will communicate
through a WCF contract
located in a separate DLL. The configuration should be determined programatically.
Try to send a serializable class as a parameter to your service.
Try to send a serializable class as a return value of your service.
After that, you'll need to think about the best/cheapest way to share the Addresses and Bindings of your services.
Hope it helps.
I was thinking about the architecture of a web application that I am planning on building and I found myself thinking a lot about a core part of the application. Since I will want to create, for example, an android application to access it, I was already thinking about having an API.
Given the fact that I will want to have an external API to my application from day one, is it a good idea to use that API as an interface between the interface layer (web) and the business layer of my application? This means that even the main interface of my application would access the data through the API. What are the downsides of this approach? performance?
In more general terms, if one is building a web application that is likely to need to be accessed in different ways, is it a good architectural design to have an API (web service) as the interface between the interface layer and business layer? Is REST a good "tool" for that?
Sounds like you've got two questions there, so my answer is in two parts.
Firstly, should you use an API between the interface layer and the business layer? This is certainly a valid approach, one that I'm using in my current project, but you'll have to decide on the benefits yourself, because only you know your project. Possibly the largest factor to consider is whether there will be enough different clients accessing the business layer to justify the extra development effort in developing an API? Often that simply means more than 1 client, as the benefits of having an API will be evident when you come to release changes or bug fixes. Also consider the added complexity, the extra code maintenance overhead and any benefits that might come from separating the interface and business layers such as increased testability.
Secondly, if you implement an API, should you use REST? REST is an architecture, which says as much about how the remainder of your application is developed as it does the API. It's no good defining resources at the API level that don't translate to the Business Layer. Rest tends to be a good approach when you want lots of people to be able to develop against your API (like NetFlix for example). In the case of my current project, we've gone for XML over HTTP, because we don't need the benefits that Rest generally offers (or SOAP for that matter).
In general, the rule of thumb is to implement the simplest solution that works, and without coding yourself into a corner, develop for today's requirements, not tomorrow's.
Chris
You will definitely need need a Web Service layer if you're going to be accessing it from a native client over the Internet.
There are obviously many approaches and solutions to achieve this however I consider the correct architectural guideline to follow is to have a well-defined Service Interface on the Server which is accessed by the Gateway on the client. You would then use POCO DTO's (Plain old DTO's) to communicate between the endpoints. The DTO's main purpose is to provide optimal representation of your web service over the wire, it also allows you to avoid having to deal with serialization as it should be handled transparently by the Client Gateway and Service Interface libraries.
It really depends on how to big your project / app is whether or not you want want to go through the effort to mapping your DTO's to the client and server domain models. For large applications the general approach would be on the client to map your DTO's to your UI Models and have your UI Views bind to that. On the server you would map your DTO's to your domain models and depending on the implementation of the service persist that.
REST is an architectural pattern which for small projects I consider an additional overhead/complexity as it is not as good programattic fit compared to RPC / Document Centric web services. In not so many words the general idea of REST is to develop your services around resources. These resources can have multiple representations which your web service should provide depending on the preferred Content-Type indicated by your HTTP Client (i.e. in the HTTP ACCEPT HEADER). The canonical urls for your web services should also be logically formed (e.g. /customers/reports/1 as opposed to /GetCustomerReports?Id=1) and your web services would ideally return the list of 'valid states your client can enter' with each response. Basically REST is a nice approach promoting a loosely-coupled architecture and re-use however requires more effort to 'adhere' to than standard RPC/Document based web services whose benefits are unlikely to be visible in small projects.
If you're still evaluating what web service technology you should use, you may want to consider using my open source web framework as it is optimized for this task. The DTO's that you use to define your web services interface with can be re-used on the client (which is not normally the case) to provide a strongly-typed interface where all the serialization is taken for you. It also has the added benefit of enabling each web service you create to be called by SOAP 1.1/1.2, XML and JSON web services automatically without any extra configuration so you can choose the most optimal end point for every client scenario, i.e. Native Desktop or Web App, etc.
My recent preference, which is based on J2EE6, is to implement the business logic in session beans and then add SOAP and RESTful web services as needed. It's very simple to add the glue to implement the web services around those session beans. That way I can provide the service that makes the most sense for a particular user application.
We've had good luck doing something like this on a project. Our web services mainly do standard content management, with a high proportion of reads (GET) to writes (PUT, POST, DELETE). So if your logic layer is similar, this is a very reasonable approach to consider.
In one case, we have a video player app on Android (Motorola Droid, Droid 2, Droid X, ...) which is supported by a set of REST web services off in the cloud. These expose a catalog of video on demand content, enable video session setup and tear-down, handle bookmarking, and so on. REST worked out very well for this.
For us one of the key advantages of REST is scalability: since RESTful GET responses may be cached in the HTTP infrastructure, many more clients can be served from the same web application.
But REST doesn't seem to fit some kinds of business logic very well. For instance in one case I wrapped a daily maintenance operation behind a web service API. It wasn't obvious what verb to use, since this operation read data from a remote source, used it to do a lot of creates and updates to a local database, then did deletes of old data, then went off and told an external system to do stuff. So I settled on making this a POST, making this part of the API non-RESTful. Even so, by having a web services layer on top of this operation, we can run the daily script on a timer, run it in response to some external event, and/or have it run as part of a higher level workflow.
Since you're using Android, take a look at the Java Restlet Framework. There's a Restlet edition supporting Android. The director of engineering at Overstock.com raved about it to me a few years ago, and everything he told us was true, it's a phenomenally well-done framework that makes things easy.
Sure, REST could be used for that. But first ask yourself, does it make sense? REST is a tool like any other, and while a good one, not always the best hammer for every nail. The advantage of building this interface RESTfully is that, IMO, it will make it easier in the future to create other uses for this data - maybe something you haven't thought of yet. If you decide to go with a REST API your next question is, what language will it speak? I've found AtomPub to be a great way for processes/applications to exchange info - and it's very extensible so you can add a lot of custom metadata and yet still be eaily parsed with any Atom libraries. Microsoft uses AtomPub in it's cloud [Azure] platform to talk between the data producers and consumers. Just a thought.
For a project I have to implement a communication between a database hosted on a web server and several clients out there in the internet.
After reading a bit and watching a few introductory videos about possible (Microsoft) technologies I figured out that I seem to have (at least) three options:
1) Windows Communication Foundation (WCF)
2) ASP.NET Web services
3) ADO.NET Data services
Because I am not familiar with any of those three technologies I have to learn (hopefully only) one of them in depth - and the question is: Which one?
Or to be more precise: Which one for the following task?
Data need to be uploaded from a client to the server/database and some other data need to be downloaded as well. On client side this will not happen interactively by an user who works in a browser but rather as an automatic process on the client which will run periodicly (every 2 hours for instance).
a) On web server side there will be:
A SQL Server database
.NET Framework 3.5 SP1
A class library representing the database structure and modeled with ADO.NET Entity Framework
(An ASP.NET web application which will present the data in the database in a browser: I put this in brackets here because this web application doesn't really matter since the mentioned data up/download won't be triggered through a browser GUI.)
b) The client side is less clear and must be more flexible. Here I have to distinguish between two requirements:
i) Priority One (in terms of the time I have available for development):
The client side is under my control, it means: I have any Windows OS on the client, I can install .NET Framework and I can decide to develop a Windows service, a Console Application, a Windows Forms application or whatever. And I have knowledge about and access to the class library mentioned above.
ii) Lower Priority, but must be a future option:
I need to expose any kind of interface description which allows other developers to build their own applications to upload/download data.
The clients which others will develop for can be operated with any kind of OS (Windows, UNIX, MacOS and more). Also they shall be as free as possible to select their own prefered programming language.
Due to the last point forcing developers to use the .NET Framework on client side is not an option. The communication on client side must be any kind of "standard" technology accessible from various platforms and languages. I was reading terms like "SOAP", "REST" or "AtomPub" during my little research and these seem to be a kind of standard protocol or communication technology (not a proprietary Microsoft invention). But I am not sure nor do I know which technology is "up to date", has the "best future", is most common and well known, is the most powerful or is the easiest to use (from a viewpoint of those other possible developers! So the question is what I have to support to make most client side developers happy).
One last point: Security is important! Uploading/Downloading data must be restricted to dedicated persons. It should not be possible to use or explore the interface without appropriate credentials.
What technology is the best to use now? (1), (2) or (3)? And why would you recommend it?
Thank you very much in advance for any advice!
Well, no. (2) old-style ASP.NET webservices is on its way out - it's old, no longer being developed - it's been replaced by WCF.
So this leaves options 1 (straight WCF) and 3 (ADO.NET Data Services - renamed WCF Data Services recently).
Both use WCF as their basic technology - so learning and knowing about WCF is a must in both cases.
With straight WCF (Option 1) you have more options - you can self-host, host your service in IIS, use different protocols and bindings and so on. But with choice comes complexity - you need to learn and know all that stuff - at least to some degree. Your client needs to be able to talk SOAP to you - just about any language (.NET of course, Java, Ruby, PHP - you name it) can talk SOAP in one way or another.
If you're mostly interested in exposing data from databases onto outside clients, I think WCF Data Services is indeed quite a good choice. It's based on REST, so you can hit your WCF Data Service with a browser and just see what happens. It's quite powerful, and even offer a LINQ client side support - you formulate a LINQ query and this gets translated into your appropriate REST call to your data service.
With WCF Data Services, your client needs nothing but a HTTP stack - even the iPhone has that :-) But with a .NET client, things are of course nicer and more comfortable and more efficient.
I'd say check out the WCF Data Service first and see if that satisfies your needs - and if not, dig deeper into WCF. Also check out WCF Data Service at a glance for an intro.
UPDATE:
Marc, do I understand you correctly
that WCF on the server fulfils this
requirement? And ADO.NET (WCF) Data
services too?
Absolutely. WCF (plain or with Data Services) on the server side does NOT dictate the client in any way, shape or form. You can hook up an iPhone to a WCF Data Service, if you really feel like it :-) WCF has been designed from the ground up to be very interoperable - actually, it's the one platform out there that implements the most WS-* industry standards for cross-platform communications.