We are trying to figure out how best to create a highly available Fuse Fabric infrastructure where there should not be any requirement for client side configuration. We mostly have CXF Rest services. If we create odd number of fabric containers and join them, will it create a highly available Fabric WITHOUT any client side configuration? Meaning, can the client point to one URL and Fuse Fabric will be able to fail over to any other container of the Fabric if one of them is down? I have read through multiple documents but could not find any direct answer.
Thanks.
You can use http-gateway http://fabric8.io/gitbook/gateway.html to expose single ip:port for a service that is highly available in fabric
Related
I am in a situation where I can use Service Fabric (locally) but cannot leverage Azure Service Bus (or anything "cloud"). What would be the corollary for queuing/pub-sub? Service Fabric is allowed since it is able to run in a local container, and is "free". Other 3rd party messaging infrastructure, like RabbitMQ, are also off the table (at the moment).
I've built systems using a locally grown bus, built on MSMQ and WCF, but I don't see how to accomplish the same thing in SF. I suspect I can have SF services use a custom ICommunicationListener that exposes msmq, but that would only be available inside the cluster (the way I understand it). I can build an HTTPBridge (in SF) in front of those to make them available outside the cluster, but then I'd lose the lifetime decoupling (client being able to call a service, using queues, even if that service isn't online at the time) since the bridge itself wouldn't benefit from any of the aspects of queuing.
I have a few possibilities but all suffer from some malady that only exists because of SF, locally. Also, the same code needs to easily deploy to full Azure SF (where I can use ASB and this issue disappears) so I don't want to build two separate systems just because of where I am hosting it in some instances.
Thanks for any tips.
You can build this yourself, for example like this. This uses a BrokerService that will distribute message-data to subscribed services and actors.
You can also run a containerized queuing platform like RabbitMQ with volumes.
By running the queue system inside the cluster you won't introduce an external dependency.
The problem is not SF, The main issue with your design is that you are coupling architectural requirements to implementations. SF runs on top of VirtualMachines, in the end, the only difference is that SF put the services in those machines, using another solution you would have an Agent Deploying these services in there or doing a Manual deployment. The challenges are the same.
It is clear from the description that the requirement in your design is a need for a message queue, the concept of queues are the same does not matter if it is Service Bus, RabbitMQ or MSMQ. Each of then will have the basic foundations of queues with specifics of each implementation, some might add transactions, some might implement multiple patterns, and so on.
If you design based on specific implementation, you will couple your solution to the implementation and make your solution hard to maintain and face challenges like you described.
Solutions like NServiceBus and Masstransit reduce a lot of these coupling from your code, and if you think these are not enough, you can create your own abstraction. Then you use configurations to tied your business logic to implementations.
Despite the above advice, I would not recommend you using different
solutions per environment, because as said previously, each solution
has it's own implementations and they might not assimilate to each other, as example, you might face issues in
production because you developed against MSMQ on DEV and TEST
environments, and when deployed to Production you use ServiceBus, they
have different limitations, like message size, retention period and son
on.
If you are willing to use MSMQ, you can add MSMQ to the VMs running your cluster and connect from your services without any issue. Take a look into this SO first: How can I use MSMQ in Azure Service Fabric
We are using prometheus in our production envirment recently. Before we only have 30-40 nodes for each service and those servers not change very often, so we just write it in the prometheus.yml, but right now it become too long to hold in one file and change much frequently then before, so my question is should i use file_sd_config to put those server list out of yml file and change those config files sepearately, or using consul for service discovery(same much easy to handle changes).
I have install 3 nodes consul cluster in data center and as i can see if i change to use consul to slove this problem , i also need to install consul client in each server(node) and define its services info. Is that correct? or does anyone have good advise.
Thanks
I totally advocate the use of a service discovery system. It may be a bit hard to deploy at first but surely it will worth it in the future.
That said, Prometheus comes with a lot of service discovery integrations. It's possible that you don't need a Consul cluster. If your servers are in a cloud provider like AWS, GCP, Azure, Openstack, etc, prometheus are able to autodiscover the instances.
If you keep running with Consul, the answer is yes, the agent must be running in every node. You can also register services and nodes via API but it's easier to deploy the agent.
I cannot seem to find any combination of tutorials or information online to set me in the right direction, so I'm hoping the community can help me out!
I have some experience with WCF in the past (mostly simple/default http implementations), but nothing to the level I am attempting with my current architecture. Unfortunately 99% of the info I'm finding for WCF is a couple of years old, and most of it does not address Azure specific details. Most books are published back in 2007, and do not address the newer IDE/Tooling or WCF updates since that time. Needless to say I have a few open questions, and would love to get pointed in the right direction after exhausting Google, Stack Overflow, MSDN & YouTube!
In a nutshell:
I want to centralize all business logic behind a single WCF service
on Azure (it will be load balanced on a Cloud Service).
I have a number of web clients that will be consuming this service.
All the clients are C#/.NET MVC projects that I control (I do not need or want the
WCF endpoints to be publicly available)
I would prefer to whitelist access to the endpoints, rather than
implement authentication (for performance & simplicity)
Hear are my questions and potential speed bumps:
Is WCF the right solution? Is there a newer better technology I should be using?
If I use a Cloud Service for my WCF solution, is WebRole or WorkerRole my best option and why? Are hosting the service as a Website an option? (It would save cost)
In my research I've landed on the fact that using NetTCP binding is faster than using the default Http bindings. But I can't find a simple example of how to set this up using VS 2013/.Net 4.5/Azure Cloud Service. Is there a good tutorial for this? Also, I'm assuming NamedPipes are not on option for me?
Since all the consumers of the WCF service will be running on Azure Websites, is NetTCP still possible? How do I create service references? I'm assuming I just use the NetTCP endpoint address, but what about whitelisting for security within the Azure infrastructure?
How can my Azure Website clients connect to TCP within Azure the fastest? Affinity groups don't seem to be an option for Websites, should I abandon this and deploy all my clients as WebRoles so they can share Affinity with my WCF Service? Is Azure smart enough to know that the website is calling a machine within the same region and keep the connection within the region? How is this ensured?
I will have a debug, stage and production environment for my WCF service. What is the best way to switch between the various endpoints on my azurewebsite client(s)? I'd prefer to do it during startup in my global.asax file using C#, rather than in my web.config. I only intend to keep one setting in my Web.Config for "Environment". Ideally I will have a Switch() statement in my startup file that will determine with WCF environment endpoint to use for my Service References.
My apologies for the array of questions. I was thinking about breaking this out into multiple posts, but keeping them in the same context seemed to be the only way to ensure that I am communicating the scope of my inquiry.
Thank you.
I found a great series of videos on Microsoft Virtual Academy that answers all of my questions:
Azure & Services
The key videos in this series are: 1,2 & 7. Here is a direct link to each one:
Intro to WCF
WCF on Azure
Advanced Topics
I have explored the web on MULE and got to understand that for Apps to communicate among themselves - even if they are deployed in the same Mule instance - they will have to use either TCP, HTTP or JMS transports.
VM isn't supported.
However I find this a bit contradictory to ESB principles. We should ideally be able to define EndPoints in and ESB and connect to that using any Transport? I may be wrong.
Also since all the apps are sharing the same JVM one would expect to be able to communicate via the in-memory VM queue rather than relying on a transactionless HTTP protocol, or TCP where number of connections one can make is dependent on server resources. Even for JMS we need to define and manage another queue and for heavy usage that may have impact on performances. Though I agree if we have distributed and clustered systems may be HTTP or JMS will be only options.
Is there any plan to incorporate VM as a inter-app communication protocol or is there any other way one Flow can communicate with another Flow Endpoint but in different app?
EDIT : - Answer from Mulesoft
http://forum.mulesoft.org/mulesoft/topics/concept_of_endpoint_and_inter_app_communication
Yes, we are thinking about inter-app communication for a future release.
Still is not clear when we are going to do it but we have a couple of ideas on how we want this feature to behave. We may create a server level configuration in which you can define resources to use in all your apps. There you would be able to define a VM connector and use it to send messages between apps in the same server.
As I said, this is just an idea.
Regarding the usage of VM as inter-app communication, only MuleSoft can answer if VM will have a future feature or not.
I don't think it's contradictory to the ESB principle. The "container" feature is pretty well defined in David A Chappell's "Enterprise Service Bus book" chapter 6. The container should try it's best to keep the applications isolated.
This will provide some benefits like "independently deployable integration services" (same chapter), easier clusterization, and other goodies.
You should approach same VM inter-app communications as if they where between apps placed in different servers.
Seems that Mule added in 3.5 version, a feature to enable communication between apps deployed in the same server. But sharing a VM connector is only available in the Enterprise edition.
Info:
http://www.mulesoft.org/documentation/display/current/Shared+Resources#SharedResources-DefiningDomains
Example:
http://blogs.mulesoft.org/optimize-resource-utilization-mule-shared-resources/
I am using OSGI with Glassfish 3.0.1.We use Jersey REST as the resource layer. We have lots of osgi services. we are planning to decouple them & deploy it in a cloud. One way we do this is by doing a http REST call. But we would like to do service to service call at the api level. One way to do this is by using DOSGI. But Glassfish 3.0.1 doesn't seem to support DOSGI. Any other suggestions?
I believe that Glassfish contains Apache Felix, which is a fully compliant OSGi Framework. Therefore you do not need explicit support from Glassfish in order to use a set of bundles that provide Remote Services (the name "DOSGI" is now deprecated). Indeed, this is kind of the point of OSGi!
Anyway the next obvious question is which Remote Services implementation to choose. I would advise you NOT to use CXF since it is too buggy and unmaintained. That leaves Eclipse ECF or Paremus RSA.
(Disclaimer: the Paremus implementation is commercial and I work for Paremus).