I am starting to plan a new platform which needs to integrate various services from various externals platforms. Essentially I'm tying together a bunch of internal, homegrown services and several outside services we license from 3rd parties.
Generally speaking the external services are all web services but they are a mishmash of REST, SOAP and XML-RPC.
Some of our internal services have REST API's but there are many things that aren't so easy: XMPP, Hessian, custom socket protocols, Java RPC, uWSGI, and the list goes on.
From my research it seems like an ESB like Apache ServiceMix might be a good fit for my needs. However it looks REALLY complex. I'm not launching rockets but I do need transactional messaging (mostly for eCommerce and entitlement stuff). I feel like the message queue ServiceMix uses under the hood (ActiveMQ) might be enough on its own.
Can anyone explain what ServiceMix provides above and beyond ActiveMQ? I know there is a lot but it is hard for an ESB n00b like me to really grasp the tangible difference when I'm waste-deep in buzzwords.
Thanks!
ServiceMix is an OSGi based container that allows you to deploy and run applications in a controlled runtime environment (like a J2EE container but less heavy weight and without programming to e.g. J2EE contracts).
Thanks to OSGi you can partition your applications into parts and update/evolve these parts independently from each other. You can upgrade parts of your application without having to take down the entire application. There is far better life cycle management in OSGi then you get with standalone Java processes.
If you think of creating an application that will evolve over time, then OSGi is something you should consider. And ServiceMix provides you a runtime OSGi container to deploy your applications to. I highly recommend the book "OSGi in Action" from Manning.
For tying together different external services that might even use different transport protocols I recommend Apache Camel, which btw also deploys nicely into ServiceMix.
Btw, existing applications can be deployed into an OSGi container with fairly little effort (without requiring code changes).
Torsten Mielke
FuseSource
Web: www.fusesource.com
Blog: http://tmielke.blogspot.com
Related
We are in the process of transitioning towards SOA.
Our current goal is to try and ensure that more of the application is developed as "Services" (mainly to improve visibility of capability, re-use and de-risk change). Some of those services will be exposed as web services, but many (and probably the majority) will not, and be used for "internal" use only to help reap some of the benefits of SOA.
For those "internal" services we are currently intending on implementing them as OSGi bundles; however we are struggling to understand how best to test them. Our goal is to enable the current System Test team to test all types of services and we have been investigating tools like SoapUI and SOA Test; however it's becoming clearer that we may face some challenges in testing our services implemented as OSGi bundles using tools like these; and indeed asking the test team to do so.
So we're looking for some advice on how best to test aspects of our capability designed to act as a "service", but implemented as an OSGi bundle instead of a web service.
What tools would people recommend, and is this a type of testing that's traditionally done by a developer during unit test, or can it be done by a less technical tester, undertaking the same basic principles of testing interfaces (i.e. inputs, processing, outputs)?
You could theoretically use a Remote Service Admin implementation (like Aries RSA or Eclipse ECF) to expose your internal services to the outside during testing to access them using an external system test tool.
I would not recommend to let an external team test your OSGi services though. It is much better to test the services in your own build using an integration testing tool like pax exam. It allows to define which bundles and other config to install. Then it boots up an OSGi framework with your setup and runs modified junit tests against it. The advantage is that such tests are quite realistic and still quite simple.
See here for some pax exam tests in aries rsa or apache karaf.
The first example uses the pax exam forked container for a very fast test (<1s per Test) while the second example uses the apache karaf container (~10s per Test) for tests that are very near a production system.
So you get much faster feedback than with an external system test team that will always lag a bit behind your current development. It also allows you to establish the policy that each team member runs the tests locally before committing.
Is it practical or wise to use Lagom in production without ConductR? The commercial licensing is putting me off. This framework looks like it could be pretty arduous to deploy and custom tooling for that can take a lot of effort to get right.
(disclaimer: I'm a Lightbend employee, currently core member of the Lagom team)
Edit (Nov 2018): Please refer to https://www.lagomframework.com/documentation/current/java/ProductionOverview.html#Running-Lagom-in-production for up to date information on this topic.
(original answer, Aug 2017)
A lot has changed in the Lightbend stack since this question was added over a year ago. For example: ConductR is now free to use in production for up to three nodes.
Also, the team behind ConductR is also working on providing tools to deploy a Lagom application on Kubernetes. The efforts on that front are very advanced and some of our sample apps can be deployed in Kubernetes already.
Your question is rather open-ended and so let's start with "it depends".
If
you're comfortable managing your scaling within the configuration of your Akka cluster,
your usage doesn't violate the open-source licensing terms of lagom, play and Akka, AND
you don't have sufficient cash flow to justify leveraging Lightbend's production suite,
then you arguably can deploy with a minimum of effort and custom tooling.
If those conditions don't hold, your options are to go elsewhere (e.g., Spring Cloud) or retain Lightbend. You may find going elsewhere has it's own cognitive load and/or commercial expense.
Hope that helps even 7 months later.
The other answers are higher level, but I can essentially say "yes." I'm currently deploying a Lagom service to be hosted on a Kubernetes cluster, and I'm not using ConductR or any of the commerically licensed components.
You will need to dig a little bit into some Play internals to start the service properly, and if you want to hook in with some other service locator you may need to implement one yourself, but it certainly isn't impossible and I think it's less effort on net.
Being honest it really worth to pay, because you got much more, good reporting dashboards, automatic cluster formation and what really cool is split brain resolver....
But sometimes agree when you are working on a project that don't have a lot of money, you can do some small tricks and get it work and may be then do all what really lucks or buy enterprise subscription.
So Lagom can be very easy being used without service discovery at all if you will delegate all to Kubernetes DNS or without Kubernetes just put load balancer before each service and use its address.
How this can works
Each lagom service that you have should be external service
Production run configuration should be mixed with
ConfigurationServiceLocatorComponents
Service that should communicate with another one should be started with extra parameter that tell lagom that this service is external one and can be communicated directly(all can be in JAVA_OPTS as for play application)
-Dlagom.services.your_service_name=http://k8s_service_name.default.svc.cluster.local:9000
My only experience with SOA comes from working with NServiceBus. It's really well designed and has some nifty features that I really like:
Processing of messages is transactional (and taken care of for you), so it's "once and only once" (as opposed to "at least once")
Built-in saga support
Automatic retry support
Pub/sub support (but I assume all service/message busses support this)
All that stuff leaves me to concentrate exclusively on the app's logic, but I've only used it in .NET, and Windows is almost certainly a non-starter for the project I'm working on. So my question is...
What service bus technologies out there exist in the *nix ecosystem (including experiences running NSB on Mono) and how do they measure up to NServiceBus running on Windows?
Bonus: In addition to the features above, what languages can send/receive/process messages from the bus? Having the flexibility to write one service in language A and another in language B would be very advantageous for this project.
SonicESB / MQ
I can speak from experience with SonicESB on Linux. It's basically SonicMQ - a JMS compliant MOM - with a process mediation layer on top; that's the service bus.
Pros
Itinerary-based routing; messages contain their own workflow for what services to visit
Intra-container messaging; increases performance by routing messages within the ESB container, avoiding the broker entirely.
Web service endpoints; ESB service may be invoked over HTTP endpoints, bridging the gap between non-Java/JMS clients and the bus.
Good development tools in eclipse for developing and debugging esb services and processes in Java.
Administration tools aren't bad either.
Cons
Transactions don't span service boundaries (may have changed since version 7.5)
No message retry mechanism; we rolled our own using a utility service and message parameter manipulation (may have changed in 7.5)
Pricey; licensed by core, so expect to pay for it.
Client Support
In terms of what technologies can send and receive to/from SonicESB, anything Java JMS, that's for sure. Sonic also offers a library in .net/C#/VB for Windows technology, so you can send and receive messages from the bus. This was a huge plus for us since our development teams use RIA's built in C#.
You may want to check out MuleESB, an open source service bus.
Hope it helps,
It has been quite a while since I've seriously used NServiceBus (in fact, back then, it wasn't even called that - it was just replacement library Udi wrote, to an awful comm library we had to use on a joint project :) ) so I don't know about all NServiceBus features.
I would say, based on what I do know, that Apache Kafka (originally developed by LinkedIn) would be a good starting point. It has relatively strong ordering guarantees, supports publish/subscribe to pools of consumers and is supportive of workflow behaviours e.g. Samza which provides a streaming framework on top of it
Also regarding the client interfaces. You can find the current list here
i'm about to decide on technology choices for an agent based application used in the transportaion systems domain.
basically there will be a central system hosting the backend, and multiple agents located across town (installed on desktops) that communicate with devices/kiosks collecting data and then transmitting them back to the central server. the central server could also be hosted on the cloud.
following are important
securing the data and communications between the device and the agent
and the agent and central server.
agents should be easily installable with little or no configuration.
near 100% uptime and availability
Does WCF fit the bill here?
if so what binding types should i go for? netTCP or wsHttp with SSL/HTTPS?
WCF is definitely a fit choice for this kind of scenario. For your bindings, the actual question is what technology you are going to use. Do you want to make the agents run in a non .NET environment like Java, then you should chose for wsHttpBinding. This binding communicates through SOAP and is very interoperable.
If you chose to use .NET agents, you might as well use netTcpBinding because they use the same WCF frameworks. It also supports binary encoding. If you really need to make a choice, take a look at the MSDN Documentation.
For your agents you could use a simple console application that runs in the background as a Windows service. WIX can help you with that (install an application as windows service), but thats all I know. WIX can also help you with basic installing and configure everything for you but it has a high learning curve so you might need to invest time in it.
Sorry for my ignorance here, but when I hear the word webserver, I immediately imagine Apache, although I know people use Microsoft's IIS too. However since I've been hanging out here at Stackoverflow I've noticed lots of people use Glassfish.
Which made me wonder, why would I want to use Glassfish (in the sense that I'm interested, but I don't really understand why it might make my life easier). From what I read it's Sun's open-source derivate of Apache's Tomcat, thus I imagine it's a good (or great) quality product. But since I don't know its strengths and weaknesses, I don't know when it would be wise to choose Glassfish over another server. Could anyone elaborate ?
GlassFish is an Application Server which can also be used as a Web Server (Http Server).
A web Server means: Handling HTTP requests (usually from browsers).
A Servlet Container (e.g. Tomcat) means: It can handle servlets & JSP.
An Application Server (e.g. GlassFish) means: It can manage Java EE applications (usually both servlet/JSP and EJBs).
You should use GlassFish for Java EE enterprise applications.
The need for a seperate Web server is mostly needed in a production environment. You would normally find a Application server to be suffice most of your development needs. A web server is capable of holding larger number of active sessions and connections, thus providing the necessary balance without performance costs.
Stick to a simple web server if you are only working with servlets/jsps. It is also to be noted that in a netbeans environment, glassfish has better support than other App servers. In the context of eclipse though, WSAD and JBoss seem to the preferred options.
Glassfish will soon release the modular kernel.
This means that the containers you need start up and shutdown as you need them. I.e no EAR deployed, EJB container won;t start up. This seems to have made it very good for development as it can start and stop very quickly. This takes it a lot closer to development environments like Rails (where redeployment is a massive part of your development)
I have used GlassFish server for developing Web Services.
It provides a very interactive Admin Console where admin can test the Web Services.
I really find it helpful while developing Web Services