I have already working Camel configuration that is watching a database table (through spring and hibernate), and when something shows up in DB, Camel consumes it and sends message to JMS broker.
This works flawlessly. And is built in DSL in MyOwnMessageRouteBuilder.configure()
Now I'd like to add monitoring that do something if no new data shows up in DB in given (like 3h) time. Is that possible in Camel at all? I can see callbacks like onCompletion or onException, but nothing like onIdle()....
Best regards
You may look at BAM
http://camel.apache.org/bam
However usually some monitoring tooling may already be able to do this, and thus you may be able to find a generic solution.
I think your best best is to use a timer/quartz route to check the database periodically and compare the timestamp of the most recent data vs. current time...if its greater than 3h old, then react accordingly...
Related
Say I have like 1000 VMs with different services running on them with different technologies used like python, NET, java and different middleware like rabbitmq, redis etc.
How can I dynamically handle the interactions between the services and provide scalability?
For Example, say I have Service A which is pushing Data to a rabbitmq then the data is processed by service B while fetching additional data from Service C. You see at the end I have a decentralized system which is pulling data somewhere and pushing it somewhere else... a total mess! Scale it up to 2000 microservices omg XD.
The moment I change one thing a lot of other systems are affected.
Do you know something maybe like an ESB where I can couple two services together with a message transform adapter in the middle of it and I can change dependenciesat runtime? Like the stream doesn't end in service F anymore and does end in G for example?
I think microservices are a good idea because they can be stateless, can scale, can easily be deployed as a container. But I don't know a good tool/program for managing the data flow. The rabbitmq doesn't support enough enterprise integration patterns. Do you have any advice?
How can I dynamically handle the interactions -
See if using an existing EIP pattern solves your problem to implement the logistics
Depending on how your design shapes up, you may need to use Distributed Lock Management
Or maybe your application is simple enough to use a Consul K/V store as a semaphore & a simple mosquitto topic based bus.
Provide scalability
What is the solution you are trying to scale? AMQP, Consul, "microservices" in themselves are very scalable & distributed
However, to scale your thought process & devops, you need to find a way to see things as patterns that help you split the problem & tackle the complexity
Do you know something maybe like an ESB where I can couple two services together with a message transform adapter in the middle of it and I can change dependenciesat runtime?
Read up on EIP. ESBs are just one of the many ways you can solve your problem. RTFM, & get some perspective.
But I don't know a good tool/program for managing the data flow.
Ask yourself if your problem is related to distributed workflow management, or if a data pipeline is what you are really looking for
Look at Spark, Storm, Luigi, Airflow - they all have a different purpose - but you will know what to do with them if you manage to read up on everything else in this post ;)
I am quite new in the HL7 field and not a developer, so sorry if my question might seem to be too obvious.
We want to develop an app for a hospital which visualises performance and patient-flow data by aggregating data from other hospital applications. Our app will both visualise realtime data and historic data. During talks with the head of IT I got confused, he explained I need to:
Develop an HL7 listener like Mirth which can receive messages of other applications which communicate via HL7 2.x standards to catch realtime data and after this organise to migrate historic data from other applications via sql queries. Sounds pretty logic, though not sure if he's an expert since he had no idea what an API was and knew nothing about FHIR.
My questions are:
1 What triggers an application to send an HL7 2.x message around to other application when for instance someone changes the status of a patient? Is it programmed to automatically send a message with each change in record just randomly around? So assuming all applications do this standardly and you just need a listener like Mirth to catch those messages and migrate into my own database?
2 Can't I use the HL7 2.x standard to pull info via a query out of a database? Meaning can it be used for two-way communication? I send query, application sends me the data in an HL7 message? Meaning I can also use it to pull historic data from another database?
3 What kind of difference would the use of FHIR standard have in this situation? I believe it can definitely be used to pull information from another database. But would it in fact make a difference compared with the tactic which the tech guy is advising me, which is migrating historic data to my own database and further just catch new changes by receiving hl7 2.x messages?
4 Would it be an advise to use an FHIR RESTful API to pull/receive info from applications which still use HL7 2.x standard? So for both historic as realtime changes? Would this be a faster way of integration, or better to use the old fashioned way the Tech guy advises me.
Very keen to know more about this, since I want to organise a strategy which is future proof and won't cost months of integration time every time we migrate to a new hospital.
Thanks for your help guys!
depends on the application. most only send data, and it's configurable when and why.
no, you use hl7 v2 to pull data out of an application, not a database - if, that is, the application supports it. Many (most?) don't. And you can only do what the applcation allows
FHIR would be a lot easier to use, but it's still settling, and you'll have trouble finding applications that offer a fhir interface this year. you'll have to talk to potential customers to find out whether it's possible. btw, FHIR can do what v2 can in this regsard - both pull and push
it's always to advisable to use FHIR - if you can. mostly, though, you'll have to use v2 because that's what's on offer.
I hope you can help me with this:
I have two database tables in separate servers, and I want them to be synchronized, I mean that when one of them is modified (Insert, delete, update), the other one is modified too. I´ve been searching for a while now and I´ve found that this can be acomplished with ActiveMQ, but, I haven´t found the way of doing It, can anybody give me a clue or a tutorial or something?.
I really appreciate your help.
Thanks in advance.
Is there any particular reason you want to mix in ActiveMQ for the task?
ActiveMQ is a message broker to send event messages around. There is no out-of-the-box database synchronization of DB events with ActiveMQ. You probably have to use Apache Camel (or custom code) to read and write the databases. That would be a non trivial task, non the less, since there are things as transactions, table locks you need to take into account.
If replication is all that is needed for HA or backup, you should really look at the built in mechanisms of SQL server.
Is it possible to use NServiceBus with a database as the queue storage instead of MSMQ? If so, how can I get started and what are the pros and cons of using a database instead of MSMQ?
If you want to use something other than MSMQ you'll have to plug in your own ITransport. I would take a look at the NSB Contrib project on GitHub, there is an implementation of of ITransport for the SQL Server Broker(messaging).
The cons I see for using a database includes cost and maintenance overhead. MSMQ comes with the OS for free and most admins have the skills to maintain it. Once you get in a DB, you have to pay for it and find someone to maintain it. This starts out ok, but once you get into multiple environments and things like clustering, licensing gets out of control.
I work in a small supermarket chain (4 stores). Each store has its own local database which contains information of each product, prices, and transactions that have ocurred on the store. In addition, each store needs to replicate this information back and forth to a central location.
Right now we are using something called SQLRemote, which is a feature of Sybase's SQL Anywhere database. It works, but sometimes fails and is difficult to manage. To its' credit, SQLRemote actually wasn't designed for this type of scenarios, so it could be said that we are using it incorrectly.
I was thinking that an ESB system such as Mule (or ChainBuilder which seems easier to set up) might be a good alternative to SQL remote. I understand that these systems can detect when changes occur in the database (i.e. when records are added, modified or deleted), and can be set up to deliver a message in a transaction.
Would this be a viable solution to my scenario?
Best regards,
Edgard
Yeah I am sure Mule should be able to do this.
However I work for a company which provides Fuse ESB which is using Apache projects such as Apache ServiceMix, Apache ActiveMQ, Apache Camel and Apache CXF.
We have a user story about a very big retailler in US which uses Fuse ESB to integrate their stores and warehouses and whatnot
http://fusesource.com/collateral/17
Fuse ESB
http://fusesource.com/products/enterprise-servicemix/
Yes, Mule can support this scenario thought it might be overkill. There are targeted database replication solutions out there. The advantage of Mule would be it's ability to handle failure and other scenarios where you need the workflow to be adapted based on what is happening. This allows you to build a very robust solution.
Mule flows could be a very good choice to address this problem. It's a new feature of Mule 3 designed for orchestrating integrations like this.