I understand ESB has in-sequence, out sequence and fault sequence, Is in-sequence same as main sequence ?
Also I would like to know if we define receive sequence in a endpoint will it override default behaviour and response is handled by specified received sequence ? Will response handled by default sequence if received sequence is not specified explicitly ?
Most of the WSO2 ESB examples in internet are based on proxy service, I would like to understand how sequence can be used without proxy service, Can you please introduce me to a such tutorial ?
Triggering Messages
Messages come into the ESB through the following triggers:
A proxy service receives messages that have been sent to a specific endpoint.
A task injects a message into the ESB at a scheduled interval.
A REST API call sends a message to the ESB.
When the ESB receives a message, it sends it either to a proxy service or to the Main sequence for handling. The proxy or sequence is configured with message mediation, which controls how the message will be transformed, filtered, forwarded, etc.
To trigger messages in the ESB, see the following topics:
Creating APIs
Working with Proxy Services
Working with Tasks
Using REST
Source = https://docs.wso2.com/display/ESB481/Triggering+Messages
About the Main and Fault Sequences
A mediation configuration holds two special sequences named main and fault. All messages that are not destined for Proxy Services are sent through the main sequence. By default, the main sequence simply sends a message without mediation, so to add message mediation, you add mediators and/or named sequences in the main sequence.
By default, the fault sequence will log the message, the payload, and any error/exception encountered, and the drop mediator stops further processing. You should configure the fault sequence with the correct error handling instead of simply dropping messages. For more information, see Error Handling.
Source = https://docs.wso2.com/display/ESB481/Mediation+Sequences#MediationSequences-mainSequence
Also I would like to know if we define receive sequence in a endpoint will it override default behaviour and response is handled by specified received sequence ? Will response handled by default sequence if received sequence is not specified explicitly ?
Correct , in a proxy though , and the default behaviour is to send to "out sequence"
I am also a beginner at this , just know what I know through (a lot of) trial and error ;)
Ok so assuming you have a default offset of 0 , you're ESB instance should have 2 default transports/ports :
HTTP = 8280
HTTPS = 8243
So , any message sent to these ports will reach the main sequence , UNLESS , they are service specific endpoints, so for example , for a soap message with endpoint :
http://localhost:8280
this will reach the default main sequence , and the following
https://localhost:8243/services/yourProxyService ,
will got to the in sequence of the yourProxyService.
Also , in [esb_home]/repository/conf/log4j.properties , set/uncomment the following lines :
log4j.logger.httpclient.wire.header=DEBUG
log4j.logger.httpclient.wire.content=DEBUG
log4j.category.org.apache.synapse=DEBUG ,
this should show you much more info in the log and make it easier to know which sequence you are in..
feel free to ask me other stuff , cool
Related
I'm using Masstransit with RabbitMq to communicate between two web applications .net core.
In the publisher application I publish a message to an exchange 'producer'.
rabbitConfigurator.Message<TMessage>(x => x.SetEntityName("producer"));
In the same application I have a consumer-A of said messages and that works fine (the consumer has an exchange and a queue as recommended by rabbit/masstransit connected to the producer exchange)
rabbitConfigurator.ReceiveEndpoint("consumer-A", x =>
{
x.Consumer<TConsumer>(context);
x.Bind("producer");
});
In a second application I'm trying to setup another consumer-B of the same message type connected to the exchange 'producer'.
rabbitConfigurator.ReceiveEndpoint("consumer-B", x =>
{
x.Consumer<TConsumer>(context);
x.Bind("producer");
});
However messages destined for this second application are getting sent to a skipped queue. From what I can see this second consumer is correctly configured in RabbitMq though.
I can't see what I'm missing.
Edit: In the producing application I can also successfully receive messages in a second consumer. But from another web application it doesn't work - perhaps something to do with the fact that there's a differetn connection/channel ?
Messages sent to the producer exchange should be in a format that can be deserialized by MassTransit. If they aren't, you'll need to add a message deserializer which can support the message type. If it is application/json then you'll need to add the RawJsonMessageDeserializer to the receive endpoint.
If the messages are in the correct format, verify that you have a consumer on the receive endpoint that can consume that message type. Message types must match entirely, including namespace, to be consumed. More details are available in the documentation.
I have an NServiceBus endpoint that handles saving documents to a document management system. After the document is saved, I call Bus.Reply(new DocumentSaved{}).
This works fine when I am sending SaveDocument from a Saga (which cares deeply about the reply), but it fails when I am sending it from my web client endpoint (i.e. an MVC project, which doesn't care at all about the reply). The failure is because my web client endpoint doesn't have a queue to process the reply.
What am I doing wrong here? (I really don't want to have to create a queue for my MVC project to hold a bunch of replies that will never ever get processed.)
Replies are just normal messages. The only thing that links original messages and relies is correlation id, which is stored in the message header and the originator address, where a reply is sent to.
This means that all rules that apply to normal messages are also applicable to replies. There are no special "reply queues". Replies go to normal queues as any other message.
I suspect that you have no message-endpoint mapping configuration in your web endpoint. I am not sure if SendOnly endpoint has any effect here, since I assume you already received a message there, which you want to send a reply to.
I would start by checking the message assembly to endpoint mapping and enabling debug level logging.
I need to implement a logic on Retry. Inbound endpoint pushes the messages to Rest (Outbound). If the REST is unavailable, I need to retry for 1 time and put it in the queue. But the second upcoming messages should not do any retry, it has to directly put the messages in to queue until the REST service is available.
Once the service is available, I need to pushes all the messages from QUEUE to REST Service (in ordering) via batch job.
Questions:
How do I know the service is unavailable for my second message? If I use until Successful, for every message it do retry and put in queue. Plm is 2nd message shouldn't do retry.
For batch, I thought of using poll, but how to tell to poll, when the service becomes available to begin the batch process. (bcz,Poll is more of with configuring timings to run batch)?
Other ticky confuses me is - Here ordering has to be preserved. once the service is available. Queue messages ( i,e Batch) has to move first to REST Services then with real time. I doubt whether Is it applicable.
It will be very helpful for the quick response to implement the logic.
Using Mule: 3.5.1
I could try something like below: using flow controls
process a message; if exception or bad response code, set a variable/property like serviceAvailable=false.
subsequent message processing will first check the property serviceAvailable to process the messages. if property is false, en-queue the messages to a DB table with status=new/unprocessed
create a flow/scheduler to process the messages from DB sequentially, but it will not check the property serviceAvailable and call the rest service.
If service throws exception it will not store the messages in db again but if processes successfully change the property serviceAvailable=true and de-queue the messages or change the status. Add another property and set it to true if there are more messages in db table like moreDBMsg=true.
New messages should not be processed/consumed until moreDBMsg=false
once moreDBMsg=false and serviceAvailable=true start processing the messages from queue.
For the timeout I would still look at the response code and catch time-outs to determine if the call was successful or requires a retry. Practically you normally do multi threading anyway, so you have multiple calls in parallel anyway. Or simply one call starts before the other ends.
That is just quite normal.
But you can simply retry calls in a queue that time out. And after x amounts of time-outs you "skip" or defer the retry.
But all of this has been done using actual Mule flow components like either:
MEL http://www.mulesoft.org/documentation/display/current/Mule+Expression+Language+Reference
Or flow controls: http://www.mulesoft.org/documentation/display/current/Choice+Flow+Control+Reference
Or for example you reference a Spring Bean and do it in native Java code.
One possibility for the queue would be to persist it in a database. Mule has database connector that has a "poll" feature, see: http://www.mulesoft.org/documentation/display/current/JDBC+Transport+Reference#JDBCTransportReference-PollingTransport
I have been trying to figure out, what exactly a inbound end point and outbound end point. For me it is bit of an eluding to understand.
what are exactly inbound and outbound endpoints for/do in mule flow? if a flow wants send message which endpoint shoud be used viz when receiving. Or when an application want to invoke a flow which end point it should communicate to?
Inbound endpoints are message sources (http://www.mulesoft.org/documentation/display/current/Message+Sources), which as the name suggests is where messages are created. They can be created based on external events (like an incoming HTTP request or JMS message) or by polling (like files in a directory).
Outbound endpoints and anything else you see in a flow (except exception strategies) are message processors (http://www.mulesoft.org/documentation/display/current/Message+Processors), which means they do something with the message in-flight the flow. Outbound endpoints are message processors that send the current message to "destinations" like a JMS queue, an HTTP server, a file, ...
Disclaimer: This is a simplified view to give you a general idea, its not the beginning or end of what you can do with mule (or other service buses)
Mule is a message processing engine. You can think of it as a giant conveyer belt. You put something on one end, and it goes along the belt and comes out the other end.
The thing that mule deals with are called messages.
The starting point is the "inbound end point" and the exit point is the "outbound end point"; between these pairs of end points you can have other things that will process the message as it travels from the start to the end.
A combination of a start end point + gubbins in the middle to do some work on the message + outbound end point is called a flow. You can chain flows together to create a workflow or process.
These processes are then packaged as an application and uploaded to the mule server. The process only runs when a message that it is listening for arrives. Otherwise the processes are sitting idle. Think of it like a car assembly line. The assembly line that fixes the seats in the car can only start when the chassis is finished; otherwise there is nothing for it to do. Once the seats are fixed, only then can the paint assembly line start, and so on.
Inbound Endpoint: To get the data in mule app from outside.
Outbound Endpoint: To pass the data from the mule app to the outside.
There are two types of connectors available based on operation.
Inbound Endpoint based connector : Inbound endpoint based connectors are the connectors which is placed inside a message
source area of a mule flow which is used to start executing or
triggering off the flow upon receiving the request. (or which is
used to receive the incoming request and pass the request to the
next message processors in a flow for further processing)
Outbound Endpoint or Process based connector: Outbound Endpoint based connectors are placed anywhere within a message processor
area of a flow at beginning or in the middle or at the end.
Regards,
Rajnish
Giving below the structure definition of Mule Message Object has Message Inbound Property Outbound Property Payload Variable Flow Variable Session Variable Attachment Exception Payload
When a connector of a flow (listening on a port) receives the payload its called Inbound endpoint. When in a flow we have a connectore placed in the middle and send a payload its called Oubound endpoint. Here the all the outbound properties sent to the Http Outbound flow become Inbound Properties within that flow.
For detailed explanation see the link below.
https://docs.mulesoft.com/mule-user-guide/v/3.8/mule-message-structure.
Background:
I'm troubleshooting a problem where messages sent by WCF over transactional MSMQ (with netMsmqBinding) seem to disappear. The code that uses WCF is in a third-party assembly which I cannot change. I have few clues to what the problem is, but plan to enable various tracing capabilities in order to pin-point where the problem relies.
Context:
I have enabled MSMQ End-to-End Tracing. It logs two events for every message that gets sent.
One event when a message is written to the outgoing queue. This message contain the MSMQ message id (which is composed by a guid and an integer, ie 7B476ADF-DEFD-49F2-AF5A-0CF27C5152C0\6481271).
Another event when that message is sent across the network.
I have enabled verbose WCF Tracing.
I also have application level logging that logs a message IDs defined by the application code (let's call this the "application message id").
I have enabled positive and negative source journaling on the MSMQ messages that get sent.
I have enabled journaling on the receiving queue.
Problem:
When messages go missing, I know the missing message's application id (it's logged by the sending side). I would now like to look at the End-to-End trace to see
whether the message was written to the outgoing queue or not.
How can I correlate the events in the End-to-End trace with the application level logs and WCF traces?
Ideas:
When sending a MSMQ message using the managed MSMQ API in System.Messaging, the message's MSMQ id is available after the message is sent. However, I have not found a way to log this when WCF is performing the send operation. The WCF trace logs a MSMQMessageId guid, but this value is, surprisingly, not the actual MSMQ id as I guessed it would be. Is it possible to access the actual MSMQ message id and log it?
Log the native thread id in the application log along with the application level id and a time stamp. The native thread id is logged to the End-to-End trace by MSMQ, so this might actually be sufficient to correlate. This is plan B for me if I don't find a more elegant solution.
You sounds like you're on the right track. However you could bump up a bit with this:
Using SvcConfigEditor.exe
Configure WCF Verbose Tracing for Propagate ACtiveity and Activity tracing
Configure WCF MessageLogging for "Malformed Messages, Service Messages, Transport Messages"
Use LogEntireMessage
In End to End, trace it All
Make sure you enable these *.config on BOTH sides, yours and the 3rd party executable.
Collect your logs files, and add them ALL to SvcTraceViewer.exe
You can configure windows MSMQ to sense subjects of messages and if subjects contains a key word fire an application. This application can logs incoming messages. In sender side you can write actual message id into subject of message and add your key word to subject. In receiver side fired application can access to actual message id near added key word in subject.
It looks like your message is being discarded by WCF because it is malformed in some way (i.e. contract mismatch, one of the WCF message size limits exceeded).
To trap this error you could write an ErrorHanlder that audits these errors.
Here a link to a sample of doing that.
Another option ,if you are using Win 2008 R2 and up, is to use the built in poison message handling. here`s a link to the the docs.
To the question, to trace end to end with an application trace identifier:
I would pass the application trace id in the message header (look here for an example).
To audit the message header on the service side i would use WCF's IOperationInvoker to intercept each call, and audit the id in the messaged header.
This can be configured in the config file for the process without altering the third party code.here`s an example of how to implement an invoker and how to set it in config.