Consumer Proxy unable to pick up messages from queue due to service configuration in flux - weblogic

The Consumer proxy is not picking up messages from queue. We have redeployed service and restarted servers. But it did not help. I am attaching logs in here.
<01-Mar-2019 10:39:53 o'clock GMT>
<01-Mar-2019 10:39:53 o'clock GMT>

According to Oracle support document 1573359.1:
CAUSE
The service has been re-deployed/changed while there were messaging being processed. Review Doc ID 1571958.1 "OSB SBConsole Activation - Limitations for configuration or deployment changes in production" for other reasons that this error can occur.
SOLUTION
Stop consumption on the jms queue, delete and re-deploy service.
Log in to Weblogic Console
Expand services -> Messaging -> JMS Modules -> Select the Queue your service is interacting with.
Select the Control tab
For both production and consumption, select pause.
Wait a short while (5 minutes) and restart the queue
Re-deploy your Proxy Services
If message still persist please check config.xml and make sure that there is a correct number of applications with name starting with "ALSB". The correct number depends on the kind of services you have deployed. JMS request-response, JMS plain request, JMS topic etc...
The easiest way to make sure that config.xml is correct is to do the following:
Delete all the JMS proxies from OSB configuration
Open WLS console go to "Deployments" and make sure that there are no application "_ALSB_xyz" deployed. If they are present delete them.
Re-deploy JMS proxies
Alternately, check Note 1382976.1 to locate the related deployments. Delete any application deployments starting with "ALSB" which are not related to any actively deployed JMS proxy service.

Related

How do I find the connection information of a RabbitMQ server that is bound to a SCDF stream deployed on Tanzu (Pivotal/PCF) environment?

This is a follow-up question of How to implement HTTP request/reply when the response comes from a rabbitMQ reply queue using Spring Integration DSL?.
We were able to build the Spring Integration application and the SCDF stream successfully locally. We could send a http request to the rabbitMQ request queue which was bound to the SCDF stream rabbit source. We could also receive the response back from the rabbitMQ response queue which was bound to the SCDF stream rabbit sink.
We have deployed the SCDF stream into PCF environment which had a binding of an internal rabbitMQ broker. Now we need to specify the spring rabbitMQ connection information in the Spring Integration application properties - currently it's using the default localhost#5762, which is no longer valid. Does anyone know how to get this rabbitMQ configuration properties? We already checked the SCDF stream rabbit source/sink log files but couldn't find the information. I know we probably need to check internally whoever set up the SCDF/rabbitMQ in PCF environment, but so far we haven't heard the answers from them.
Also, it appears we can have a different approach that binds both the SCDF stream and the integration application to a separate rabbitMQ instance (instead of using the existing one bundled with the SCDF configuration). Is it a recommended solution?
Thanks,
It is unclear whether you're using the SCDF tile or the SCDF OSS (via manfest.yml) on PCF.
Suppose you're using the OSS, AFA. In that case, you are providing the right RMQ service-instance configuration (that you pre-created) in the manifest.yml, then SCDF would automatically propagate that RMQ service instance and bind it to the apps it is deploying to your ORG/Space. You don't need to muck around with connection credentials manually.
On the other hand, if you are using the SCDF Tile, the SCDF service broker will auto-create the RMQ SI and automatically bind it to the apps it deploys.
In summary, there's no reason to manually pass the connection credentials or pack them as application properties inside your apps. You can automate all this provided you're configuring all this correctly.

Weblogic migratable JMS consumer doesn't follow the service to the new managed server if the old server remains running

I have a JMS service targeted at a migratable target (using an Auto-Migrate Exactly-Once policy) in a cluster which consists of 2 managed servers, at any point of time the service is hosted at one of them and the consumer (which is targeted at the cluster) is supposed to receive messages seamlessly no matter where the service is hosted.
When I manually switch the host of the migratable target (clicking migrate), without turning the hosting managed server off, the consumer fails to receive messages sent to the queues, unless I turn off the previous hosting managed server forcing the consumer to the new host.
I can rule out sender problems, I can see the messages in the queue right after them being sent.
I'll be grateful if anyone can advice on how to configure either the consumer or the migratable service to work seamlessly when migration happens.
I think that may just be a misunderstanding of how migration works. The docs state Auto-Migrate Exactly-Once:
indicates that if at least one Managed Server in the candidate list
is running, then the JMS service will be active somewhere in the
cluster if servers should fail or are shut down (either gracefully or
forcibly). For example, a migratable target hosting a path service
should use this option so if its hosting server fails or is shut down,
the path service will automatically migrate to another server and so
will always be active in the cluster. Note that this value can lead to
target grouping. For example, if you have five exactly-once migratable
targets and only one server member is started, then all five
migratable targets will be activated on that server member.
The docs also state:
Manual Service Migration—the manual migration of pinned JTA and
JMS-related services (for example, JMS server, SAF agent, path
service, and custom store) after the host server instance fails
Your server/service has neither failed or shut down, you are forcing it to migrate with a healthy host still running, so it has not met the criteria for migration.
See more here as well.
I have some experience that sounds reminiscent of what you're looking at. There was some WLS-specific capability around recognizing reconfiguration in JMS destinations as part of their clustered server design.
In one case I had to call a WLS-specific method: weblogic.jms.extensions.WLSession.setExceptionListener(). This was on their implementation of the JMS Session interface. This is analogous to the standard JMS Connection.setExceptionListener().
With this WLS-specific capability, the WLSession.setExceptionListener() callback would occur at a point where the consuming client should tear down and re-establish the connection / session / consumer in reaction to a reconfiguration (migration) that had happened.

JMS queue not synchronized in instances of GlassFish cluster

I have a problem using Message Driven Beans in CLUSTERED Glassfish 3.1.1. The problem is with the queue in the Glassfish, the queue is not synchronized between the instances. I am trying by best to explain the scenario below.
I created 2 instances in a GlassFish cluster, created a JMS QueueConnectionFactory, created a JMS Queue. Their targets were made towards the cluster. Then I deployed the web application and the MessageDrivenBean module in the cluster. The web application sends a TextMessage to the JMS Queue.
Everything works well here, like the message is sent to the queue and served by the message driven beans in both the instances.
Then I disable the MessageDrivenBean module. Request the web application which sends the message to the JMS Queue in both the instances. Then I shutdown myInstance2. Re-deploy the MDB in the cluster. Now here is the problem, the MessageDrivenBean only receives the messages of myInstance1 and not the messages sent to myInstance2 queue. The messages in the queue of myInstance2 are only served when myInstance2 is started. Can anyone help me here with the settings that GlassFish uses to synchronize the queue in both the instance so that even for some reason when one instance is down and there are messages in that instance’s queue, the other instance will take the messages of that queue and serve them.
I am using OpenMQ, GlassFish 3.1.1 and I have turned on the HA(high availability) option in GlassFish, but still it does not work.
Thanks
The high-availability options for GlassFish and the high-availability options for Message Queue are configured separately. You need to configure your message queue cluster to be an "enhanced cluster" rather than a "conventional cluster". This is described in the GlassFish 3.1 High Availability Administration Guide.

Can't read from remote transactional private queue using WCF in workgroup mode (can do using System.Messaging !)

I have spent days reading MSDN, forums and article about this, and cannot find a solution to my problem.
As a PoC, I need to consume a queue from more than one machine since I need fault tolerance on the consumers side. Performance is not an issue since less than 100 messages a day should by exchanged.
I have coded two trivial console application , one as client, the other one as server. Using Framework 4.0 (tested also on 3.5). Messages are using transactions.
Everything runs fines on a single machine (Windows 7), even when running multiple consumers application instance.
Now I have a 2012 and a 2008 R2 virtual test servers running in the same domain (but don't want to use AD integration anyway). I am using IP address or "." in endpoint address attribute to prevent from DNS / AD resolution side effects.
Everything works fine IF the the queue is hosted by the consumer and the producer is submitting messages on the remote private queue. This is also true if I exchange the consumer / producer role of the 2012 and 2008 server.
But I have NEVER been able to make this run, using WCF, when the consumer is reading from remote queue and the producer is submitting messages localy. Submition never fails, my problem is on the consumer side.
My wish is to make this run using netMsmqBinding, but I also tried using msmqIntegrationBinding. For each test, I adapted code and configuration, then confirmed this was running ok when the consumer was consuming from the local queue.
The last test I have done is using WCF (msmqIntegrationBinding) only on the producer (local queue) and System.Messaging.MessageQueue on the consumer (remote queue) : It works fine ! => My goal is to make the same using WCF and netMsmqBinding on both sides.
In my point of view, I have proved this problem is a WCF issue, not an MSMQ one. This has nothing to do with security, authentication, firewall, transport, protocol, MSMQ version etc.
Errors info using MS Service Trace Viewer :
Using msmqIntegrationBinding when receiving the message (openning queue was ok) : An error occurred while receiving a message from the queue: The transaction specified cannot be imported. (-1072824242, 0xc00e004e). Ensure that MSMQ is installed and running. Make sure the queue is available to receive from.
Using netMsmqBinding, on opening the queue : An error occurred when converting the '172.22.1.9\private$\Test' queue path name to the format name: The queue path name specified is invalid. (-1072824300, 0xc00e0014). All operations on the queued channel failed. Ensure that the queue address is valid. MSMQ must be installed with Active Directory integration enabled and access to it is available.
If someone can help to find why my configuration cannot be handled by WCF, a much elegant and configurable way than Messaging, I would greatly appreciate !
Thank you.
You may need to post you consumer code and config to give more of an idea but it could be the construction of the queue name - e.g.
FormatName:DIRECT=TCP:192.168.0.2\SomeQueue
There are several different ways to connect to a queue and it changes when you are remote or local as well.
I have found this article in the past to help:
http://blogs.msdn.com/b/johnbreakwell/archive/2009/02/26/difference-between-path-name-and-format-name-when-accessing-msmq-queues.aspx
Also, MessageQueue Constructor on MSDN...
http://msdn.microsoft.com/en-us/library/ch1d814t.aspx

Why are my WebLogic clustered MDB app deployments in warning state?

I have a WebLogic cluster on which I've deployed numerous topics and applications that use them. My applications uniformly show themselves in a Warning status. Looking at Monitoring on the deployment, I see the MDB application connects to Server #1, but on server #2 it shows this:
MDB application appName is NOT connected to messaging system.
My JMS Server is targetted to a migratable target, which is in turn targetted to the #1 server and has a cluster identified. And messages sent to either server all flow as expected. I just don't know why these deployments show in a Warning state.
WebLogic 11g
This can be avoided by using the parameter below
<start-mdbs-with-application>false</start-mdbs-with-application>
In the weblogic-application.xml, Setting start-mdbs-with-application to false forces MDBs to defer starting until after the server instance opens its listen port, near the end of the server boot up process.
If you want to perform startup tasks after JMS and JDBC services are available, but before applications and modules have been activated, you can select the Run Before Application Deployments option in the Administration Console (or set the StartupClassMBean’s LoadBeforeAppActivation attribute to “true”).
If you want to perform startup tasks before JMS and JDBC services are available, you can select the Run Before Application Activations option in the Administration Console (or set the StartupClassMBean’s LoadBeforeAppDeployments attribute to “true”).
Refer :http://docs.oracle.com/cd/E13222_01/wls/docs81/ejb/message_beans.html
this is applicable for the versions till 12c and later
I don't like unanswered questions, so I'm going to answer this one.
The problem is resolved, though I was not involved in its resolution. At present the problem only exists for the length of time it takes the JMS subsystem to fully initialize. During that period (with many queues, it can take a while) the JNDI system throws errors and the apps are truly in warning state. Once the JMS is fully initialized, everything goes green.
My belief is that someone corrected something in the JMS Server / Cluster config. I'll never know what it was.