We are using custom loggers through Java code in OSB. It was working fine till before Production environment. In production under heavy load, all application logs are getting mixed up. I need to Configure Work Manager (WM) for each application. For an OSB project, I have configured for Proxy service one WM and for multiple Business Services One WM setting max Thread Constraint to 7 [Done multiple trial and error for Max Thread]. But still under heavy loads logs are getting mixed up. What values should I actually set for Work Manager properties?
Related
I'm migrating a service based integration platform from .Net Framework to .Net Core. The original versions of the integration platform have proven very successful and compared to replacing it with a 'off the shelf' integration solution, it has a far better ROI.
So after redeveloping the code, all tests has been working very well and have achieved higher levels of performance with a single IIS server that I could with 2 IIS servers with the original versions.
Except... If I go over ~3 message/sec with multiple clients, I start seeing duplicate GUID key errors when trying to save instrumentation data to my DB. All these errors are generated from the on-ramp service. The on-ramp places the message on a queue. The messages are then consumed by an off-ramp service and sent to the destination (for this load test the destination is a file folder).
Even though the off-ramp is also running on the same server as the on-ramp, we do not see any duplication errors generated by the off-ramp. I suspect this is due to the queue creating a linier process, so only one instance of the off-ramp is running at any time vs the on-ramp that has up to 4 clients firing concurrent messages at it's API.
Initially I thought the issue was caused by a static global variable class I had implemented, crossing process boundaries. But I would expect that the issue would be seen with the off-ramp as well, as the service architecture for both are virtually identical.
Summary of thoughts on issue:
If it is a pure coding issue, then errors would happen at low messaging rates.
The error would also be seen on the off-ramp if the GUID duplication was chance.
The on and off ramps are both running on the same server, but duplication only seen on the on ramp. IE on ramp not impacting the off ramp and visa versa.
Duplication has to be due to shared memory between concurrently running on-ramp instances, generated by multiple client scenario.
To try and resolve the issue I removed the static global variable class but I'm still seeing the duplication errors.
This issue was never observed in the original IIS implementation (after millions of message processed). I suspect the issue is with process isolation in the IIS hosted Kestrel .Net Core service host. From what I have read there is good isolation between different apps (based on IIS path) but not within the same app. So basically within the same IIS app pool. This could explain why .Net Core does not support multiple app running in the same IIS app pool.
If any one has a good idea how i can achieve process isolation between instances of the same app running in the same IIS app pool I would appreciate your thoughts/suggestions.
After running more tests I was able to resolve the issue. The problem was with the scope of the instrumentation variable. At low rates there was never a problem, but at high throughput, the same instrumentation object was being accessed by a second instance of the process.
The issue was difficult to track down due to the short lived nature of the integration services.
Thanks to anyone who reviewed the question.
Martin
I am evaluating Hangfire for an upcoming ASP.net Core project that has several scheduled and reoccurring tasks that need to execute independently of users clicking on web pages. I know that HangFire can do this if the web application is started because a request has come in. I need to know whether or not HangFire can execute a scheduled task between being rebooted and the first web request coming in.
Example: Web server is rebooted at 11pm, and no web requests will come in to cause the web server to spin up until 5am the next morning. A scheduled task needs to be performed at 1AM. Will Hangfire execute this task even though the web application hasn't been started by an incoming request?
If it can, is there a certain setup I need to do to allow this?
Details, if needed:
We are going to be using Kestrel hosted in a windows service and sitting behind an NGINX reverse proxy. This setup could be modified if needed to make HangFire meet this requirement.
When running under IIS it would be a real problem, see Making ASP.NET application always running
But it should not be problem for ASP.NET CORE with kestrel, see
It is not necessary for ASP.NET Core, because application is exposed
by a console application that it already always on – there are no
timeouts, no suspends and other optimization techniques yet. All you
need to do is to use supervisor as written in the official docs for
Linux, or use Windows Service with automatic start time, when running
on Windows.
I have a WebLogic cluster on which I've deployed numerous topics and applications that use them. My applications uniformly show themselves in a Warning status. Looking at Monitoring on the deployment, I see the MDB application connects to Server #1, but on server #2 it shows this:
MDB application appName is NOT connected to messaging system.
My JMS Server is targetted to a migratable target, which is in turn targetted to the #1 server and has a cluster identified. And messages sent to either server all flow as expected. I just don't know why these deployments show in a Warning state.
WebLogic 11g
This can be avoided by using the parameter below
<start-mdbs-with-application>false</start-mdbs-with-application>
In the weblogic-application.xml, Setting start-mdbs-with-application to false forces MDBs to defer starting until after the server instance opens its listen port, near the end of the server boot up process.
If you want to perform startup tasks after JMS and JDBC services are available, but before applications and modules have been activated, you can select the Run Before Application Deployments option in the Administration Console (or set the StartupClassMBean’s LoadBeforeAppActivation attribute to “true”).
If you want to perform startup tasks before JMS and JDBC services are available, you can select the Run Before Application Activations option in the Administration Console (or set the StartupClassMBean’s LoadBeforeAppDeployments attribute to “true”).
Refer :http://docs.oracle.com/cd/E13222_01/wls/docs81/ejb/message_beans.html
this is applicable for the versions till 12c and later
I don't like unanswered questions, so I'm going to answer this one.
The problem is resolved, though I was not involved in its resolution. At present the problem only exists for the length of time it takes the JMS subsystem to fully initialize. During that period (with many queues, it can take a while) the JNDI system throws errors and the apps are truly in warning state. Once the JMS is fully initialized, everything goes green.
My belief is that someone corrected something in the JMS Server / Cluster config. I'll never know what it was.
I have a WCF Service Deployed on IIS. (BasicHTTPBinding with [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)])
I have built custom in-memory session management and Now I am facing a strange problem that is IIS 7 Restarts Automatically without even throwing any kind of warning or error not even in EventLog. This problem leads to destroy the all available sessions.
I discovered this issue after logging the Application_Start and Application_End methods using log for net and also i put the break point in application_start and it paused there in between test execution.
This happens rarely but i need to know why it happens and if it is normal and acceptable or not. if not then what may be the possible reasons of this.
Regards
Mubashar Ahmad
Could it be the app pool being re-cycled? IIS 6 has this set on by default to 1740 minutes. As for IIS 7 I guess you would have the same kind of setting? I know in IIS 6 this "event" is not logged as 'n error.
IIS recycles worker processes either when it detects an "unhealthy" process, or after certain operator-configurable limits are reached.
Among the limits are:
memory threshold
after a configured number of requests
elapsed time
time of day
more info
The Session timeout (which is separate to the app pool recycling) is set to 90 minutes by default, this is set at the application level. This also means anything being held in Session will be blown away at that time. You can set it via the properties of the virtual directory/application in IIS6, and via SessionState->Open Feature in IIS7 (when you have the application selected).
Also note that session timeout can be set via the web.config of an ASP.Net application, should your web services be hosted in one of those.
When I start a Weblogic instance with a deployed application, the deployment is sometimes left in prepared state, not in active state. I have to go to Weblogic Console and start the deployment manually, which is quite slow and annoying repetetive work. Since this is done on a development machine — sometimes 50 times a day, — there are no security implication as the server is only visible on the local network. Is there some way to have it always start the deployment active?
Note that I'm not redeploying the application, I instead have it "constantly deployed" and stop/start the Weblogic instance using the scripts in bin directory.
If you are running weblogic in development mode, you can use the autodeploy folder for your app. See details here: http://download.oracle.com/docs/cd/E11035_01/wls100/deployment/autodeploy.html#wp1021620
Think this should solve your problem