I am working on an application in Silverlight 5. We use WCF for all of our network communication, and it mostly works well. However, we have a couple of Virtual Machines that we use for testing where the app fails, tries to restart itself, fails, etc. in an endless loop. I have added a lot of tracing code, and a lot of try catches, and I have it isolated all the way down to the line of code that is failing, but I still can't get an actual error message from the failure, just the crash. Originally, it was failing on this line of WCF code:
return await Task<List<Instance>.Factory.FromAsync(Channel.BeginGetInstance, Channel.EndGetInstance, null);
In case it had something to do with the use of async/await, I went back to our old code with callbacks. I still get the same failure, but now I can see the call to the WCF function completes successfully, but the log statement on the first line of the callback never happens, so it seems like its dying before or outside of the callback.
One other note, it appears the code we have in Application_UnhandledException is not firing, but the code in Application_Exit does run, I see that as the last line in the log file.
I tried to setup remote debugging, but I am unable to connect to the app before it crashes and recycles, so that didn't help either.
I also used TCPView to watch the network traffic, and it looks like communication is happening in both directions.
If anyone has any suggestions of anything else to try, I would greatly appreciate it.
I spent 10 days chasing my tail on this before finally realizing my problem. There was a bug in the error logging code. It was generating an error, I was just not seeing it. Once I realized that and got the error message, the actual underlying bug was fixed in about 5 minutes. Good lesson though, never assume the underlying code is working, no matter how simple it is.
Related
In our webapi code we use SignalR and redis backplane. I'm see the problem where our code hangs after sometime.
StackExchange.Redis.StrongName.dll!StackExchange.Redis.SocketManager.WriteAllQueues() Line 288
Actually the code for caching the data works (I can see data getting populated in redis server) but after couple web request/response the code hangs. I've installed the latest package 'StackExchange.Redis 1.1.608'. Unfortunately in VS I don't see my code in stack when I hit break-all.
Any ideas what might be wrong or where to look for the problem. I wish I could put more details here but this is all I got. Thanks!
here is the snapshot of threads when I hit break-all in VS
[Threads]:
Those are threads created by the StackExchange.Redis library to read and write to the socket connected to Redis.
It is normal for the writer thread to be be blocked inside WriteAllQueues() because it is calling Monitor.Wait(). The thread will awaken when there is work to do.
The read thread probably blocks on reading from the socket, but I can't find in the source code where this happens.
I'm working on a console app developed by a guy who doesn't work here any longer. While debugging, the ContextSwitchDeadlock exception was thrown (I found this question on the exception). If I ignore it, the app will eventually work through the loop it occurs in. The app runs as a scheduled task every day, but this particular process is not called every single time.
I'm wondering if it is OK to allow this exception to go to production. The author of this app put it in production with this exception, and its been running ever since. Should I just make my (unrelated to this exception) updates and leave the app as is? Or should I try to address the issue? Addressing it seems daunting to me :/
Ben. I would say 'NO'. Unless your exception is a ThreadAbortException (i.e. the user closed a window and so the process is dead) or some such thing, an exception like this could open your code up to cascading failures. Based on what we do where I work:
I think, as a band aid, you should encapsulate the offending code with a Try-Catch, and wire it up to send you an email every time it Catches so you have documentation on what's going on AND so that you prevent cascading failures from propagating throughout your code (quarantine the problem).
Towards a fix (when you have time), debug it and step through to figure out why your main thread is taking so long, and if you can, create a worker thread to handle that (DISCLAIMER: this would be my opening attack angle at this problem, based on the answer from the link you provided. I have NOT tested this, nor do I have experience enough to definitively say this will work).
EDIT: After running into this error for a particularly long running process myself, I came across this slew of answers on msdn:
http://social.msdn.microsoft.com/Forums/en/vsto/thread/bf71a6a8-2a6a-4c0a-ab7b-effb09451a89
While I resolved my error (I was reading a System.IO.FileStream into a String Builder instead of using a String and the StreamReader ReadToEnd method), I think it might be helpful to you.
Error code:"
The request channel timed out while waiting for a reply after 00:09:59.6320000. Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding."
This error occurs infrequently when calling a Wcf service methods. It doesn't matter what method is. I have created test methods that returns simple strings. Sometimes it times out, sometimes it works perfectly. The strange thing is that when the WCF service is published on one server(for testing purposes)- there is no timeout. When I publish it on another server(live/public) there occurs these timeouts infrequently. I have set the timeout to 10 min as you could see above.
The webconfig setting should be correct, because it works for the one server. The only change made is the ip address. I know this is very difficult to answer and a bit ambiguous.
I'm sure this problem is too high level for me to solve, or maybe I'm making a simple mistake and it is too obvious for me to notice. If you could give me a pointer or just friendly advice on this problem I would really really appreciate it. I am shooting in the dark here. I thank you for your interest, proved by you reading up to here.
does it happen first time you call the service? if not, but does subsequently, it could be that the service instance has been locked by the calling thread - look into multiple instances or allowing concurrent use, obviously taking into account the thread safety requirements of your code
Does anyone know how one can automatically reply to messages with status MSGW that block a job on an IBM i-series (AS/400)?
I'm using the jt400/jtopen library to access a program on an AS/400 from Java. I'm using the com.ibm.as400.access.ProgramCall class, which works fine, unless the program fails for some reason. As with almost any program, failures will happen sometimes, but unfortunately, in this case, it does not result in a status message or an exception. Instead, the calling thread just hangs. What's worse, any call to the AS/400 to get information on the Job (another class in jt400 that mostly does what you would expect) backing the queue will hang as well.
I could of course monitor the thread in which the call runs and simply kill it after waiting for a while, but that's a last resort. Getting an error message back from the system would be nice.
You could try execute this command before invoke your pcml with com.ibm.as400.access.CommandCall.run() method:
CHGJOB INQMSGRPY(*DFT)
It sets 'C' as default answer for all messages.
but you should ensure you have log of the messages in order to know the problem which generates this message
Regards,
I don't believe Java can directly trap errors that occur on the other side of that API. What I've done is to 'harden' the RPG (IBM i side) program so that it monitors for errors rather than let the default error handler get them. When an error occurs, the RPG program gracefully terminates and passes back an error code or even the entire message back to the Java application.
I've found that you can use the timeout mechanism of ExecutorService to interrupt a ProgramCall in MSGW.
You must discard the AS400 object afterwards, and the server job is still in MSGW, but at least you can continue on the Javaside.
(You need to use a separate AS400 object if you want to investigate on the hanging job.)
I'm having a problem where the WCF service hangs after 13-14 asynchronous process calls from the client. This occurs all the time. The client is a mobile JavaFX app. There is no specific error outputted in the server as well as in client. Someone suggested that it might be a throttling issue.
I've set the service side .config parameters maxConcurrent calls from 10 to 500
<serviceThrottling maxConcurrentCalls="500" maxConcurrentSessions="500” />
So this means, it should be able to accept more than 10 calls, right? However, it didn't resolve this issue. Still hangs on the 13-14th process call.
Only one client is connecting to this web service.
What do you think is wrong?
Do you close the client after doing your call?
When I encountered this problem, I did not close it, and the open requests blocked the service after a short time.
Edit: Ok, I know nothing about JavaFX =) The code below is C#, sorry. But you can surely do something similar.
Use either
WcfClient client = new WcfClient()
// ...
client.Close()
or
using(WcfClient client = new WcfClient()){
// ...
}
Similar problem here - I have an app calling from one process to another, locally, named pipes.
Calls are really light in code- basically takex an array of serializable objects, queues them on other side. Occasionally it hangs. Restarts afte rtimeout. no data lost, but... as the data is financial data, and the receiving app an autoamted trading system, that may result in very bad financial issues. Not been able to reproduce it yet.
This could very easily be caused by any deadlock condition in your code. If your service locks up and starts eating up 100% or CPU you have a dead lock. Create a dump file and see where your code was at.
I ran into the same issue my first WCF app it was a dictionary that i wasn't making sure was synchronized in logging code.
The SvcTraceViewer is super helpful in figuring out tough wcf