Debugging a VB6 COM+ Web Application Hang - com

I work on a Classic ASP web application that uses several old COM components written in VB6. All of the VB6 components are registered in a COM+ application that run in their own dllhost process. A large majority of the application has been converted to .Net, but there are still a lot of legacy pages and components. COM Interop is used in both directions, calling some .Net assemblies from classic ASP and VB6 as well as calling VB6 components from ASP.Net. The application is running on Windows Server 2008 R2 (IIS 7.5) in the classic pipeline mode.
For the most part the application works fine. The transition to .Net effort was ultimately abandoned, with a new product being developed instead. In the meantime, the old product must be maintained in it's heterogeneous state.
I am having trouble tracking down an intermittent problem where the web application hangs. Users just see a blank screen while their browser waits and the server never responds. The hang persists until I manually kill the dllhost process that's hosting the VB6 components, so I believe the problem is buried there. Probably a memory leak or runaway circular loop.
There are thousands of users on the system daily, but the problem only happens once or twice a week. Fortunately we have a web farm that automatically pulls a server out when it stops responding, so the customer impact is zero. Still, I would like to figure out what's going on.
I have recompiled all of the VB6 components to include debugging symbols and redeployed to production. When the problem happens, I use the 32-bit task manager (c:\windows\syswow64\taskmgr.exe) to take a crash dump of the dllhost process. I end up with a dllhost.dmp file, which I bring down to my development workstation and open in VS2010. I have the .pdb symbol files that VB6 created in my symbols path. When I start the debugging session in VS2010, I can go to the Modules screen and see that indeed all of the symbols for my components are loaded.
Where to go from here? The call stack doesn't show any of my own components. It looks like this:
The disassembly at the top of the call stack looks like this:
Not sure what else I can do. I examined all of the locals at every frame of the call stack and it's gibberish to me. I don't see any references to any of my own components.
Perhaps WinDbg would yield more information? Not sure where to start with it.
I'm pretty sure that if I could just find what VB6 class/method was being called when the hang occurred that I could get to the bottom of it. I've tried adding some logging, and the results are inconsistent.
Perhaps there's nothing wrong with my VB6 components at all, but I'm hitting on some bug within Windows or IIS?
Any advice would be appreciated, but throwing away VB6 is not an option at this point. Thanks.

Not a complete answer, but CoRegisterSurrogateEx is documented to block as long as the surrogate process is running:
The CoRegisterSurrogateEx function is a blocking function. It does not
return until COM+ has determined that the process will be shut down.
Before calling this function, initialize COM on this thread as a
multi-threaded apartment (MTA).
So I don't think the error is on this call-stack. (You can see it was still blocking on a WaitForSingleObject call, most likely the mechanism it uses to block until the process is to be shut down).

Related

How can i see If an object is destroyed in a DLL without using Visual Studio

The main software that we built is built using VB6 (old school, I know). We're having some problem for viewing a report using ActiveReports. We did many Debug DLLs and were unable to find why the problem happen. Note that restarting the server solves the issue, which brings us to think that the problem is resource related.
Is there a way I can see how much memory is being used by a DLL? I would like to see if an object in that dll might not be released correctly.
I tried using the performance monitor, but this is not as precise as needed.
Is there anything i could use to monitor a Dll's usage?
We are just doing it the hard way and monitoring a server running multiple instances of the DLL.

VS6 COM App Crashing with Potential Memory Integrity Error

We're porting an old Visual Studio 6 application to Windows 7. The application heavily uses COM and the old RogueWave database interface. We're having a very random sever crash which looks like some sort of memory integrity problem. The crash occurs in the COM application. The application is started using COM, after starting up it tries to connect to the database. One of the following will occur:
1) Sometimes the COM application will have a sever crash
2) Sometimes it will work
3) Sometimes it will not be able to connect to the database and never will even with retries.
4) Sometimes when calling the RogueWave database creation, it does not return.
When starting the COM application several times in a row it will display any number of these symptoms.
At this point I have not been able to identify what is causing the instability.
The application will also crash when started normally, not using COM, but crashes less frequently.
Previously I saw a problem with identical symptoms in another area of this product. I was able to identify that the symptoms were tied to calls into a dll which were returning a CString that was created as a local variable. The dll is built using _AFXDLL preprocessor definition. After removing the calls to this dll this problem no longer occurs. This problem was occurring in many places.
At first I thought that this problem was caused because the dll was not built as an AFX EXTENSION DLL, but after reading through the documentation the _AFXDLL preprocessor definition will also build the dll as an AFX_EXTENSION_DLL. I'm used to using the _AFXEXT preprocessor directive to build AFX_EXTENSION dlls.
The COM application that is now crashing does not use this DLL. There is another DLL used by the COM application that has 1 exported function that returns a CString created as a local variable. This exported function is not called during the database connection.
Anyone have any ideas what would be causing this apparent memory integrity crash in a COM application? Thank you in advance for your help.
This turned out to be caused by a bug in the Oracle 10g express client dll.

Check memory consume of vb.net application

I am getting a System.OutOfMemoryException from my vb.net application, it happened from one certain user. I am trying to figure out whether my application takes too much memory space or the PC has low memory. But she just upgraded her desktop as i3 Intel CPU and higher memory.
Is there any tool that I check memory consumption when code is executed as go through lines?
By the way, it is developed by vs2010 and It is a Windows application.
<--Edit-->
I found the problem from my application. I did not mention because I did not know what the problem was. I am using a Google map inside of vb.net application. It takes a memory whenever I search a map on web browser which is on a form . Even though I close the form, it is not disappeared from
memory. It is just going up. That goes away when I close MDI form. FYI..
The best tool we've worked so far is the JetBrains profiler (http://www.jetbrains.com/profiler/features/).
That's a common error. It normally happens when the application enters a loop that never ends... (infinite loop).
Is it a Console, WinForms, ASP, WPF?
Try to identify during which Event your application throws the exception and put a try catch to print out more details.
Just check out this cool WPF performance suite to check out if you have memory leaks:
Performance Toolkit
Or check out if you can use GC.Collect() to find the leak.
Also you can accecss in Process Explorer to any .Net memory data and peaks.
Regards

IIS 5.0 Calling COM+ Objects from ASP scripts

Note: I couldn't decide whether this was more appropriate for Stack Overflow or Serverfault, so if you have some insight into that, let me know.
Background: Recently, my server (Windows 2000, MS SQL 2005, IIS 5.0, ASP Classic) experienced a spike in traffic to a specific set of ASP pages. This spike caused a massive drain on the processor, spiking it at 100% and causing all kinds of timeout problems for the visitors.
We've actually handled larger volumes in traffic than this without error. The problem seemed to be that the specific ASP scripts being called were using a huge amount of processor time. Using the Process Explorer from Sysinternals, I found that dllhost.exe was taking up all of the processor time. Looking at its threads, the culprit was calls to COMSVCS.DLL, which seems to be COM+ objects.
So, it seems like my ASP pages are calling COM+ objects and it's killing my processor.
Here's the question: How do I determine which parts of my ASP scripts are calling the COM+ objects, and how would I begin to improve performance from these parts? I have basically no background in Windows programming, so I am at a loss of how to begin.
Thanks for your help.
Neither COM+ or DLLHOST are likely your problem, they are just the containers that the web site and COM objects are running in. The actual objects they are being "fed" are your issue and/or the ways/frequency they are being called by the web app.
A more productive way to isolate the problem would be to look at the IIS logs for the pages with the longest processing time and have a programmer analyze what is going on in that page and what objects are being called.
Specifically, check the "time-taken" column in the IIS log.
For determining what objects are being called, look for
<OBJECT ID=MyObject RUNAT=SERVER PROGID=MyDll.MyObject></OBJECT>
or
set myObject = server.createobject("MyDll.MyObject")
inside your ASP pages.
Beware that this could be calling standard DLL (not COM+ objects). The method for instantiation is the same for both types.
If you want to know what COM+ processes are you running, check out the Component Services app.
alt text http://img38.imageshack.us/img38/5062/capturerm.png

What can cause Visual Studio 2008 to hang while debugging?

I have a vb.net project which sometimes, when running in the IDE, suddenly hangs. Normally this wouldn't be a problem. Just hit 'pause', look at the currently running threads, and find the deadlock (or whatever else).
But now I'm running into a situation where not only does the program hang, but trying to pause it causes visual studio itself to hang. In order to get control back, I have to kill the program-being-debugged's process, at which point visual studio comes back to life and says it was unable to pause execution. This is frustrating, because killing the process means the program state is lost (of course), so I don't know where the hang is.
So are there any common causes for this behavior? What should I be looking for?
if your program installs global hooks (which communicate with app) - this might be the case. A hook tries to communicate with your app (which is paused by debugger) and gets locked. And debugger is unable to receive its window messages: classic deadlock between hooked debugger (with hook dll) and a hooking app.
Finding a specific fix for a Visual Studio problem can be tricky: http://social.msdn.microsoft.com/Search/en-US/?query=visual%20studio%202008%20hang&ac=3
Additionally, not all hotfixes from Microsoft get released directly to the public. Some are only given out to customers whose systems are exhibiting a specific problem. So you have to contact Microsoft to the get the fix. They do this to limit the potential downside of releasing a hotfix that may break something else. So if all else fails, give them a call.
Here are some other things that I like to do when Visual Studio starts acting up:
Delete old breakpoints and watch variables.
With visual studio not running, delete the intellisense file (.ncb)
Clean the solution and then do a rebuild of all of the code.
Check hotfixes and service packs. I've seen a bug related to .net programming and debugging hangs. (VS hangs for me when debugging C++ 32-bit apps on 64-bit os:es sometimes.)
I just had a very similiar issue (VS fails to break execution), using Debug -> Delete all breakpoints solved the issue.
I'd check the code of the program being debugged, I'm thinking there may be an infinite loop or race condition in the code you're trying to debug. This has been the case for me in the past, especially on a single-core laptop I used to have. Can you give any information about where in the program you think execution is when you try to pause?