VS6 COM App Crashing with Potential Memory Integrity Error - dll

We're porting an old Visual Studio 6 application to Windows 7. The application heavily uses COM and the old RogueWave database interface. We're having a very random sever crash which looks like some sort of memory integrity problem. The crash occurs in the COM application. The application is started using COM, after starting up it tries to connect to the database. One of the following will occur:
1) Sometimes the COM application will have a sever crash
2) Sometimes it will work
3) Sometimes it will not be able to connect to the database and never will even with retries.
4) Sometimes when calling the RogueWave database creation, it does not return.
When starting the COM application several times in a row it will display any number of these symptoms.
At this point I have not been able to identify what is causing the instability.
The application will also crash when started normally, not using COM, but crashes less frequently.
Previously I saw a problem with identical symptoms in another area of this product. I was able to identify that the symptoms were tied to calls into a dll which were returning a CString that was created as a local variable. The dll is built using _AFXDLL preprocessor definition. After removing the calls to this dll this problem no longer occurs. This problem was occurring in many places.
At first I thought that this problem was caused because the dll was not built as an AFX EXTENSION DLL, but after reading through the documentation the _AFXDLL preprocessor definition will also build the dll as an AFX_EXTENSION_DLL. I'm used to using the _AFXEXT preprocessor directive to build AFX_EXTENSION dlls.
The COM application that is now crashing does not use this DLL. There is another DLL used by the COM application that has 1 exported function that returns a CString created as a local variable. This exported function is not called during the database connection.
Anyone have any ideas what would be causing this apparent memory integrity crash in a COM application? Thank you in advance for your help.

This turned out to be caused by a bug in the Oracle 10g express client dll.

Related

How can i see If an object is destroyed in a DLL without using Visual Studio

The main software that we built is built using VB6 (old school, I know). We're having some problem for viewing a report using ActiveReports. We did many Debug DLLs and were unable to find why the problem happen. Note that restarting the server solves the issue, which brings us to think that the problem is resource related.
Is there a way I can see how much memory is being used by a DLL? I would like to see if an object in that dll might not be released correctly.
I tried using the performance monitor, but this is not as precise as needed.
Is there anything i could use to monitor a Dll's usage?
We are just doing it the hard way and monitoring a server running multiple instances of the DLL.

Debugging a VB6 COM+ Web Application Hang

I work on a Classic ASP web application that uses several old COM components written in VB6. All of the VB6 components are registered in a COM+ application that run in their own dllhost process. A large majority of the application has been converted to .Net, but there are still a lot of legacy pages and components. COM Interop is used in both directions, calling some .Net assemblies from classic ASP and VB6 as well as calling VB6 components from ASP.Net. The application is running on Windows Server 2008 R2 (IIS 7.5) in the classic pipeline mode.
For the most part the application works fine. The transition to .Net effort was ultimately abandoned, with a new product being developed instead. In the meantime, the old product must be maintained in it's heterogeneous state.
I am having trouble tracking down an intermittent problem where the web application hangs. Users just see a blank screen while their browser waits and the server never responds. The hang persists until I manually kill the dllhost process that's hosting the VB6 components, so I believe the problem is buried there. Probably a memory leak or runaway circular loop.
There are thousands of users on the system daily, but the problem only happens once or twice a week. Fortunately we have a web farm that automatically pulls a server out when it stops responding, so the customer impact is zero. Still, I would like to figure out what's going on.
I have recompiled all of the VB6 components to include debugging symbols and redeployed to production. When the problem happens, I use the 32-bit task manager (c:\windows\syswow64\taskmgr.exe) to take a crash dump of the dllhost process. I end up with a dllhost.dmp file, which I bring down to my development workstation and open in VS2010. I have the .pdb symbol files that VB6 created in my symbols path. When I start the debugging session in VS2010, I can go to the Modules screen and see that indeed all of the symbols for my components are loaded.
Where to go from here? The call stack doesn't show any of my own components. It looks like this:
The disassembly at the top of the call stack looks like this:
Not sure what else I can do. I examined all of the locals at every frame of the call stack and it's gibberish to me. I don't see any references to any of my own components.
Perhaps WinDbg would yield more information? Not sure where to start with it.
I'm pretty sure that if I could just find what VB6 class/method was being called when the hang occurred that I could get to the bottom of it. I've tried adding some logging, and the results are inconsistent.
Perhaps there's nothing wrong with my VB6 components at all, but I'm hitting on some bug within Windows or IIS?
Any advice would be appreciated, but throwing away VB6 is not an option at this point. Thanks.
Not a complete answer, but CoRegisterSurrogateEx is documented to block as long as the surrogate process is running:
The CoRegisterSurrogateEx function is a blocking function. It does not
return until COM+ has determined that the process will be shut down.
Before calling this function, initialize COM on this thread as a
multi-threaded apartment (MTA).
So I don't think the error is on this call-stack. (You can see it was still blocking on a WaitForSingleObject call, most likely the mechanism it uses to block until the process is to be shut down).

How to debug C++/CLI MFC app that crashes before InitInstance

I have this monstrous C++ application built using MFC(mainly) and COM. It links to several libraries and does a lot of scientific computing. So, now I want to add some new features to it and as an organizational policy, we are developing everything new using .NET. So, this new UI feature is going to be built using WPF and will be consumed in this existing C++ application.
I know how to use a WPF control in a C++ application, so that's not the problem. The problem is that when I try to turn on CLR on this project, it takes a lot of time (around 10 minutes) in linking stage to perform the linking and produce a mixed mode executable. In the end, it manages to do so and successfully creates the executable. But the problem is that whenever I launch this executable, it crashes. I tried to debug InitInstance but it crashes somewhere before this. I am a little stumped on what to try next.
Does anybody have any idea what might be the cause of this.
The target framework of the mixed mode assembly is 4.0 (as it should be) so THIS is not the problem here.
FYI, ILDasm fails to load this exe as well. It takes a lot of time, gives no error but it doesn't load it either. This gives me the impression that somehow managed image is not created properly.
Not sure if this is necessary, but it doesn't hurt either. New MFC projects have this in the constructor of your CWinApp-derived class:
#ifdef _MANAGED
// If the application is built using Common Language Runtime support (/clr):
// 1) This additional setting is needed for Restart Manager support to work properly.
// 2) In your project, you must add a reference to System.Windows.Forms in order to build.
System::Windows::Forms::Application::SetUnhandledExceptionMode(System::Windows::Forms::UnhandledExceptionMode::ThrowException);
#endif

.NET out-of-process COM objects sharing static instances in API calls

It's hard to explain our situaction.
We have a 3-tier application. The engine is a DLL coded in C++, then we have a VB6 ActiveX EXE that access to the engine via API calls, and at the top level we have a Excel Addin (in C# using VSTO framework) calling the middle layer with an interop DLL. At this time, each "connection" from the Addin to the engine creates a new EXE (VB6 uses API calls to access to the engine) and all works fine.
Now we are moving the middle layer to .NET, it works 'presumably' fine (it pass all our Unit test) but, we found an error when we open 2 "connections" at same time (ups, no unit test check this situation because it's a new behavour). The DLL have static objects that it's shared over all instances in the same process and we have interactions between "connections". In our old version each "connection" creates a new EXE with no memory sharing between processes, now it's the same process and they share memory and the static objects.
Following the tips from this question. We tried to build a COM EXE in C# to do an out-of-process objects in the middle layer but we have the same result. They share the static objects, at the end, each connection not creates a independent process.
It's clear, but not affordable at this time, moving API calls to ATL or changing the static objects to instanciable references with a handle and change all the API calls to get/set this handlers. I reviewed all examples in MS All-in-one but I didn't find any solution. Neither it's possible to keep only one connection at time, each workbook can have one connection and in the future we want to explore a Web application with multiple connections at same time.
Any suggestion?
Thanks in advance,
Whether COM starts new EXE per each COM object, or uses single EXE to instantiate all the object is controlled by flags parameters passed to CoRegisterClassObject. See
http://msdn.microsoft.com/en-us/library/ms693407(v=vs.85).aspx, and
http://msdn.microsoft.com/en-us/library/ms679697(v=vs.85).aspx
You need to pass REGCLS_SINGLEUSE or REGCLS_MULTI_SEPARATE flags.
Now, the trick is to pass this flag, as you might not call this method directly - the details depend on how you implemented the COM EXE.
it's not clear from the question, but it sounds like the "middle layer" you have was built as a VB6 EXE, and you're trying to replace it with a .net DLL. If that's the case, you'll definitely get the behavior you describe.
With a VB6 EXE com project, instantiating a new object starts a new process. With a .net dll (or a Vb6 dll really) you +won't+ get a new process.
You'd either need to create a .net EXE that exposes COM objects just like your VB6 exe does, or (sounds like you've already investigated this) you'll need to refactor your EXE objects to properly handle multiple instances within a single process.
Honestly, it'd probably be better to do that latter, since relying on singletons like this is generally a bad code smell. But it a pinch, you should be able to replicate the behavior of the VB6 exe with a .net project. You just can't do it in a dll.
Was your middle layer created in .Net? If it was, you might be facing the issue that your COM class is been created as a native .net object instead of a COM object. The solution usually involve using Primary Interop Assemblies. Take a look on this SO question to see if it matches your problem.

How to create a cross-platform DLL in .net

I have a interesting problem: Where I work we've built a home-grown ERP system in VB6 that we are slowly moving over into vb.net. There are some projects have are in .net: we have a hand-held C# project that uses a web service to talk to our database, I've built some reporting screens using Crystal and some smaller maintenance screens.
Well as we have been plotting the conversion out, we want to have a way to separate our business logic and UI so that the UI can be a win/web form or a Smart Device project. Is this even possible? I try to reference the DLL in a test I have and it gives me this error when trying to debug using a emulator
Deployment and/or registration failed with error: 0x8973190e. Error writing file '%csidl_program_files%\smartdeviceproject1\system.windows.forms.dll'. Error 0x80070070: There is not enough space on the disk.
I'm not sure what it's doing... I take my DLL out and it works fine. Does anyone know of a way I can create a DLL that can target all of these UI without may changes?
This post here
helped me alot. Using a linked projects with conditional complation would seem to work in my case.