I'm reorganzing a legacy mixed (managed and unmanaged DLLs) application so that the main application segment is unmanaged MFC and that will call a C++ DLL compiled with /clr flag that will bridge the communication between the managed (C# DLLs) and unmanaged code. Unfortuantely, my changed have resulted in an Access violation that occurs before the application InitInstance() is called. This makes it very difficult to debug. The only information I get is the following stack trace.
> 64006108()
ntdll.dll!_ZwCreateMutant#16() + 0xc bytes
kernel32.dll!_CreateMutexW#12() + 0x7a bytes
So, here are some sceanrios I've tried.
- Turned on Exceptions->Win32 Exceptions->c0000005 Access Violation to break when Thrown. Still the most detail I get is from the above stack trace. I've tried the application with F10, but it fails before any breakpoints are hit and fails with the above stack trace.
- I've stubbed out the bridge DLL so that it only has one method that returns a bool and that method is coded to just return false (no C# code called).
bool DllPassthrough::IsFailed() { return false; }
If the stubbed out DLL is compiled with the /clr flag, the application fails. If it is compiled without the /clr flag, the application runs.
- I've created a stub MFC application using the Visual Studio wizard for multidocument applications and call DllPassthrough::IsFailed(). This succeeds even with the /clr flag used to compile the DLL.
- I've tried doing a manual LoadLibrary on winmm.lib as outlined in the following note Access violation when using c++/cli. The application still fails.
So, my questions are how to solve the problem? Any hints, strategies, or previous incidents. And, failing that, how can I get more information on what code segment or library is causing the access exception? If I try more involved workarounds like doing LoadLibrary calls, I'd like to narrow it to the failing libraries.
Thanks. BTW, we are using Visual Studio 2008 and the project is being built against the .NET 2.0 framework for the managed sections.
I believe I solve my problem. By systematically removing each library reference and
commenting out the calls to that particular library in the application code (unmanaged), I eventually removed the problem library and got the program to run. It's a brute force way of diagnosing the problem, and fortunately I didn't have to remove too many libraries to solve it. I'd still be curious if anyone has a comment if the library could have been identified though the debugger.
So, the next step is to move these library calls to managed code and pass the information back to the unmanaged side via my bridge DLL. BTW, I reintegrated the winmm.lib into the project and it still works.
Related
I have a DirectShow filter (MonogramAAC encoder) successfully compiling. It also registers successfully, however, when I try to instantiate the filter in GrapheditPlus, it shows a CO_E_ERRORINDLL. After inspecting the filter properties under GraphEditPlus, the filter was actually linked to RegSvr32.exe instead of the actual host DLL!
Here's the evidence:
I'm sure the .def file in the linker is correct, I also tested that it IS being considered by the compiler and it is correctly pointing to the filter DLL.
A little bakground info:
I couldn't get the original project to compile under Visual Studio 2017 after automated conversion. (I couldn't resolve strange MFC and runtime library linker errors between the Monogram filter and the libaac.lib library, that I'm also able to compile successfully).
Finally I decided to recreate a brand-new VS2017 Project mimicking the original one and got it to compile and register (with the only warning being the mismatch between the Project name and the output library), but obviously I missed something important while recreating the project.
Any pointers?
Ok, I managed to find the reason for this unexpected behavior in this old post.
RegSvr32 just calls the DllRegisterServer entry point in your code.
What gets written to the registry is entirely up to your filter. If
you are using the standard baseclass call to AMovieDllRegisterServer2
as your DllRegisterServer, I would suspect that something went wrong
in setting up g_hInst, which should have been done by the call
DllEntryPoint in your DllMain.
…
The DllMain is not being called, so g_hInst stays 0 and
GetModuleFileNameA(...) returns the name of the currently executing
program instead of my filter.
The filter is using MFC (dynamically linked), and its DllMain() version is not calling the DllEntryPoint() function. I had to override the MFC DllMain() by adding extern "C" { int _afxForceUSRDLL; } as described in an answer to this question and call DllEntryPoint() myself to fix the registration problem. (I also found a missing #define was the reason for the property page not showing up).
Recently I've been working on an external .exe to communicate with AutoCAD. Since it was an outside .EXE I had to work with the COM to get what I wanted. (Else I could've used the .NET API which is (in my opinion) much better).
In any case, I keep getting COM Interop errors (Call was rejected by Callee) and I'm having a lot of trouble dealing with it. I could use timers but sometimes it will over wait, or under wait for a particular situation. Also, this isn't the best solution if I have to use a slower computer.
Microsoft's solution:
Add the following class to your application.
Add a reference to "Microsoft Development Environment 8.0." This
adds references to EnvDTE and EnvDTE80 to your solution.
Add a reference to System.Windows.Forms.
In the code, create an instance of EnvDTE80, as outlined in the
following example.
Call Message.Register to handle thread errors.
Call your automation code as usual.
When your automation code is finished, call Message.Revoke to remove
the thread error handlers.
http://msdn.microsoft.com/en-us/library/vstudio/ms228772(v=vs.100).aspx
Importing
In order to use the error management code (once the class is added) we will need to add an import such as:
Imports WindowsApplication1.ConsoleApplication2
I've created a mixed DLL (C++/CLI) and after successfully calling it from a plain ANSI C application, I've moved on to calling it from a C++ COM server (using the same C entry points). However even before the COM server successfully starts or calls into the DLL I get a "access violation" in ntdll.dll. The call stack just has ntdll.dll!ExecuteHandler2 repeated multiple times to the point where a stack over flow is reported in the VS debug output. I can see my mixed mode DLL and mscoree.dll are loaded.
I added the mixed DLL to the delay loaded DLL options of the COM server and it appears to work.
Why does the mixed DLL need to be delay loaded in the C++ COM Server when the C application I wrote seems to work fine without delay loading? How do I go about debugging this problem (unless this is expected, however I couldn't find anything about it)?
Looks like a COM Appartment needs to be initialized. COM appartments are used to care of threads synchronization.
I compile a DLL with mingw 4.5.0 and use it as a Game Maker 8.0 extension. Game Maker dynamically loads the dll. Everything appears to work (the dll functions are called and provide correct return values), but when I close Game Maker a dialog pops up: "Microsoft Visual C++ Runtime Library", "This application has requested the Runtime to terminate in an unusual way." After that, the process continues to linger in the background for a few seconds and then disappears.
This also happens when none of the functions of the dll are actually called. There is no DllMain, and all static/global variables are basic data types or std::string (it actually also happens when I remove the std::strings).
The dll statically links with zlib and libpng. The problem seems to vanish when I pass -static to the linker which (I assume) also links the runtime statically. However, this considerably inflates the size of my DLL, and it is at best a workaround until I understand what is going on.
Any ideas on what might be the cause?
Update: Actually, it seems that the problem only happens when two extensons are loaded in Game Maker: One with a dll linked with -static, and the other without. Linking both without -static makes the problem disappear. However, I still don't understand the problem, because the dlls never directly interact or share data structures.
Update 2: I recently found out that this might be related to strange behavior of Game Maker itself. It seems that global variables aren't initialized correctly when the DLL is loaded, which might cause a crash on unloading if global objects try to free memory they don't own. That would mean the -static was just a random factor that changed the value of the uninitialized memory the globals were associated with.
Update 3: Modified the above to include the info that this is about Game Maker extensions, since this is likely relevant as per Update 2.
It's just a guess. Try to look into something like the "static initialization order fiasco" that is described here. It may be that your problem is more related to the destructors (since it happens on close).
In short: I want to monitor selected calls from an application to a DLL.
We have an old VB6 application for which we lost the source code (the company wasn't using source control back then..). This application uses a 3rd party DLL.
I want to use this DLL in a new C++ application. Unfortunately the DLL API is only partially documented, so I don't know how to call some functions. I do have the functions signature.
Since the VB6 application uses this DLL, I want to see how it calls several functions. So far I've tried or looked at -
APIHijack - requires me to write C++ code for each function. Since I only need to log the values, it seems like an overkill.
EasyHook - same as 1, but allows writing in the code in .NET language.
OllyDbg with uHooker - I still have to write code for each function, this time in Python. Also, I have to do many conversions in Python using the struct module, since most functions pass values using pointers.
Since I only need to log functions parameters I want a simple solution. Is there any automated tool, for which I could tell which functions to monitor and their signature, and then get a detailed log file?
A "static" solution (in the sense it can capture a stack trace on demand) would be Process Monitor.
A more dynamic solution would be ApiMonitor, but it may be too old to be compatible with the applications to monitor. Worth a try though.
Some more Google searching found what I was looking for: WinAPIOverride32. It allows writing text files such as:
CustomApi.dll|void NameOfFunction(long param1, double& param2);
Later on, these files can be used inside the program to log all calls to NameOfFunction. Now I just need to figure out how to log arrays and structs parameters.
Visual Studio Addin Runtime Flow here:
Runtime Flow in real time monitors and logs function calls and
function parameters in your running .NET application and shows a stack
trace tree. No instrumentation or source code required for monitoring.
If you just want to see the function interfaces of the DLL, you could try "Dependecies" (https://lucasg.github.io/Dependencies/). This is a nice remake of the DependencyWalker in as OpenSource.
This only allows you to see the dependencies of the DLL, with the corresponding function names (however, not the calling structure). Unfortunately, I don't believe it will tell you which specific functions in a DLL are being used by the calling DLL/EXE.