How to debug VC++ dll in VB6 - vb.net

I tried a few different ways, but none of them works.
Method 1 I tried:
Start the VB program and run until before the DLL runs.
Open VC++ (no project) and select Build | Start Debug | Attach to Process
Attach to the VB program
Open the DLL source code file in VC++, select a breakpoint in the code.
Continue running the VB program, and it didn't jump to the VC++.
Method 2 I tried:
"After a long struggle found the answer for this. Actually we have to give the path of VB6.exe in the Projects->settings->debug->category->general->Executable for debug session.Then when you try to debug the VC++ dll by having a breakpoint in the function that needs to be checked,a new VB6 IDE would be opened.There please select the VB6 Dll project that actually calls the VC++6 dll. run that application and when the function call to the VC++6 comes in the VB6 function, automatically the control goes to the VC++ IDE and you can debug it as usua" From MSDN---->It just went to the next line of code in VB application, didn't go to the VC++ IDE like method 1.
The VB application declares dll file like this:
dim query as object
query=CreateObject("ClientServer.getResultClass")
query.execute(parameter1,parameter2)
I'm not sure if I need to put the debug dll into a specific place or I may need to register the debug dll file (I checked gedit, the ClientServer.getResultClass is registered).

You're on the right track with Method 1. You need to make sure you attach the 'Native' debugger, and that the DLL includes debug info, and that the source code matches the DLL. If all three conditions are met, it should work. Note that it's almost always necessary to use a freshly built DLL to ensure that the third condition is met.

Related

Changing a DLL that is used by an Agilent Vee Pro 6.01 Compiled Program

I have an executable that looks for a particular DLL. I have changed the source for DLL and recompiled it (written and recompiled in VB6). Once I replace the DLL, the executable hits a runtime error when it gets to using that particular DLL. Works ok when I recompile the executable.
So my question is, with same DLL path, same name, and virtually identical DLLs, why does the executable need to be recompiled?
This is driving me bananas so any thoughts would be appreciated. Thanks, Callum.
A VB6 (or any COM) DLL has unique IDs for itself and its public interfaces, if you recompile these can change and any existing code bound to the old IDs fails.
Tldr; Tick "Binary Compatibility" in the DLL's project options & select the old working DLL as the thing to maintain compatibility with & recompile.
Detailed explanation: I keep hearing about DLL hell - what is this?

How can I debug a referenced DLL file from Visual Studio 2013?

To start, I know this question has been asked a million times all over the place, but as I have no experience with VB/Visual Studio, I can't find a solution that I can make sense of.
I am creating a new VB.net project(solution?), and am making calls to functions from a C library that are in a dll file. The dll file does have a pdb file and they are both stored at the same location.
In the code below it shows how I declare the functions in my VB.net code, but I have not figured out how to attach the pdb file to the project.
Declare Function InitRelay Lib "Z:\Devel\RelayAPI\Debug\RelayAPI.dll" (ByVal setbaud As Action(Of Short), ByVal getit As Func(Of Short, Short), ByVal putit As Action(Of Short), ByVal flushit As Action, ByVal delay As Action(Of Short)) As Byte
Declare Sub FreeRelay Lib "Z:\Devel\RelayAPI\Debug\RelayAPI.dll" ()
...
I am getting an exception somewhere in the DLL file, but the way I have it set up, I can not debug the dll file. Whether its adding breakpoints, or print statements, I need a way to see where in the dll the project fails.
Questions I have looked at:
How to debug a referenced dll - I tried following the menu path given in the accepted answer, but when I go to Project >> Project Properties I see no Build option. It also says I can load symbols directly in the IDE if I don't want to copy any files, but I cannot find an explanation on how to do it. EDIT - As Plutonix says below, C# Build is the equivalent of VB's Compile tab. I checked and my Debug Info is set to Full, so this does not solve my problem.
Debugging a third-party DLL in Visual Studio? - This talks about a DLL in a .NET language but mine is in C. It also only tells you how to view the code, which I already can do. I have access to the .c and .h files that are used to create the dll, I just cannot debug them at runtime.
When you debug a DLL, you can start debugging from:
The project used to create the executable that calls the DLL.
- or -
The project used to create the DLL itself.
There are a couple of ways to debug a reference DLL file using VS 2013, so here the method Microsoft uses
-->To specify an executable for the debug session
In Solution Explorer, select the project that creates the DLL.
From the View menu, choose Property Pages.
In the Property Pages dialog box, open the Configuration Properties folder and select the Debugging category.
In the Command box, specify the path name for the container. For example, C:\Program Files\MyApplication\MYAPP.EXE.
In the Command Arguments box, specify any necessary arguments for the executable.
Here I included 2 different methods for debugging a DLL file directly from Microsoft.
1> http://msdn.microsoft.com/en-us/library/605a12zt.aspx (Native Mode)
2> http://msdn.microsoft.com/en-us/library/kbaht4dh.aspx (Mixed Mode)
If this was anyhelp to you or solved your problem please make sure to drop a "vote up"
Ref. All information is from personal experience and Microsoft

VB6 reference confusion

I am a .NET Developer. I have a good understanding of how references work in .NET i.e. if you want to use AssemblyA.ClassA.MethodA in AssemblyB.classB.MethodB then you add a reference in Assembly A to Assembly B.
I am looking at a VB6 app and I get an error when I open it i.e. Errors during load. Refer to frmMain.log. I open frmMain.log and it says: "2142: Class MSComDlg.CommonDialog of control CommonDialog1 was not a loaded control class.". I have used WinMerge to compare the source code in the faulty project to a previous version of the project (which does not have the problem). The only difference is that the faulty project contains the following line in the client.vdp file:
Reference=*\G{08DBEFD7-6A19-4DCE-A533-5BDBB93683C8}#1.2#0#..\..\..\..\..\Windows\SysWOW64\Comdlg32.oca#Microsoft Common Dialog Control 6.0 (SP3)
Removing this line seems to resolve the problem. Why does this resolve the problem?
I have used Winmerge to compare yesterdays revision of the project (which does not have the problem) to todays revision of the project (which does have the problem). The only difference is the reference in the original post in the VBP file.
I would assume that someone installed the development version of this control on a different machine and then set a reference to it. When you try to open the project on your machine the reference fails. Or the control was uninstalled. It seems that the project was not using the reference and that it can be safely removed as it will still compile without the reference.
However, I have seen vb6 actually remove non-licensed controls from the form in question in this situation. So it may be that you can now compile because the form is missing a component that was there previously. Have you checked that frmMain has not changed, both the .frm and .frx files?

asm: Call a DLL

I disassemled a game's DLL and want to insert some code.
I need asm code to call another DLL in the current directory(I'm on Windows).
The background is, that I want to be able to execute custom code in my DLL,
but I can't load the DLL. So my idea was to load the DLL via modified game DLL.
There may be a function in the game which gives me the current directory path the DLL's are but I think I won't find it.
The calls you are looking for are LoadLibrary, which will search in a selection of places including the current directory for the DLL and then load it, then GetProcAddress.
If the DLL makes any other Win32 calls it is probably already linked against kernel32.dll, so that's all you need to do.
It is arguable as to whether modifying the DLL or using DLL injection is faster in terms of how long it takes to write the code since you're going to have to reverse engineer anyway, however, one advantage of pure DLL injection is that all existing code remains unmodified in terms of the installation, making these modifications easier to undo should the user wish to "unpatch" whatever you are doing.
Microsoft Detours comes with setdll.exe and withdll.exe, those utilities will let you start an exe with a custom dll file.

ActiveX component can't create an object

I am getting error,
ActiveX component can't create object.
Actually I have VB6 DLLs where business logic resides and I am calling that DLL function in my VB.NET application, all DLLs are dependent on other DLLs. I am getting the error from one of the DLLs
Any ideas?
Usually that error points to a COM dll that is not registered. You need to look at the chain of calls and find out which COM components you need and make sure they are available. Process Monitor (from sysinternals) may help you find it when it reads from the registry and can't find it.
Four common causes are:
1. You do not have a required TLB or ActiveX DLL/OCX file
2. A TLB or ActiveX DLL/OCX needed by the project is present but not registered on your system
3. The VB runtimes are an earlier version than the one you need to run the project
4. A required TLB or ActiveX DLL/OCX file is corrupt
Do you have access to the troublesome DLL source code?
Also check permissions. The user account that the vb.net application is running under will need permissions to be able to create the vb6 dll
Process Monitor will definately help find the missing DLL, but frequently the task can be achieved more quickly with Dependency Walker.