CreateProcess with CREATE_SUSPENDED cannot be ResumeThread-ed - vba

Question

I'm stuck using WINAPI's CreateProcess with CREATE_SUSPENDED and ResumeThread from VBA.
I would like to start a process (and receive it's Process ID) and be able to suspend and resume its main thread (depending on a more complicated scheme considering the resource utilisation use of my computer - not detailed here). I came up with the below code and have the following problems:
LastDllError is 18 after calling CreateProcess although the return
value is nonzero. What does this mean?
ResumeThread fails with ERROR_INVALID_HANDLE, and does not resume
it. What is wrong here?
My code:
Option Explicit
Private Type SECURITY_ATTRIBUTES
nLength As Long
lpSecurityDescriptor As Long
bInheritHandle As Long
End Type
Private Type STARTUPINFO
cb As Long
lpReserved As String
lpDesktop As String
lpTitle As String
dwX As Long
dwY As Long
dwXSize As Long
dwYSize As Long
dwXCountChars As Long
dwYCountChars As Long
dwFillAttribute As Long
dwFlags As Long
wShowWindow As Integer
cbReserved2 As Integer
lpReserved2 As Byte
hStdInput As Long
hStdOutput As Long
hStdError As Long
End Type
Private Type PROCESS_INFORMATION
hProcess As Long
hThread As Long
dwProcessId As Long
dwThreadId As Long
End Type
Private Const CREATE_SUSPENDED As Long = 4
Private Declare Function CreateProcess Lib "kernel32" Alias "CreateProcessA" ( _
ByVal lpApplicationName As String, _
ByVal lpCommandLine As String, _
ByRef lpProcessAttributes As SECURITY_ATTRIBUTES, _
ByRef lpThreadAttributes As SECURITY_ATTRIBUTES, _
ByVal bInheritHandles As Long, _
ByVal dwCreationFlags As Long, _
ByRef lpEnvironment As Any, _
ByVal lpCurrentDirectory As String, _
ByRef lpStartupInfo As STARTUPINFO, _
ByRef lpProcessInformation As PROCESS_INFORMATION) As Long
Private Declare Function SuspendThread Lib "kernel32" (hThread As Long) As Long
Private Declare Function ResumeThread Lib "kernel32" (hThread As Long) As Long
Private Declare Function CloseHandle Lib "kernel32.dll" (ByVal hObject As Long) As Long
Private Declare Function DebugActiveProcess Lib "kernel32" (ByVal dwProcessId As Long) As Long
Private Declare Function DebugActiveProcessStop Lib "kernel32" (ByVal dwProcessId As Long) As Long
Private Declare Sub Sleep Lib "kernel32" (ByVal dwMilliseconds As Long)
Public Function WinApi_CreateProcess(strCommandLine As String, Optional strCurrentDirectory As String = vbNullString) As Long
If strCurrentDirectory = vbNullString Then
strCurrentDirectory = ThisWorkbook.Path
End If
Dim sap As SECURITY_ATTRIBUTES: sap.nLength = Len(sap)
Dim sat As SECURITY_ATTRIBUTES: sat.nLength = Len(sat)
Dim si As STARTUPINFO: si.cb = Len(si)
Dim pi As PROCESS_INFORMATION
Debug.Print Err.LastDllError ' 0 => ERROR_SUCCESS
Dim dwResult As Long: dwResult = CreateProcess(vbNullString, strCommandLine, sap, sat, 0, CREATE_SUSPENDED, 0, strCurrentDirectory, si, pi)
Debug.Print Err.LastDllError ' 18 => ERROR_NO_MORE_FILES (but dwResult <> 0 => Success)
If dwResult = 0 Then
WinApi_CreateProcess = 0: Exit Function
End If
CloseHandle pi.hProcess
Debug.Print Err.LastDllError ' 0 => ERROR_SUCCESS
Dim dwSuspendCount As Long: dwSuspendCount = ResumeThread(pi.hThread)
Debug.Print dwSuspendCount ' -1
If dwSuspendCount = -1 Then
Debug.Print Err.LastDllError ' 6 => ERROR_INVALID_HANDLE
CloseHandle pi.hThread
WinApi_CreateProcess = 0: Exit Function
Else
Debug.Print Err.LastDllError ' Not this branch
CloseHandle pi.hThread
WinApi_CreateProcess = pi.dwProcessId: Exit Function
End If
End Function

Answer 1

After modify all pointer types to LongPtr instead of Long, I can use the sample to reproduce your problem with 64-bit Excel. You could also refer to the statement here
Private Type SECURITY_ATTRIBUTES
nLength As Long
lpSecurityDescriptor As LongPtr
bInheritHandle As Long
End Type
Private Type STARTUPINFO
cb As Long
lpReserved As String
lpDesktop As String
lpTitle As String
dwX As Long
dwY As Long
dwXSize As Long
dwYSize As Long
dwXCountChars As Long
dwYCountChars As Long
dwFillAttribute As Long
dwFlags As Long
wShowWindow As Integer
cbReserved2 As Integer
lpReserved2 As LongPtr
hStdInput As LongPtr
hStdOutput As LongPtr
hStdError As LongPtr
End Type
Private Type PROCESS_INFORMATION
hProcess As LongPtr
hThread As LongPtr
dwProcessId As Long
dwThreadId As Long
End Type
Then I get true result for CreateProcess and ERROR_INVALID_HANDLE error for ResumeThread, I found that ResumeThread parameter did not declare with ByVal(the same as SuspendThread). After add ByVal, the sample works for me.
Declare PtrSafe Function SuspendThread Lib "kernel32" (ByVal hThread As LongPtr) As Long
Declare PtrSafe Function ResumeThread Lib "kernel32" (ByVal hThread As LongPtr) As Long

Answer 2

LastDllError is 18 after calling CreateProcess although the return value is nonzero. What does this mean?
It means you are using Err.LastDllError incorrectly. If CreateProcess() succeeds (returns non-zero), the value of Err.LastDllError is indeterminate, so just ignore it. Its value will be meaningful only if CreateProcess() fails (returns zero).
ResumeThread fails with ERROR_INVALID_HANDLE, and does not resume it. What is wrong here?
You are checking the return value of ResumeThread() incorrectly, so you are checking Err.LastDllError at the wrong time, again.
Per the ResumeThread() documentation:
If the function succeeds, the return value is the thread's previous suspend count.
If the function fails, the return value is (DWORD) -1. To get extended error information, call GetLastError.
In this case, you are checking the return value of ResumeThread() against 0, but the process was created in a suspended state, so its main thread's suspend count would be 1, and so ResumeThread() should be returning 1 if the thread is successfully resumed, but you are treating that as a failure condition instead of a success condition.
You need to change this:
If ResumeThread(pi.hThread) <> 0 Then
To this:
If ResumeThread(pi.hThread) = -1 Then
And clean up your use of Err.LastDllError, eg:
Public Function WinApi_CreateProcess(strCommandLine As String, Optional strCurrentDirectory As String = vbNullString) As Long
If strCurrentDirectory = vbNullString Then
strCurrentDirectory = ThisWorkbook.Path
End If
Dim sap As SECURITY_ATTRIBUTES: sap.nLength = Len(sap)
Dim sat As SECURITY_ATTRIBUTES: sat.nLength = Len(sat)
Dim si As STARTUPINFO: si.cb = Len(si)
Dim pi As PROCESS_INFORMATION
Dim dwResult As Long: dwResult = CreateProcess(vbNullString, strCommandLine, sap, sat, 0, CREATE_SUSPENDED, 0, strCurrentDirectory, si, pi)
If dwResult = 0 Then
Debug.Print Err.LastDllError
WinApi_CreateProcess = 0: Exit Function
End If
CloseHandle pi.hProcess
Dim dwSuspendCount As Long: dwSuspendCount = ResumeThread(pi.hThread)
If dwSuspendCount = -1 Then
Debug.Print Err.LastDllError
CloseHandle pi.hThread
WinApi_CreateProcess = 0: Exit Function
End If
CloseHandle pi.hThread
WinApi_CreateProcess = pi.dwProcessId
End Function
But, you really don't need to create a process suspended and then resume it just to grab its process ID. Get rid of CREATE_SUSPENDED and ResumeThread() altogether, you don't actually need them in this situation:
Public Function WinApi_CreateProcess(strCommandLine As String, Optional strCurrentDirectory As String = vbNullString) As Long
If strCurrentDirectory = vbNullString Then
strCurrentDirectory = ThisWorkbook.Path
End If
Dim sap As SECURITY_ATTRIBUTES: sap.nLength = Len(sap)
Dim sat As SECURITY_ATTRIBUTES: sat.nLength = Len(sat)
Dim si As STARTUPINFO: si.cb = Len(si)
Dim pi As PROCESS_INFORMATION
Dim dwResult As Long: dwResult = CreateProcess(vbNullString, strCommandLine, sap, sat, 0, 0, 0, strCurrentDirectory, si, pi)
If dwResult = 0 Then
Debug.Print Err.LastDllError
WinApi_CreateProcess = 0: Exit Function
End If
CloseHandle pi.hThread
CloseHandle pi.hProcess
WinApi_CreateProcess = pi.dwProcessId
End Function