Imported native function doesn't work in .NET 4.0 - vb.net

Question

I am migrating project from .net 3.5 to .net 4.0 and faced the following issue.
There are 2 DllImport statements:
<DllImport("hid64.dll")> _
Public Sub GenerateHardwareID( _
<MarshalAs(UnmanagedType.LPArray, SizeParamIndex:=1)> ByVal Buffer As Byte(), _
ByVal BufferLength As Int32)
End Sub
<DllImport("hid64.dll")> _
Public Function BufferToString( _
<MarshalAs(UnmanagedType.LPArray, SizeParamIndex:=1)> ByVal Buffer As Byte(), _
ByVal BufferLength As Int32) As <MarshalAs(UnmanagedType.LPWStr)> String
End Function
For the .NET 3.5 both functions work well. But for the .NET 4.0 the call of the BufferToString function breaks execution of the programm without raising any exception.
I played around with CallingConvention, CharSet and so on fields of the DllImport attribute:
http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.dllimportattribute.aspx
without success.
This variant:
<DllImport("hid64.dll", CharSet:=CharSet.Auto, PreserveSig:=False, SetLastError:=True)> _
Public Function BufferToString( _
<MarshalAs(UnmanagedType.LPArray)> ByVal Buffer As Byte(), _
ByVal BufferLength As Int32) As <MarshalAs(UnmanagedType.LPWStr)> String
End Function
does not break execution of the programm but the function returns 'Nothing'.

Answer 1

Here's what I believe to be the most likely explanation.
The BufferToString function has a return value that is a string. The p/invoke marshaller has to marshal that from native to managed. It does that by assuming that the native code returns a null-terminated character pointer that was allocated by the COM allocator. When it has finished transferring the content to a .net string it calls CoTaskMemFree on the pointer. If that memory was not allocated by the COM allocator then you may see failures at this point.
To get around the problem you have a few options. You could change the p/invoke for BufferToString to return IntPtr. Copy the contents to a .net string with Marshal.PtrToStringUni. This then leaves you with the responsibility of disposing of the unmanaged memory. Presumably the unmanaged library offers you a mechanism to do that.
If you wrote the unmanaged library then you can use an alternative solution. Leave the p/invoke exactly as it currently is but change the unmanaged library to allocate the return value using CoTaskMemAlloc. That then will match with the p/invoke marshaller's assumptions.