COM interface (MIDL): size_is for an optional [out] parameter - com

In short, in the following definition:
HRESULT GetStuff([in] long count,
[out, size_is(count)] long * a,
[out, size_is(count)] long * b);
which fills a and b with count elements, is it legal for the caller to set a and / or b to null?
I want to allow the caller to query only one set of results, so the method may be called with
long only_a_s[10];
itf->GetStuff(10, a, 0);
Do I need to modify the MIDL declaration? I'm unsertain how the pointer/pointer_default attributes play into this.
Notes: There's overhead in acquiring them separately, but so is acquiring values the caller doesn't need, so separate getters or always having to get both is sub-par. I know it does work for inproc / in-apartment calls, but would the MIDL-generated proxy/stub deal with that correctly?

You cannot pass a null pointer as an argument (a.k.a. top-level pointer), as they're [ref] by default.
But you may pass a null pointer, in or out, for non-top-level pointers.
The IDL method definition goes like this:
HRESULT GetStuffAB([in] long countA,
[in] long countB,
[out, size_is(, *pItemsA)] long **pA,
[out, size_is(, *pItemsB)] long **pB,
[out] long *pItemsA,
[out] long *pItemsB);
The C++ implementation:
HRESULT CMyClass::GetStuffAB(long countA,
long countB,
long **pA,
long **pB,
long *pItemsA,
long *pItemsB)
{
// COM stubs will check this for you
// However, you should (must?) manually check in same apartment calls
if (!pA) return E_POINTER;
if (!pB) return E_POINTER;
if (!pItemsA) return E_POINTER;
if (!pitemsB) return E_POINTER;
*pA = nullptr;
*pB = nullptr;
*pItemsA = 0L;
*pItemsB = 0L;
if (countA < 0) return E_INVALIDARG;
if (countB < 0) return E_INVALIDARG;
// Get amount of As into *pItemsA if countA > 0
// Get amount of Bs into *pItemsB if countB > 0
if (*pItemsA < 0) return E_FAIL;
if (*pItemsB < 0) return E_FAIL;
if (*pItemsA > 0)
{
*pA = CoTaskMemAlloc(sizeof(long) * *pItemsA);
if (!*pA) return E_OUTOFMEMORY;
}
if (*pItemsB > 0)
{
*pB = CoTaskMemAlloc(sizeof(long) * *pItemsB);
if (!*pB)
{
if (*pA)
{
// You should not assume the memory will be freed by the caller
// in such drastic situations, so free and clear *pA here before returning
CoTaskMemFree(*pA);
*pA = nullptr;
}
return E_OUTOFMEMORY;
}
}
// Get As into *pA and Bs into *pB
// Or just copy them if getting the amounts implied getting the items
// You could just as well always return S_OK
return (*pItemsA > 0 || *pItemsB > 0) ? S_OK : S_FALSE;
}
Not shown is whatever other code you must implement yourself to get the amount of As and the amount of Bs, and the As and Bs themselves.
If you must get the items to know the amounts, and you're not using RAII, you should free those resources manually before returning.
As an alternative to using CoTaskMemAlloc and CoTaskMemFree, you may want to use ATL's CComHeapPtr, which will automatically free memory RAII-style, thus simplifying your code. Just make sure you call Detach into *pA and *pB before returning successfully.

Related

Generate a tree of structs with testing/quick, respecting invariants

I have a tree of structs which I'd like to test using testing/quick, but constraining it to within my invariants.
This example code works:
var rnd = rand.New(rand.NewSource(time.Now().UnixNano()))
type X struct {
HasChildren bool
Children []*X
}
func TestSomething(t *testing.T) {
x, _ := quick.Value(reflect.TypeOf(X{}), rnd)
_ = x
// test some stuff here
}
But we hold HasChildren = true whenever len(Children) > 0 as an invariant, so it'd be better to ensure that whatever quick.Value() generates respects that (rather than finding "bugs" that don't actually exist).
I figured I could define a Generate function which uses quick.Value() to populate all the variable members:
func (X) Generate(rand *rand.Rand, size int) reflect.Value {
x := X{}
throwaway, _ := quick.Value(reflect.TypeOf([]*X{}), rand)
x.Children = throwaway.Interface().([]*X)
if len(x.Children) > 0 {
x.HasChildren = true
} else {
x.HasChildren = false
}
return reflect.ValueOf(x)
}
But this is panicking:
panic: value method main.X.Generate called using nil *X pointer [recovered]
And when I change Children from []*X to []X, it dies with a stack overflow.
The documentation is very thin on examples, and I'm finding almost nothing in web searches either.
How can this be done?
Looking at the testing/quick source code it seems that you can't create recursive custom generators and at the same time reuse the quick library facilities to generate the array part of the struct, because the size parameter, that is designed to limit the number of recursive calls, cannot be passed back into quick.Value(...)
https://golang.org/src/testing/quick/quick.go (see around line 50)
in your case this lead to an infinite tree that quickly "explodes" with 1..50 leafs at each level (that's the reason for the stack overflow).
If the function quick.sizedValue() had been public we could have used it to accomplish your task, but unfortunately this is not the case.
BTW since HasChildren is an invariant, can't you simply make it a struct method?
type X struct {
Children []*X
}
func (me *X) HasChildren() bool {
return len(me.Children) > 0
}
func main() {
.... generate X ....
if x.HasChildren() {
.....
}
}

C General Protection Fault trying to access object in array

In my program, I store objective-c objects in a c array, like this
va_start(list, o);
retval->objs = malloc(SIZE * count);
retval->objs[0] = (__bridge void *)o;
for (int i = 1; i < count; i++)
{
id o = va_arg(list, id);
retval->objs[i] = (__bridge void *)o;
}
va_end(list);
(count is a number containing how many objects will be added; that value is always correct)
objs is a void ** and is part of retval, which is a pointer to a struct. As of now, SIZE is defined as 100. Increasing and decreasing that had no effect.
As you can see, I bridge o to a void *, as I have to. objs, when all the objects are added, contains 3 objective-c objects. When I try to access a value like this
void *obj = CLArrayObjectAtIndex(_arr, ind);
return (__bridge id)obj;
this is the CLArrayObjectAtIndex() function
void *CLArrayObjectAtIndex(CLArrayType *arr, int ind)
{
void *o = arr->objs[ind];
if (o)
return o;
else
perror("Attempt to access NULL object or index out of bounds."), abort();
}
if the index (ind) is 0, it works. If the index is 1, the program crashes when it returns in main. If the index is 2, the program crashes as soon as I try to access it. If the index is 1, the value returned above is correct, but when the program crashes on return it is nil.
If the index is 1, the EXC_BAD_ACCESS code is 1; if the index is 2, the code is EXC_I386_GPFLT, a general protection fault. I already checked here for an explanation of this exception, although I couldn't find anything helpful. So, does anybody see why this error may be occurring?
when you store obj-c objects in C array don't just bridge cast them since that way arc doesn't know they are still used and releases them. __bridge_retain them so they stay around later, when you free the array __bridge_transfer them to give them back to ARC
also don't define size as 100.. sizeof(id) should work. You only need to store pointers

What is the correct way to cast when using ATL and IUnknownPtr?

During the modification of an existing ATL COM object I came across an article from the "The Old New Thing" blog called "The ways people mess up IUnknown::QueryInterface" and there was a discussion in the comments section that started when one of the respondents (Norman Diamond) pointed out that that in one of the article's examples that the cast to void** was incorrect.
However when I try and correct my code to do the casting properly I end up with a memory leak.
The example was as follows:
IShellFolder *psf = some object;
IUnknown *punk = NULL;
psf->QueryInterface(IID_IUnknown, (void**)&punk);
Norman said
punk is not a void*. punk is an IUnknown*.
void** is not a universal pointer type. void* is a universal pointer type, and char* and relatives are grandparented in to be equivalent in that way, but void** is not.
If you want to obey the calling convention and avoid horrible deaths, you have to do this:
IUnknown *punk;
void *punkvoid;
psf->QueryInterface(IID_IUnknown, &punkvoid);
punk = (IUnknown *)punkvoid;
Lots of other MSDN contributors made the same identical mistake.... some people might say that it works in all VC++ implementations to date, but that doesn't make it correct code, and it's still violating the calling convention.
In light of this I went to change my old code - which was as follows:
#include <comdef.h>
...
HRESULT FinalConstruct()
{
if (m_dwROTCookie != 0)
return E_FAIL;
//Check whether there already is an instance of the Object
IUnknownPtr pUnk = NULL;
if (GetActiveObject(CLSID_Object, NULL, &pUnk) == S_OK)
{
TRACE_WARNING("An instance of Object already exists in the current context");
return S_OK;
}
HRESULT hr = QueryInterface(IID_IUnknown, reinterpret_cast<void **>(&pUnk));
hr = RegisterActiveObject(pUnk, CLSID_Object, ACTIVEOBJECT_WEAK, m_dwROTCookie);
if (FAILED(hr))
return hr;
hr = CoLockObjectExternal(pUnk, TRUE, TRUE);
pUnk = NULL;
ATLASSERT(m_dwRef == 2);
return hr;
}
I then changed it as follows:
HRESULT FinalConstruct()
{
if (m_dwROTCookie != 0)
return E_FAIL;
//Check whether there already is an instance of the Object
IUnknownPtr pUnk = NULL;
if (GetActiveObject(CLSID_Object, NULL, &pUnk) == S_OK)
{
TRACE_WARNING("An instance of Object already exists in the current context");
return S_OK;
}
void* pUnkVoid = NULL;
HRESULT hr = QueryInterface(IID_IUnknown, &pUnkVoid);
if (SUCCEEDED(hr)
{
pUnk = reinterpret_cast<IUnknown*>(pUnkVoid);
hr = RegisterActiveObject(pUnk, CLSID_Object, ACTIVEOBJECT_WEAK, m_dwROTCookie);
if (FAILED(hr))
return hr;
hr = CoLockObjectExternal(pUnk, TRUE, TRUE);
pUnk = NULL;
}
ATLASSERT(m_dwRef == 2);
return hr;
However now my application has a memory leak from this the COM Object
You likely have a memory leak because you call GetActiveObject() and QueryInterface() which upon success increment the reference count on the object, but don't call Release() later to decrement the reference count.
Mmm, I think that rather than assigning the void* to pUnk I should be using:
pUnk.Attach(reinterpret_cast<IUnknown*>(pUnkVoid));

How to iterate the vtable of COM coclass?

How can I iterate/access the vtable of COM coclass which will implement the methods of its exposed interfaces?
I need to access the part of the vtable where all addresses of exposed methods of its interfaces are stored.
e.g. Math is COM object, its exposed interface is "Operations" and "Sum" is the method of this interface, how do I get the address of "Sum"?
I'm not going to ask why are you doing it this way, but perhaps this could help...
Every COM object must implement at least the IUnknown interface. Hence, the first four bytes of the COM object instance is the pointer to IUnknown object. The first four bytes of the IUnknown object (and any other object with virtual functions) is the pointer to vtbl.
(There is no error checking in this example, so please don't split hair on that subject.)
I used an instance of IReferenceClock for demonstration.
int main()
{
CoInitialize( NULL );
IReferenceClock* pRefClock;
HRESULT hr = CoCreateInstance( CLSID_SystemClock, NULL, CLSCTX_INPROC_SERVER, IID_IReferenceClock, (void**)&pRefClock );
DWORD* pIUnknownAddress = (DWORD*)pRefClock;
DWORD* pVTBLaddress = (DWORD*)*pIUnknownAddress;
// for example, the next interface could be accessed like this
DWORD* pNextInterfaceAddress = ( (DWORD*)pRefClock ) + 1;
DWORD* pNextVTBLaddress = (DWORD*)*pNextInterfaceAddress;
// and you would access virtual functions in the same way as QueryInterface, AddRef and Release below in this example
HRESULT (__stdcall *pQueryInterfaceFunction)(void*, REFIID, void**);
ULONG (__stdcall *pAddRef)( void* );
ULONG (__stdcall *pRelease)( void* );
// IUnknown looks like this:
//
// virtual HRESULT QueryInterface( REFIID riid, void** ppvObject);
// virtual ULONG AddRef( void );
// virtual ULONG Release( void );
//
// So, the first function in vtbl is QueryInterface, the second is AddRef...
pQueryInterfaceFunction = (HRESULT (__stdcall*)(void*, REFIID, void**))*pVTBLaddress;
pAddRef = (ULONG (__stdcall *)( void* ))*( pVTBLaddress + 1 );
pRelease = (ULONG (__stdcall *)( void* ))*( pVTBLaddress + 2 );
// Note: extra void* is actually this pointer.. see below that we pass pRefClock to every call
IUnknown* pUnknown;
UINT nRefCount;
hr = pQueryInterfaceFunction( pRefClock, IID_IUnknown, (void**)&pUnknown );
if( SUCCEEDED( hr ) )
{
nRefCount = pUnknown->Release();
ATLTRACE( TEXT( "nRefCount = %d\n" ), nRefCount );
}
nRefCount = pAddRef( pRefClock );
ATLTRACE( TEXT( "nRefCount after AddRef() call = %d\n" ), nRefCount );
nRefCount = pRelease( pRefClock );
ATLTRACE( TEXT( "nRefCount after Release() call = %d\n" ), nRefCount );
nRefCount = pRefClock->Release();
CoUninitialize();
return 0;
}
Sorry to answer with a question, but I have to ask "from where?"
If you mean, how can you iterate through the vtable from a COM client, I don't think you can. On the client side, all you have is a proxy that knows how to communicate (maybe cross-apartment or cross-process) with the COM server. You could maybe probe the vtable of that proxy, but it can never tell you the addresses of the functions inside the COM server.
Of course, if the server is actually running in a different process, the address of the functions might be of little use to you. Even if the server is in the same process, but in a different apartment, getting function addresses might be dangerous: you could call the functions directly, circumventing COM's interception, and break the server class's assumptions around calling thread, etc.
I guess that iterating the vtable is a means-to-an-end...? Maybe post what you're actually trying to do and I think COM probably has a way to do it.

What is the fastest way to compare two byte arrays?

I am trying to compare two long bytearrays in VB.NET and have run into a snag. Comparing two 50 megabyte files takes almost two minutes, so I'm clearly doing something wrong. I'm on an x64 machine with tons of memory so there are no issues there. Here is the code that I'm using at the moment and would like to change.
_Bytes and item.Bytes are the two different arrays to compare and are already the same length.
For Each B In item.Bytes
If B <> _Bytes(I) Then
Mismatch = True
Exit For
End If
I += 1
Next
I need to be able to compare as fast as possible files that are potentially hundreds of megabytes and even possibly a gigabyte or two. Any suggests or algorithms that would be able to do this faster?
Item.bytes is an object taken from the database/filesystem that is returned to compare, because its byte length matches the item that the user wants to add. By comparing the two arrays I can then determine if the user has added something new to the DB and if not then I can just map them to the other file and not waste hard disk drive space.
[Update]
I converted the arrays to local variables of Byte() and then did the same comparison, same code and it ran in like one second (I have to benchmark it still and compare it to others), but if you do the same thing with local variables and use a generic array it becomes massively slower. I’m not sure why, but it raises a lot more questions for me about the use of arrays.
What is the _Bytes(I) call doing? It's not loading the file each time, is it? Even with buffering, that would be bad news!
There will be plenty of ways to micro-optimise this in terms of looking at longs at a time, potentially using unsafe code etc - but I'd just concentrate on getting reasonable performance first. Clearly there's something very odd going on.
I suggest you extract the comparison code into a separate function which takes two byte arrays. That way you know you won't be doing anything odd. I'd also use a simple For loop rather than For Each in this case - it'll be simpler. Oh, and check whether the lengths are correct first :)
EDIT: Here's the code (untested, but simple enough) that I'd use. It's in C# for the minute - I'll convert it in a sec:
public static bool Equals(byte[] first, byte[] second)
{
if (first == second)
{
return true;
}
if (first == null || second == null)
{
return false;
}
if (first.Length != second.Length)
{
return false;
}
for (int i=0; i < first.Length; i++)
{
if (first[i] != second[i])
{
return false;
}
}
return true;
}
EDIT: And here's the VB:
Public Shared Function ArraysEqual(ByVal first As Byte(), _
ByVal second As Byte()) As Boolean
If (first Is second) Then
Return True
End If
If (first Is Nothing OrElse second Is Nothing) Then
Return False
End If
If (first.Length <> second.Length) Then
Return False
End If
For i as Integer = 0 To first.Length - 1
If (first(i) <> second(i)) Then
Return False
End If
Next i
Return True
End Function
The fastest way to compare two byte arrays of equal size is to use interop. Run the following code on a console application:
using System;
using System.Runtime.InteropServices;
using System.Security;
namespace CompareByteArray
{
class Program
{
static void Main(string[] args)
{
const int SIZE = 100000;
const int TEST_COUNT = 100;
byte[] arrayA = new byte[SIZE];
byte[] arrayB = new byte[SIZE];
for (int i = 0; i < SIZE; i++)
{
arrayA[i] = 0x22;
arrayB[i] = 0x22;
}
{
DateTime before = DateTime.Now;
for (int i = 0; i < TEST_COUNT; i++)
{
int result = MemCmp_Safe(arrayA, arrayB, (UIntPtr)SIZE);
if (result != 0) throw new Exception();
}
DateTime after = DateTime.Now;
Console.WriteLine("MemCmp_Safe: {0}", after - before);
}
{
DateTime before = DateTime.Now;
for (int i = 0; i < TEST_COUNT; i++)
{
int result = MemCmp_Unsafe(arrayA, arrayB, (UIntPtr)SIZE);
if (result != 0) throw new Exception();
}
DateTime after = DateTime.Now;
Console.WriteLine("MemCmp_Unsafe: {0}", after - before);
}
{
DateTime before = DateTime.Now;
for (int i = 0; i < TEST_COUNT; i++)
{
int result = MemCmp_Pure(arrayA, arrayB, SIZE);
if (result != 0) throw new Exception();
}
DateTime after = DateTime.Now;
Console.WriteLine("MemCmp_Pure: {0}", after - before);
}
return;
}
[DllImport("msvcrt.dll", CallingConvention = CallingConvention.Cdecl, EntryPoint="memcmp", ExactSpelling=true)]
[SuppressUnmanagedCodeSecurity]
static extern int memcmp_1(byte[] b1, byte[] b2, UIntPtr count);
[DllImport("msvcrt.dll", CallingConvention = CallingConvention.Cdecl, EntryPoint = "memcmp", ExactSpelling = true)]
[SuppressUnmanagedCodeSecurity]
static extern unsafe int memcmp_2(byte* b1, byte* b2, UIntPtr count);
public static int MemCmp_Safe(byte[] a, byte[] b, UIntPtr count)
{
return memcmp_1(a, b, count);
}
public unsafe static int MemCmp_Unsafe(byte[] a, byte[] b, UIntPtr count)
{
fixed(byte* p_a = a)
{
fixed (byte* p_b = b)
{
return memcmp_2(p_a, p_b, count);
}
}
}
public static int MemCmp_Pure(byte[] a, byte[] b, int count)
{
int result = 0;
for (int i = 0; i < count && result == 0; i += 1)
{
result = a[0] - b[0];
}
return result;
}
}
}
If you don't need to know the byte, use 64-bit ints that gives you 8 at once. Actually, you can figure out the wrong byte, once you've isolated it to a set of 8.
Use BinaryReader:
saveTime = binReader.ReadInt32()
Or for arrays of ints:
Dim count As Integer = binReader.Read(testArray, 0, 3)
Better approach... If you are just trying to see if the two are different then save some time by not having to go through the entire byte array and generate a hash of each byte array as strings and compare the strings. MD5 should work fine and is pretty efficient.
I see two things that might help:
First, rather than always accessing the second array as item.Bytes, use a local variable to point directly at the array. That is, before starting the loop, do something like this:
array2 = item.Bytes
That will save the overhead of dereferencing from the object each time you want a byte. That could be expensive in Visual Basic, especially if there's a Getter method on that property.
Also, use a "definite loop" instead of "for each". You already know the length of the arrays, so just code the loop using that value. This will avoid the overhead of treating the array as a collection. The loop would look something like this:
For i = 1 to max Step 1
If (array1(i) <> array2(i))
Exit For
EndIf
Next
Not strictly related to the comparison algorithm:
Are you sure your bottleneck is not related to the memory available and the time used to load the byte arrays? Loading two 2 GB byte arrays just to compare them could bring most machines to their knees. If the program design allows, try using streams to read smaller chunks instead.