When a foreign function declared as
int open_db(char *path, Db **db)
creates internally an instance of Db and assigns the pointer to *db, what is the most efficient way to handle this from Chez Scheme?
The only thing I could come up with was allocating memory for a C-pointer using foreign-alloc, passing the address to it, copying the address and then immediately freeing this memory:
(define open_db (foreign-procedure "open_db" (string void*) int))
(define-record-type db (fields (mutable ptr)))
(define (open-db path)
(let ((pptr (foreign-alloc (foreign-sizeof 'void*))))
(open_db path pptr)
(let ((ptr (foreign-ref 'void* pptr 0)))
(foreign-free pptr)
(make-db ptr))))
Is there a way to avoid this temporary allocation of memory for the pointer?
Related
I am considering using flatbuffers' serialized struct as a key in a key-value store. Here is an example of the structs that I want to use as a key in rocksdb.
struct Foo {
foo_id: int64;
foo_type: int32;
}
I read the documentation and figured that the layout of a struct is deterministic. Does that mean it is suitable to be used as a key? If yes, how do I serialize a struct and deserialize it back. It seems like Table has API for serialization/deserialization but struct does not (?).
I tried serializing struct doing it as follows:
constexpr int key_size = sizeof(Foo);
using FooKey = std::array<char, key_size>;
FooKey get_foo_key(const Foo& foo_object) {
FooKey key;
std::memcpy(&key, &foo_object, key_size);
return key;
}
const Foo* get_foo(const FooKey& key) {
return reinterpret_cast<const Foo*>(&key);
}
I did some sanity checks and the above seems to work in my Ubuntu 18 docker image and is blazing fast. So my questions are as follows:
Is this a safe thing to do on a machine if it passes FLATBUFFERS_LITTLEENDIAN and uint8/char equivalence checks? Or are there any other checks needed?
Are there any other caveats that I should be aware of when doing it as demonstrated above?
Thanks in advance !
You don't actually need to go via std::array, the Foo struct is already a block of memory that is safe to copy or cast as you wish. It needs no serialization functions.
Like you said, that memory contains little endian data, so FLATBUFFERS_LITTLEENDIAN must pass. Actually even on a big endian machine you may copy these structures all you want, as long as you use the accessors to read the fields (which do a byteswap on access on big endian). The only thing that won't work on big endian is casting the struct to, say, an int64_t * to read the first field without using the accessor methods.
The other caveat to certain casting operations is strict aliasing, if you have that turned on certain casts may be undefined behavior.
Also note that in this example Foo will be 16 bytes in size on all platforms, because of alignment.
On MacRuby Pointer to typedef struct, I learnt how to dereference a pointer created with
x=Pointer.new_with_type
...
==> use x.value, or x[0]
Works a treat !
Now I want to learn what I believe to be the "opposite". I'm trying to use this API.
OSStatus SecKeychainCopySettings (
SecKeychainRef keychain,
SecKeychainSettings *outSettings
);
Second parameter must be a Pointer. But I never manage to get the real outSettings of the keychain opened, I only get the default settings.
framework 'Security'
keychainObject = Pointer.new_with_type('^{OpaqueSecKeychainRef}')
SecKeychainOpen("/Users/charbon/Library/Keychains/Josja.keychain",keychainObject)
#attempt #1
settings=Pointer.new_with_type('{SecKeychainSettings=IBBI}')
SecKeychainCopySettings(keychainObject.value, settings)
p settings.value
#<SecKeychainSettings version=0 lockOnSleep=false useLockInterval=false lockInterval=0>
#attempt #2
settings2=SecKeychainSettings.new
result = SecKeychainCopySettings(keychainObject.value, settings2)
p settings2
#<SecKeychainSettings version=0 lockOnSleep=false useLockInterval=false lockInterval=0>
The settings of the Keychain opened should read
#<SecKeychainSettings version=0 lockOnSleep=true useLockInterval=true lockInterval=1800>
What am I missing ?
Got it !
The doc to SecKeychainCopySettings mentions
outSettings
On return, a pointer to a keychain settings structure.
Since this structure is versioned, you must allocate the memory for it
and fill in the version of the structure before passing it to the
function.
So we can't just create a Pointer to SecKeychainSettings. We must set the version of the Struct that is pointed by the pointer to to something.
settings=Pointer.new_with_type('{SecKeychainSettings=IBBI}')
#settings[0] dereferences the Pointer
#for some reason, settings[0][0]=1 does not work, nor settings[0].version=1
settings[0]=[1,false,false,0]
#we are redefining the complete SecKeychainSettings struct
# [0]=version [1]=lockOnSleep [2]=useLockInterval [3]=lockInterval
result = SecKeychainCopySettings(keychainObject.value, settings)
p settings
=> #<SecKeychainSettings version=1 lockOnSleep=true useLockInterval=false lockInterval=300> irb(main):019:0>
I am using DMD64 D Compiler v2.063.2 on Ubuntu 13.04 64-bit.
I have written a class as below:
class FixedList(T){
// list
private T[] list;
// number of items
private size_t numberOfItems;
// capacity
private size_t capacity;
// mutex
private Mutex listMutex;
// get capacity
#property public size_t Capacity(){ return capacity; }
#property public shared size_t Capacity(){ return capacity; }
// constructor
public this( size_t capacity ){
// initialise
numberOfItems = 0;
this.capacity = capacity;
writeln("Cons Normal");
}
// constructor
public shared this( size_t capacity ){
// initialise
numberOfItems = 0;
this.capacity = capacity;
// create mutex
listMutex = cast(shared)(new Mutex());
writeln("Cons Shared");
}
}
While class is written in this way, in main function, I wrote that code:
auto list1 = new shared FixedList!int( 128 );
auto list2 = new FixedList!int( 128 );
Output with this, there is no error at all and the output is as below:
Cons Shared
Cons Normal
What I do next is to remove both writeln lines from the code, and when I recompile the code, it starts showing error messages as below:
src/webapp.d(61): Error: constructor lists.FixedList!(int).FixedList.this called with argument types:
((int) shared)
matches both:
lists.d(28): lists.FixedList!(int).FixedList.this(ulong capacity)
and:
lists.d(37): lists.FixedList!(int).FixedList.this(ulong capacity)
src/app.d(61): Error: no constructor for FixedList
src/app.d(62): Error: constructor lists.FixedList!(int).FixedList.this called with argument types:
((int))
matches both:
lists.d(28): lists.FixedList!(int).FixedList.this(ulong capacity)
and:
lists.d(37): lists.FixedList!(int).FixedList.this(ulong capacity)
src/app.d(62): Error: no constructor for FixedList
make: *** [all] Error 1
Basically writeln function is preventing the error. Actually writeln is preventing in many places and I am not sure about why this is happening.
I even tried to compile the the code with m32 flag for 32-bit, but it is still same. Am I doing something wrong, or is this a bug?
pure, nothrow, and #safe are inferred for template functions. As FixedList is templated, its constructors are templated. writeln is not (and cannot be) pure as it does I/O. So, while writeln is in the constructors, they are inferred to not be pure, but everything else that the constructors are doing is pure, so without the calls to writeln, they become pure.
Under some circumstances, the compiler is able to alter the return type of pure functions to implicitly convert it to immutable or shared. This works, because in those cases, the compiler knows that what's being returned is a new, unique object and that casting it to immutable or shared would not violate the type system. Not all pure functions qualify, as the parameter types can affect whether the compiler can guarantee that the return value is unique, but many pure functions are able to take advantage of this and implicitly convert their return value to immutable or shared. This is useful, because it can avoid code duplication (for different return types) or copying - since if the type returned doesn't match what you need with regards to immutable or shared, and you can't guarantee that it's not referred to elsewhere, you have to copy it to get the type that you want. In this case, the compiler is able to make the guarantee that the object is not referred to elsewhere, so it can safely cast it for you.
Constructors effectively return new values, so they can be affected by this feature. This makes it so that if a constructor is pure, you can often construct immutable and shared values from it without having to duplicate the constructor (like you'd have to do if it weren't pure). As with other pure functions, whether this works or not depends on the constructor's parameter types, but it's frequently possible, and it helps avoid code duplication.
What's causing you problems is that when FixedList's constructors are both pure, the compiler is able to use either of them to construct a shared object. So, it doesn't know which one to choose, and gives you an ambiguity error.
I've reported this as a bug on the theory that the compiler should probably prefer the constructer which is explicitly marked as shared, but what the compiler devs will decide, I don't know. The ability to implicitly convert return values from pure functions is a fairly new feature and exactly when we can and can't do those implicit conversions is still being explored, which can result both in unanticipated problems (like this one probably is) as well as compiler bugs (e.g. there's at least one case with immutable, where it currently does the conversion when it shouldn't). I'm sure that these issues will be ironed out fairly quickly though.
A pure constructor can build a shared object without being marked shared itself.
Apparently, pureness is inferred for constructors.
writeln is not pure. So, with it in place, the constructors are not pure.
When writeln is removed, the constructors become pure. Both constructors now match the shared call.
int atClass1::read_file
(String^ file_path, /* Path tofile */
HdfCallVars % ret_vals)
This is my function. Within it I have lots of native c++ code. I ran into a serious problem though
/* Iterate through the links, filling in needed data as discovered. */
io_err = H5Literate (group_id, H5_INDEX_NAME, H5_ITER_NATIVE,
&i, get_sonar_data, (void*)& ret_vals);
will not compile! Says ret_vals is managed and I can't do pointerey ampersandey stuff to it. Am I in trouble? Or is there a way out of my dilemma? The H5 function is a call into the HDF5 library.
thanks,
saroj
In .Net there is no guarantee that an object will remain on it current memory position as the garbage collector will "compact" the heap space when it wants to.
To get a native pointer to a managed object you should "pin" the object:
pin_ptr<HdfCallVars> pinned = &ret_vals;
io_err = H5Literate (group_id, H5_INDEX_NAME, H5_ITER_NATIVE,
&i, get_sonar_data, (void*)pinned);
Note that the pointer will be unpinned after the variable pinned goes out of scope, if H5Literate stores the pointer for future use, you should pin the value using System::Runtime::InteropServices::GCHandle, like this:
GCHandle ^handle = GCHandle::Alloc(ret_vals);
io_err = H5Literate (group_id, H5_INDEX_NAME, H5_ITER_NATIVE,
&i, get_sonar_data, (void*)handle->AddrOfPinnedObject());
When you don't need the pointer anymore, you should free it:
handle->Free();
I'm confused about COM string assignments. Which of the following string assignment is correct. Why?
CComBSTR str;
.
.
Obj->str = L"" //Option1
OR should it be
Obj->str = CComBSTR(L"") //Option2
What is the reason
A real BSTR is:
temporarily allocated from the COM heap (via SysAllocString() and family)
a data structure in which the string data is preceded by its length, stored in a 32-bit value.
passed as a pointer to the fifth byte of that data structure, where the string data resides.
See the documentation:
MSDN: BSTR
Most functions which accept a BSTR will not crash when passed a BSTR created the simple assignment. This leads to confusion as people observe what seems to be working code from which they infer that a BSTR can be initialized just like any WCHAR *. That inference is incorrect.
Only real BSTRs can be passed to OLE Automation interfaces.
By using the CComBSTR() constructor, which calls SysAllocString(), your code will create a real BSTR. The CComBSTR() destructor will take care of returning the allocated storage to the system via SysFreeString().
If you pass the CComBSTR() to an API which takes ownership, be sure to call the .Detach() method to ensure the BSTR is not freed. BSTRs are not reference counted (unlike COM objects, which are), and therefore an attempt to free a BSTR more than once will crash.
If you use str = CComBSTR(L"") you use the constructor:
CComBSTR( LPCSTR pSrc );
If you use str = L"" you use the assignment operator:
CComBSTR& operator =(LPCSTR pSrc);
They both would initialize the CComBSTR object correctly.
Personally, I'd prefer option 1, because that doesn't require constructing a new CComBSTR object. (Whether their code does so behind the scenes is a different story, of course.)
Option 1 is preferred because it only does one allocation for the string where as option 2 does 2 (not withstanding the creation of a new temporary object for no particular reason). Unlike the bstr_t type in VC++ the ATL one does not do referenced counted strings so it will copy the entire string across.