I'm in a C++/CLI project, and I have a byte* variable that I want to fully convert into a managed array<byte> as efficiently as possible.
Currently, the only way that I've seen is to manually create the managed array<byte> object, and then copy individual bytes from the byte* variable, as shown below:
void Foo(byte* source, int bytesCount)
{
auto buffer = gcnew array<byte>(bytesCount);
for (int i = 0; i < bytesCount; ++i)
{
buffer[i] = source[i];
}
}
Is there any other way to do this more efficiently? Ideally, to not have to copy the memory at all.
If not, is there any way to do this more cleanly?
You can't create a managed array from an unmanaged buffer without copying.
However, you don't need to copy the individual bytes in a loop, though. You can pin the managed array (see pin_ptr) and then copy the bytes directly from the unmanaged buffer into the memory of the managed array, such as with memcpy() or equivalent.
void Foo(byte* source, int bytesCount)
{
auto buffer = gcnew array<byte>(bytesCount);
{
pin_ptr<byte> p = &buffer[0];
byte *cp = p;
memcpy(cp, source, bytesCount);
}
// use buffer as needed...
}
Related
I have array of objects holding primitive types and enums; how do I marshal a pointer to this data to a native function with the signature native_func(void* ptr[]).
array<System::Object^>^ values = gcnew array<System::Object>(64);
// ... populate the managed array with primitives ...
// data is pinned and won't be moved by the GC
pin_ptr<object> pinned = &values[0];
// not sure what do here... data is corrupted in the native code
native_func((void**)pinned);
Thanks!
EDIT. My second attempt was to do the following:
pin_ptr<object> pinned = &values[0];
void* testArray[64];
for (auto i = 0; i < values->Length; i++)
{
testArray[i] = (void*)Marshal::UnsafeAddrOfPinnedArrayElement(values, i);
}
native_func(testArray);
Now, the addresses stored in testArray are getting passed correctly to the native side but the contents of the memory is not what I am expecting. What am I doing wrong now?
Enums are not blittable so marshaling an array of objects require special consideration (i.e. you can't just pin_ptr the array and pass it over the native/managed boundary). I decided to use a VARIANT to hold the primitive & enum values and did so with the following code:
// allocate a managed array of size 64 (it's enough for my application)
array<System::Object^>^ values = gcnew array<System::Object>(64);
// stack allocate a native array of size 64
VARIANT nativeValueArray[64] = {};
// ... populate the managed array ...
for (auto i = 0; i < values->Length; i++)
{
Marshal::GetNativeVariantForObject(values[i], (IntPtr)(void*)&nativeValueArray[i]);
}
// pass the array of native VARIANTS to the native function "native_function"
native_function(nativeValueArray);
The native function's signature became
void native_function(VARIANT values[]);
There might be a more efficient way to do this but this is what I was able to come up with. Please let me know if you have a more efficient way to accomplish what am I doing.
I'm trying to use FreeRTOS to write ADC data to SD card on the STM32F7 and I'm using V1 of the CMSIS-RTOS API. I'm using mail queues and I have a struct that holds an array.
typedef struct
{
uint16_t data[2048];
} ADC_DATA;
on the ADC half/Full complete interrupts, I add the data to the queue and I have a consumer task that writes this data to the sd card. My issue is in my Consumer Task, I have to do a memcpy to another array and then write the contents of that array to the sd card.
void vConsumer(void const * argument)
{
ADC_DATA *rx_data;
for(;;)
{
writeEvent = osMailGet(adcDataMailId, osWaitForever);
if(writeEvent.status == osEventMail)
{
// write Data to SD
rx_data = writeEvent.value.p;
memcpy(sd_buff, rx_data->data, sizeof(sd_buff));
if(wav_write_result == FR_OK)
{
if( f_write(&wavFile, (uint8_t *)sd_buff, SD_WRITE_BUF_SIZE, (void*)&bytes_written) == FR_OK)
{
file_size+=bytes_written;
}
}
osMailFree(adcDataMailId, rx_data);
}
}
This works as intended but if I try to change this line to
f_write(&wavFile, (uint8_t *)rx_data->data, SD_WRITE_BUF_SIZE, (void*)&bytes_written) == FR_OK)
so as to get rid of the memcpy, f_write returns FR_DISK_ERR. Can anyone help shine a light on why this happens, I feel like the extra memcpy is useless and you should just be able to pass the pointer to the queue straight to f_write.
So just a few thoughts here:
memcpy
Usually I copy only the necessary amount of data. If I have the size of the actual data I'll add a boundary check and pass it to memcpy.
Your problem
I am just guessing here, but if you check the struct definition, the data field has the type uint16_t and you cast it to a byte pointer. Also the FatFs documentation expects a void* for the type of buf.
EDIT: Could you post more details of sd_buff
I am newbie of C++/CLI.
I already know that the pin_ptr's functionality is making GC not to learn to specified object.
now let me show you msdn's example.
https://msdn.microsoft.com/en-us//library/1dz8byfh.aspx
// pin_ptr_1.cpp
// compile with: /clr
using namespace System;
#define SIZE 10
#pragma unmanaged
// native function that initializes an array
void native_function(int* p) {
for(int i = 0 ; i < 10 ; i++)
p[i] = i;
}
#pragma managed
public ref class A {
private:
array<int>^ arr; // CLR integer array
public:
A() {
arr = gcnew array<int>(SIZE);
}
void load() {
pin_ptr<int> p = &arr[0]; // pin pointer to first element in arr
int* np = p; // pointer to the first element in arr
native_function(np); // pass pointer to native function
}
int sum() {
int total = 0;
for (int i = 0 ; i < SIZE ; i++)
total += arr[i];
return total;
}
};
int main() {
A^ a = gcnew A;
a->load(); // initialize managed array using the native function
Console::WriteLine(a->sum());
}
hear is the question.
Isn't it okay, the passed object(arr) not pinned ?
because the unmanaged code(native_function) is sync operation and finished before the C++/CLI code (load)
is there any chance the gc destory arr, even though the main logic is running?
(I think A is main's stack variable and arr is A's member variable, so while running main, it should visible)
if so, how can we guarantee that the A is there before invoking load?
(only while not running in native-code?)
int main() {
A^ a = gcnew A;
// I Think A or arr can be destroyed in here, if it is able to be destroyed in native_function.
a->load();
...
}
Thanks, in advance.
The problem that is solved by pinning a pointer is not a normal concurrency issue. It might be true that no other thread will preempt the execution of your native function. However, you have to count in the garbage collector, which might kick in whenever the .NET runtime sees fit. For instance, the system might run low on memory, so the runtime decides to collect disposed objects. This might happen while your native function executes, and the garbage collector might relocate the array it is using, so the pointer you passed in isn't valid anymore.
The golden rule of thumb is to pin ALL array pointers and ALL string pointers before passing them to a native function. ALWAYS. Don't think about it, just do it as a rule. Your code might work fine for a long time without pinning, but one day bad luck will strike you just when it's most annoying.
I have a System::String^ variable in C++ code. This variable should be converted to std::string which is later converted to const char* via c_str.
// original string
System::String^ path = ...;
// convert to std::string
msclr::interop::marshal_context context;
std::string filename(context.marshal_as<std::string>(path));
// call API function that internally connects to sqlite3 using sqlite3_open as
// sqlite3_open(filename.c_str())
// https://www.sqlite.org/c3ref/open.html -
// const char *filename, /* Database filename (UTF-8) */
doCalculation(filename)
It works well with the ASCII paths, but fails if the path contains non-latin characters.
So Somehow I need to convert marshalled std::string from current implementation (ASCII?) to UTF8.
I've tried
std::wstring dbPath(context.marshal_as<std::wstring>(path));
std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>, wchar_t> convert;
std::string dbPathU8 = convert.to_bytes(dbPath);
but it does not work.
What you want to do is use the .Net methods to convert directly to UTF-8.
The available methods in the Encoding class aren't exactly what you're looking for (direct from managed String to unmanaged string or byte array), so we'll need an intermediary and some manual copying.
String^ path = ...;
// First, convert to a managed array of the bytes you want.
array<Byte>^ bytes = Encoding::UTF8->GetBytes(path);
// Then, copy those bytes from the managed byte array to an unmanaged string.
std::string str;
str.resize(bytes->Length);
Marshal::Copy(bytes, 0, IntPtr(str.data()), bytes->Length);
// OR, copy directly to the char* you want eventually.
char* chars = new char[bytes->Length + 1]; // or malloc(), or whatever.
Marshal::Copy(bytes, 0, IntPtr(chars), bytes->Length);
chars[bytes->Length] = '\0'; // null terminate.
// don't forget to free the buffer when you're done with it!
There are several GetBytes variants available, but the parameters to them seem to either be both managed, or both unmanaged. (String^ and array^, or char* and byte*, but not String^ and byte*.) Therefore, we have the Encoding class create a managed byte array, then we use the Marshal::Copy method to copy those bytes either to the unmanaged string object, or directly to a char*.
When I return a direct ByteBuffer to JNI, how long until it can get reclaimed by the JVM/GC?
Suppose I have a function like this:
void* func()
{
[ ... ]
jobject result = env->CallStaticObjectMethod(testClass, doSomethingMethod);
void* pointerToMemory = env->GetDirectBufferAddress(result);
return pointerToMemory;
}
The JVM can't possibly know how long I'm going to use that pointerToMemory, right? What if I want to hold on to that address and the corresponding memory for a while?
Suppose I want to circumvent this issue and return a byte[] from Java to JNI like this:
ByteBuffer buf;
byte[] b = new byte[1000];
buf = ByteBuffer.wrap(b);
buf.order(ByteOrder.BIG_ENDIAN);
return buf.array();
AND THEN do the same as above, I store a pointer to that byte[] and want to hold on to it for a while. How / when / why is the JVM going to come after that backing byte[] from Java?
void* function()
{
jbyteArray byteArr = (jbytearray)env->CallStaticObjectMethod(testClass, doSomethingMethod);
jbyte *b= env->GetByteArrayElements(byteArr, 0);
return b;
}
The short answer is: If the function implements a native method, the pointer will be invalid as soon as you return.
To avoid this, you should get a global reference for all objects that you intend to keep valid after returning. See the documentation on local and global references for more information.
To understand better how JNI manages references from native code, see the documentation on PushLocalFrame/PopLocalFrame.