unfortunately I cannot resort to C# in my current project, so I'll have to solve this without the unsafe keyword.
I've got a bitmap, and I need to access the pixels and channel values directly. I'd like to go beyond Marshal.ReadByte() and Marshal.WriteByte() (and definitely beyond GetPixel and SetPixel).
Is there a way to put all the pixel data of the bitmap into a Byte array that works on both 32 and 64 bit systems? I want the exact same layout as the original bitmap, so the padding for each row (if it exists) also needs to be included.
Marshal doesn't seem to have something akin to:
byte[] ReadBytes(IntPtr start, int offset, int count)
Unless I totally missed it...
Any help greatly appreciated,
David
ps. So far all my images are in 32BppPArgb pixelformat.
Marshal does have a Method that does exactly what you are asking. See Marshall.Copy()
public static void Copy(
IntPtr source,
byte[] destination,
int startIndex,
int length
)
Copies data from an unmanaged memory
pointer to a managed 8-bit unsigned
integer array.
And there are overloads to go the other direction as well
Would something like this do? (untested):
Public Shared Function BytesFromBitmap(ByVal Image As Drawing.Bitmap) As Byte()
Using buffer As New IO.MemoryStream()
image.Save(result, Drawing.Imaging.ImageFormat.Bmp)
Using rdr As New IO.BinaryReader(buffer)
Return rdr.ReadBytes(buffer.Length)
End Using
End Using
End Function
It won't let you manipulate the pixels in a Drawing.Bitmap object directly, but it will let you copy that bitmap to a byte array, as per the question title.
Another option is serialization via the BinaryFormatter, but I think that will still require you to pass it through a MemoryStream.
VB does not offer methods for direct memory access. You have two choices:
Use the Marshal class
Write a small unsafe C# (or C++/CLI) library that handles only these operations and reference it from your VB code.
Alright, there is a third option. VB.Net does not inherently support direct memory access, but it can be accomplished. It's just ugly and prone to errors. Nonetheless, if you're willing to put in the effort, you can try building a bitmap access library using these techniques combined with the approach referenced previously.
shf301 is right on the money, but I'd like to add a link to a comprehensive explanation/tutorial on fast pixel data access. Rather than saving the image to a stream and accessing a file-in-memory, it would be better to lock the bitmap, copy pixel data out, access it, and copy it back in. The performance of this technique is pretty good.
Code is in c#, but the approach is language-neutral and easy to read.
http://ilab.ahemm.org/tutBitmap.html
Related
I read the official documentation for the StringToHGlobalUni method and it says that the FreeHGlobal method should be use to free memory.
The memory allocated by StringToHGlobalUni method is on the native heap, so I don't see why the delete operator could not be used.
I searched a lot, but I couldn't find any explanation why I can't use the delete operator.
I'm new to this and some explanations would help me. Can the delete operator be used or not ?
Code
const wchar_t* filePath = (const wchar_t*)(Marshal::StringToHGlobalUni(inputFilePath)).ToPointer();
Marshal::FreeHGlobal(IntPtr((void*)filePath));
Don't use Marshal::StringToHGlobalUni in C++/CLI.
Use either
PtrToStringChars (accesses Unicode characters in-place, no allocation) or
marshal_as<std::wstring> (manages the allocation with a smart pointer class that will free it correctly and automatically)
Apart from the fact that StringToHGlobalUni requires you to free the memory manually, the name of that function is completely misleading. It has absolutely no connection to HGLOBAL whatsoever.
How to transfer this code from C# to VB.net?
This is from C#
if (local > 0)
{ //Local patches set offsets to data located elsewhere in this section
IntPtr start = data + section->_localPatchesOffset;
LocalPatch* patch = (LocalPatch*)start;
while ((int)patch - (int)start < local && patch->_dataOffset >= 0)
{ //Make the pointer offset relative to itself so it's self-contained
int ptrOffset = patch->_pointerOffset;
int* ptr = (int*)(data + ptrOffset);
*ptr = patch->_dataOffset - ptrOffset;
patch++;
}
}
Or I have this sample from C:
What exactly does it mean (byte*) in C
For example I have some byte array (like MemoryStream loaded from hole file....)
Dim arr As Byte() = New Byte() {1, 2, 33, 4, 55, 6, 7, 8, 9, 10, 10, 114, .....}
For example 33 is (at fromoffset 3) and 114 is (at tooffset 12)
This is not I think... Arr(fromoffset) = Arr(tooffset)
This is from C:
*(byte**)(SectionStart + LF->fromOffset) = SectionStart + LF->toOffset;
As jmc points out, you won't be able to convert that code to vb then compile it. I think your only option will be to make a c# DLL project, put that code into it (with any extra code it needs to function) as a method and then compile the c#
After you've done this you can reference the DLL, or you can even add the C# project to your solution continuing your VB item, and reference the project. The whole lot will compile and even the debugger will happily step from your VB.NET code into the c# and back again; these are not two separate languages when compiled - you could think of each language as merely being a whole lot of syntactic sugar for IL; but as VB simply doesn't have the sugar for unsafe stuff you'll have to do it in C#
The (byte*) is a pointer to a function with a parameter that is a pointer to a pointer of type byte.
Regarding unsafe pointers, the following is an excerpt from my Enhancing Visual Basic Book:
A Note on Recovering VarPtr, ObjPtr, StrPtr, VarPtrArray, and VarPtrStringArray.
If upgraded VB6 code required the “undocumented” VB6 functions VarPtr, ObjPtr, StrPtr, VarPtrArray, or VarPtrStringArray, use this one VB.NET function to duplicate all of them:
Imports System.Runtime.InteropServices
'--------------------------------------------------------------------------------------
' This Module Provides the memory addresses of Numeric Variables, Objects, and Strings.
'--------------------------------------------------------------------------------------
Module modVarPtr
'VB.NET version of VB6 VarPtr, ObjPtr, StrPtr, etc. (ALL of them are now supported by this one function).
'----------------------------------------------------------------------------------------------------------------------
'NOTES: Strings are not BSTR, so this returns the text address, not a BSTR pointer to the address pointing to it. This
' provides C# 'Unsafe Pointers' for VB.NET! Use the returned address right away before the Garbage Collector
' tries to change it! If we must hold it a short time, do not execute the GC.Free() instruction until done.
' We can break this code out in-line, using GC.Free() once we are done, but do not go on Break in the meantime.
' Garbage Collection will be held up until we GC.Free(), until then preventing the release of unused objects.
'----------------------------------------------------------------------------------------------------------------------
Public Function VarPtr(ByVal o As Object) As IntPtr 'Object 'catch all' (Return can also be 'As Integer').
Dim GC As GCHandle = GCHandle.Alloc(o, GCHandleType.Pinned) 'Get a trackable handle and pin (lock) the o address.
VarPtr = GC.AddrOfPinnedObject 'Get an IntPtr to the pinned o (var's data address).
GC.Free() 'free the allocated space used and unlock o address.
End Function 'Be aware IntPtr is 32-bit only on x86 compiles, otherwise it is 64-bit on 64-bit systems.
End Module
I am amaze so many VB6 users demanded a full VB OOPL environment with absolutely no sacrifice of OOPL functionality or data safety in the then-proposed VB.NET in 1999, with fist-shaking threats to abandon VB if their demands were not met. Yet, when they got exactly that, they had the gall to whine about the loss of the unsafe VarPtr, which did not jive with OOPL or data safety, but threatened user-demanded specifications. What was the result? They again threatened to abandon VB. There is a funny anecdote to describe such people, but I doubt they would appreciate it. Me? I love making fun of myself. I am a geek, and my jokes about geeks are merciless!
Finally, notice VB.NET Varptr again allows us to directly copy structure object data using the CopyMemory Pinvoke. However, even though we can grab class object memory addresses as well, we cannot use CopyMemory to copy data from or to it due to violating protected memory. Although some creative developers have managed to achieve this using intermediate memory areas, we can actually bypass employing intermediate buffers and use the .NET methods they recommend, the StructToPtr and PtrToStructure marshalling methods, to actually copy data directly to and from our local memory data and class objects. Even so, because these Unsafe Pointers are an allowed extension of OOPL rules, Microsoft should consider adding unsafe pointers to VB.NET.
So I am working on an Excel Project that is going to load a C++ dll using VBA. What I'd like to do is to be able to pass an Excel range with no specific type (data can be numerical or categorical) to the C++ dll (The best way I can describe my Excel range is of the type variant).
So the steps probably involve:
Load the dll in VBA
Send the excel range to dll (The range may contain columns of numbers and/or columns of strings)
manipulate the data from excel in the dll file
I am thinking of using excel variant and C++ variant. But it's not clear for me how to use the C++ variant as I couldn't find any good documentations on it.
Another suggestion I received was to ues COM programming.
My Questions:
Could a kind soul possibly provide pointers for me on how to proceed? (e.g. by providing the C++ prototype, and a simple example of how to handle the variant)
Does anyone know any good documentation/tutorial on using C++ Variants (and perhaps jointly with VBA)?
Is using COMs preferable to using VARIANTS if speed is an issue?
Is using the C API an option?
UPDATE:
The size of the ranges I need to manipulate can be large (~ 500,000 rows).
Speed is a factor, thus, I'd like to avoid unnecessary copying as much as possible.
Provided you only want to pass data to the dll (and not pointers to actual Excel objects such as Range), you have two fundamental options:
You have huge data sets and want to avoid copying as much as possible.
In this case you might want to pass that same Variant array you get by calling Range.Value. In order to do that, you will have to write a little TLB to reference from VB, in which you would describe your exported C++ function as expecting a SAFEARRAY(VARIANT)*. This is because the Declare operator will not let you actually pass a SAFEARRAY*.
The function will look like this:
LONG __stdcall ReturnArrLowerBound(SAFEARRAY** ppArr)
{
if (ppArr == NULL) return -1;
SAFEARRAY* pArr = (*ppArr);
LONG res = 0;
SafeArrayGetLBound(pArr, 1, &res);
return res;
}
And the TLB descripion will look like that:
[
uuid(A686B138-D8CE-462e-AEF2-75DA4DBF1C75)
]
library foo
{
[
dllname("TestSafearray.dll")
]
module vb
{
[entry("ReturnArrLowerBound")]
LONG __stdcall ReturnArrLowerBound(SAFEARRAY(VARIANT)* ppArr);
}
}
And your C++ project will obviously include a def file:
LIBRARY "TestSafearray"
EXPORTS
ReturnArrLowerBound
Your data sets are reasonably sized and you don't mind a little bit of copying.
Then make your C++ function to accept a mere int[] and Declare it in VB as accepting arr() as Long. On VB side, allocate an array on Longs and copy the elements into it from the Range.Value array.
I'm implementing a dynamic language that will compile to C#, and it's implementing its own reflection API (.NET's is too slow, and the DLR is limited only to more recent and resourceful implementations).
For this, I've implemented a simple .GetField(string f) and .SetField(string f, object val) interface. Until recently, the implementation just switches over all possible field string values and makes the corresponding action.
Also, this dynamic language has the possibility to define anonymous objects. For those anonymous objects, at first, I had implemented a simple hash algorithm.
By now, I am looking for ways to optimize the dynamic parts of the language, and I have come across the fact that a hash algorithm for anonymous objects would be overkill. This is because the objects are usually small. I'd say the objects contain 2 or 3 fields, normally. Very rarely, they would contain more than 15 fields. It would take more time to actually hash the string and perform the lookup than if I would test for equality between them all. (This is not tested, just theoretical).
The first thing I did was to -- at compile-time -- create a red-black tree for each anonymous object declaration and have it laid onto an array so that the object can look for it in a very optimized way.
I am still divided, though, if that's the best way to do this. I could go for a perfect hashing function. Even more radically, I'm thinking about dropping the need for strings and actually work with a struct of 2 longs.
Those two longs will be encoded to support 10 chars (A-za-z0-9_) each, which is mostly a good prediction of the size of the fields. For fields larger than this, a special function (slower) receiving a string will also be provided.
The result will be that strings will be inlined (not references), and their comparisons will be as cheap as a long comparison.
Anyway, it's a little hard to find good information about this kind of optimization, since this is normally thought on a vm-level, not a static language compilation implementation.
Does anyone have any thoughts or tips about the best data structure to handle dynamic calls?
Edit:
For now, I'm really going with the string as long representation and a linear binary tree lookup.
I don't know if this is helpful, but I'll chuck it out in case;
If this is compiling to C#, do you know the complete list of fields at compile time? So as an idea, if your code reads
// dynamic
myObject.foo = "some value";
myObject.bar = 32;
then during the parse, your symbol table can build an int for each field name;
// parsing code
symbols[0] == "foo"
symbols[1] == "bar"
then generate code using arrays or lists;
// generated c#
runtimeObject[0] = "some value"; // assign myobject.foo
runtimeObject[1] = 32; // assign myobject.bar
and build up reflection as a separate array;
runtimeObject.FieldNames[0] == "foo"; // Dictionary<int, string>
runtimeObject.FieldIds["foo"] === 0; // Dictionary<string, int>
As I say, thrown out in the hope it'll be useful. No idea if it will!
Since you are likely to be using the same field and method names repeatedly, something like string interning would work well to quickly generate keys for your hash tables. It would also make string equality comparisons constant-time.
For such a small data set (expected upper bounds of 15) I think almost any hashing will be more expensive then a tree or even a list lookup, but that is really dependent on your hashing algorithm.
If you want to use a dictionary/hash then you'll need to make sure the objects you use for the key return a hash code quickly (perhaps a single constant hash code that's built once). If you can prevent collisions inside of an object (sounds pretty doable) then you'll gain the speed and scalability (well for any realistic object/class size) of a hash table.
Something that comes to mind is Ruby's symbols and message passing. I believe Ruby's symbols act as a constant to just a memory reference. So comparison is constant, they are very lite, and you can use symbols like variables (I'm a little hazy on this and don't have a Ruby interpreter on this machine). Ruby's method "calling" really turns into message passing. Something like: obj.func(arg) turns into obj.send(:func, arg) (":func" is the symbol). I would imagine that symbol makes looking up the message handler (as I'll call it) inside the object pretty efficient since it's hash code most likely doesn't need to be calculated like most objects.
Perhaps something similar could be done in .NET.
What is the most similar thing in VB.NET to a pointer, meaning like C pointers?
I have a TreeView within a class. I need to expose some specific nodes (or leaves) that can be modified by external objects.
C#, and I also believe VB.Net, will work on the concept of references. Essentially, it means when you say
A a = new A()
the 'a' is a reference, and not the actual object.
So if I go
B b = a
b is another reference to the same underlying object.
When you want to expose any internal objects, you can simply do so by exposing 'properties'. Make sure, that you do not provide setters for the properties, or that if you do, there is code to check if the value is legal.
ByRef is used when you want to pass the object as a parameter, and when you want the called method to be able to change the reference (as opposed to the object).
As mentioned above, if you post some code, it will be easier to explain.
Nathan W has already suggested the IntPtr structure which can represent a pointer or handle, however, whilst this structure is part and parcel of the .NET framework, .NET really doesn't have pointers per-say, and certainly not like C pointers.
This is primarily because the .NET Framework is a "managed" platform and memory is managed, assigned, allocated and deallocated by the CLR without you, the developer, having to worry about it (i.e. no malloc commands!) It's mostly because of this memory management that you don't really have access to direct memory addresses.
The closest thing within the .NET Framework that can be thought of as a "pointer" (but really isn't one) is the delegate. You can think of a delegate as a "function pointer", however, it's not really a pointer in the strictest sense. Delegates add type-safety to calling functions, allowing code that "invokes" a delegate instance to ensure that it is calling the correct method with the correct parameters. This is unlike "traditional" pointers as they are not type-safe, and merely reference a memory address.
Delegates are everywhere in the .NET Framework, and whenever you use an event, or respond to an event, you're using delegates.
If you want to use C# rather than VB.NET, you can write code that is marked as "unsafe". This allows code within the unsafe block to run outside of the protection of the CLR. This, in turn, allows usage of "real" pointers, just like C, however, they still do have some limitations (such as what can be at the memory address that is pointed to).
Best way to do it is to just allocate everything manually:
You can move up OR down each Stack at free will without Pushing or Popping.
Dim Stack(4095) as Byte 'for 8bit - 1 bytes each entry
Dim Stack(4095) as Integer 'for 16bit - 2 bytes each entry
Dim Stack(4095) as Long 'for 32bit - 4 bytes each entry
Dim Stack(4095) as Double 'for 64 bit - 8 bytes each entry
Dim i as integer 'Where i is your Stack Pointer(0 through 4095)
Dim int as integer 'Byte Integer Long or Double (8, 16, 32, 64 bit)
for i = 0 to 4095
int = i
Stack(i) = int/256 'For 8bit Byte
Stack(i) = int 'For 16bit Integer
Stack(i) = Microsoft.VisualBasic.MKL$(int) 'For 32bit Long
Stack(i) = Microsoft.VisualBasic.MKD$(int) 'For 64bit Double
MsgBox(Microsoft.VisualBasic.HEX$(Stack(i))) 'To See Bitwise Length Per Entry
next i
If you're looking to pass something back from a subroutine you can pass it by reference - as in "ByRef myParamter as Object".
Best answer depends, to some degree, on what you're trying to do.
If you are using VB the only thing that is really close to a pointer is a IntPtr. If you have access to C# you can use unsafe C# code to do pointer work.