I'm looking for a quick way to serialize custom structures consisting of basic value types and strings.
Using C++CLI to pin the pointer of the structure instance and destination array and then memcpy the data over is working quite well for all the value types. However, if I include any reference types such as string then all I get is the reference address.
Expected as much since otherwise it would be impossible for the structure to have a fixed.. structure. I figured that maybe, if I make the string fixed size, it might place it inside the structure though. Adding < VBFixedString(256) > to the string declaration did not achieve that.
Is there anything else that would place the actual data inside the structure?
Pinning a managed object and memcpy-ing the content will never give you what you want. Any managed object, be it String, a character array, or anything else will show up as a reference, and you'll just get a memory location.
If I read between the lines, it sounds like you need to call some C or C++ (not C++/CLI) code, and pass it a C struct that looks similar to this:
struct UnmanagedFoo
{
int a_number;
char a_string[256];
};
If that's the case, then I'd solve this by setting up the automatic marshaling to handle this for you. Here's how you'd define that struct so that it marshals properly. (I'm using C# syntax here, but it should be an easy conversion to VB.net syntax.)
[StructLayout(LayoutKind.Sequential, CharSet=CharSet.Ansi)]
public struct ManagedFoo
{
public int a_number;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst=256)]
public string a_string;
}
Explanation:
StructLayout(LayoutKind.Sequential) specifies that the fields should be in the declared order. The default LayoutKind, Auto, allows the fields to be re-ordered if the compiler wants.
CharSet=CharSet.Ansi specifies the type of strings to marshal. You can specify CharSet.Ansi to get char strings on the C++ side, or CharSet.Unicode to get wchar_t strings in C++.
MarshalAs(UnmanagedType.ByValTStr) specifies a string inline to the struct, which is what you were asking about. There are several other string types, with different semantics, see the UnmanagedType page on MSDN for descriptions.
SizeConst=256 specifies the size of the character array. Note that this specifies the number of characters (or when doing arrays, number of array elements), not the number of bytes.
Now, these marshal attributes are an instruction to the built-in marshaler in .Net, which you can call directly from your VB.Net code. To use it, call Marshal.StructureToPtr to go from the .Net object to unmanaged memory, and Marshal.PtrToStructure to go from unmanaged memory to a .Net object. MSDN has some good examples of calling those two methods, take a look at the linked pages.
Wait, what about C++/CLI? Yes, you could use C++/CLI to marshal from the .Net object to a C struct. If your structs get too complex to represent with the MarshalAs attribute, it's highly appropriate to do that. In that case, here's what you do: Declare your .Net struct like I listed above, without the MarshalAs or StructLayout. Also declare the C struct, plain and ordinary, also as listed above. When you need to switch from one to the other, copy things field by field, not a big memcpy. Yes, all the fields that are basic types (integers, doubles, etc.) will be a repetitive output.a_number = input.a_number, but that's the proper way to do it.
Related
To my surprise this block
type Object *struct{
X int
}
compiles in golang. However, I don't know how to create an instance of the underlying struct.
Functionally, what I wanted to achieve is to remove all the stars from all type signatures without hacks (redefining the type and other tricks). This would make the type/structs very much like Java classes.
The question is - is this construction supported in golang? Or should I stick to putting stars everywhere?
If you don't want to pass pointers around everywhere, you don't have to. You could just pass your structs around by value.
E.g.
Define your struct as:
type Object struct{
X int
}
And then define your functions as:
func DoStuffToObject(obj Object) {
// Do things with obj here
}
There's nothing wrong with passing around objects by value if that's what you wish to do.
I use pin_ptr for cli::array types and everything works fine.
Is it possible to do the same with System::Collection::Generic::List which I believe is a contiguous block of memory?
The obvious
List<double>^ stuff = gcnew List<double>( 10 );
cli::pin_ptr<double> resultPtr = &stuff[ 0 ];
gives a compiler error "error C2102: '&' requires l-value" presumably because the indexed property returns something that is not a l-value! So is there another way to do this. I have played around with interior_ptr as well but have not found anything that works yet.
I know that I could call ToArray on the List but the whole point is to not copy stuff around.
No, this is not possible.
True, a List does use an array behind the scenes, but the [] operator is different. With an array, [] is simple pointer math, but with a List, [] is a full-fledged method call. That's why the & isn't working: you can take the address of an array location, but you can't take the address of a value returned from a method.
Think about it like this: If they wanted to, they could change the implementation of List without changing its external interface. It would be possible to change List to store the list contents in memory gzip-compressed. In that case, stuff[0] is generated on-the-fly by the [] method which does the decompression, so there is no single memory location that contains stuff[0] to pin.
Edit
Yes, internal to the List class, the contents are contiguous in memory. You can see this in the source that Microsoft has provided. However, the List class does not make that array public: The public interface to the List class is the public methods & properties, only. The public methods & properties present a contract, and the array that the values are stored in are not part of that contract. Microsoft would never do this, but they could do a gzip-compressed implementation of List, and the public contract of the List class wouldn't change. You should only write your code to the public methods & properties of a class, not to the internals that may change at any time.
I have a class call CalcArray that has an array of doubles called Amounts(), and two ints, StartPeriod and EndPeriod.
The user almost always wants to interact with the items in the array, not the Periods or the object itself. So ideally, I'd like:
property AnAmount() as CalcArray 'So the user can talk to the object if they need to
property AnAmount(i as Integer) as Double 'So the user can just get the value directly
This seems to work sometimes and not others. Is this simply a syntax issue? or is such an overload not possible?
You can do this with a function returning a different based on how it is called. Especially since you have a param, a function might be more appropriate:
Public Function AnAmount(Of T)(parm As SomeType) As T
to use it:
Dim n as Decimal
n = AnAmount(Of Decimal)(foo)
Its very useful as a way to avoid returning an object and then have to use CType to convert the return. In this case, an amount implies a value type, but the function would accept Point, Rectangle etc as T, so you might need to check valid type requests.
You may be bumping into the limitation that a function or property cannot vary by only the return type. In general if the signature has changed, the output type can change also on an overload. Look out also for the limitation for using default properties requires an argument. In some cases class inheritance is the issue, properties and functions being shadowed may explicitly be required to nominate Shadows, Overloads, Overrides etc. or the shadowing will be disallowed by the language.
If these don't cover the cases you've seen, try to catch an example of the problem and study all locations of the same named property in your solution, reporting the results here.
This question is merely out of interest and trying to understand something about memory management in object-oriented languages. It is not specific to one language, but I just want to understand as a general principle.
What I want to know is how the definition of an object reference is stored compared to the instance of that reference.
When you define and object in OO source code, e.g. in Java, without instantiating it:
String s;
How does this get stored? How does the memory usage of this definition differ from when the object is actually instantiated:
s = new String("abc");
? Is there a general principle that applies to all OO languages in terms of how memory is allocated or do different language implementers use different techniques for allocating memory?
Normaly when we declare a refrence like String s; it is created as a normal variable just like int , float but this type of variable hold the memory address ( it similar concept as pointers in C language) but when we use s = new String("abc");, it creates an object in heap and assign that address to the reference variable s.
In Java byte code, all Objects are stored as Objects. Explicit type-checking is added when needed. So for example this Java function
public Integer getValue(Object number){
int i = ((Number) number).toInt();
return new Integer(i);
}
is translated to a bytecode like this:
(accepts java.lang.Object, returns java.lang.Integer)
-read the first argument as an Object
-if the value is not a Number, raise an exception
-call the virtual method toInt(java.lang.Integer) of the value
and remember the int result
-use the value as an int argument
-instantiate a new java.lang.Integer
-call the constructor(int) of java.lang.Integer on the new number,
getting an Object back
[since the declared return value of Number.toInt is the same
as the return value of our function, no type checking is needed]
-return the value
So, types of unused variables get stripped out by the compiler. Types of public and protected fields are stored with its class.
The runtime type of an Object is stored with the object. In C++, it is a pointer to the Virtual Method Table. In Java, it is as a 16-bit index into the table of all loaded classes.
The Java class file stores an index of all dependent classes in a similar table. Only the class names are stored here. All field descriptions then point to this table.
So, when you write String s = new String("abc") (or even String s = "abc"), your class stores:
it is dependent on the class java.lang.String in the table of dependencies
"abc" in the table of String literals
your method loading a String literal by ID
(in the first case) your method calling a constructor of its first dependent class (String) with the first dependent class (String) as an argument.
the compiler can prove storing the new String in a String variable is safe, so it skips the type checking.
A class can be loaded as soon as it is referenced, or as late as its first use (in which case it is refered to by its depending class and ID within the class). I think the latter is always the case nowadays.
When a class is loaded:
-its class loader is asked to retreive the class by its name.
-(in the case of the system loader) the class loader looks
for the corresponding file in the program JAR, in the system library
and in all libraries referenced.
-the byte stream is then decoded into a structure in memory
-(in the case of early loading) all dependent classes are loaded recursively
if not already loaded
-it is stored in the class table
-(in the case of late loading) its static initialiser is run
(possibly loading more classes in the process).
In C++, none of the class loading takes place, as all user classes and most libraries are stored in the program as a mere virtual method table and the corresponding method. All of the system functions (not classes) can still be stored in a DLL (in case of Windows) or a similar file and loaded by the library at runtime. If a type checking is implied by an explicit type-cast, it is performed on the virtual method table. Also note that C++ did not have a type checking mechanism for a while.
I am returning a list of items (user defined class) in a REST service using WCF. I am returning the items as JSON and it is used in some client side javascript (so the 'schema' of the class was derived from what the javascript library required). The class is fairly basic, strings and a bool. The bool is optional, so if it is absent the javascript library uses a default value, and if it is present (true or false) the value is used.
The problem is if I use a bool, the value is defaulted to false when serialized, and if i use a bool?, the member is still sent accross in the JSON and defaulted to null which causes problems with the library (it wont fall back to the default value).
I know I can probably mess around the the javascript library, but I would like to find a way to just not send any members which are null so they dont show up in the serialized JSON at all.
Any ideas?
You could do a little bit of packing and unpacking before and after the serialization. E.g.:
You could make two different versions
of the class, one with and one
without the bool, and convert as
appropriate before and after
transmitting. (sends the least amount
of data, if # of bytes is a greater
consideration than code complexity)
You could add another bool that tells
whether the first bool is supposed to
be null.