How can I go about storing a vb.net user defined object in a sql database. I am not trying to replicate the properties with columns. I mean something along the lines of converting or encoding my object to a byte array and then storing that in a field in the db. Like when you store an instance of an object in session, but I need the info to persist past the current session.
#Orion Edwards
It's not a matter of stances. It's because one day, you will change your code. Then you will try de-serialize the old object, and YOUR PROGRAM WILL CRASH.
My Program will not "CRASH", it will throw an exception. Lucky for me .net has a whole set of classes dedicated for such an occasion. At which time I will refresh my stale data and put it back in the db. That is the point of this one field (or stance, as the case may be).
You can use serialization - it allows you to store your object at least in 3 forms: binary (suitable for BLOBs), XML (take advantage of MSSQL's XML data type) or just plain text (store in varchar or text column)
Before you head down this road towards your own eventual insanity, you should take a look at this (or one day repeat it):
http://thedailywtf.com/Articles/The-Mythical-Business-Layer.aspx
Persisting objects in a database is not a good idea. It kills all the good things that a database is designed to do.
You could use the BinaryFormatter class to serialize your object to a binary format, then save the resulting string in your database.
The XmlSerializer or the DataContractSerializer in .net 3.x will do the job for you.
#aku, lomaxx and bdukes - your solutions are what I was looking for.
#1800 INFORMATION - while i appreciate your stance on the matter, this is a special case of data that I get from a webservice that gets refreshed only about once a month. I dont need the data persisted in db form because thats what the webservice is for. Below is the code I finally got to work.
Serialize
#'res is my object to serialize
Dim xml_serializer As System.Xml.Serialization.XmlSerializer
Dim string_writer As New System.IO.StringWriter()
xml_serializer = New System.Xml.Serialization.XmlSerializer(res.GetType)
xml_serializer.Serialize(string_writer, res)
Deserialize
#'string_writer and xml_serializer from above
Dim serialization As String = string_writer.ToString
Dim string_reader As System.IO.StringReader
string_reader = New System.IO.StringReader(serialization)
Dim res2 As testsedie.EligibilityResponse
res2 = xml_serializer.Deserialize(string_reader)
What you want to do is called "Serializing" your object, and .Net has a few different ways to go about it. One is the XmlSerializer class in the System.Xml.Serialization namespace.
Another is in the System.Runtime.Serialization namespace. This has support for a SOAP formatter, a binary formatter, and a base class you can inherit from that all implement a common interface.
For what you are talking about, the BinaryFormatter suggested earlier will probably have the best performance.
I'm backing #1800 Information on this one.
Serializing objects for long-term storage is never a good idea
while i appreciate your stance on the matter, this is a special case of data that I get from a webservice that gets refreshed only about once a month.
It's not a matter of stances. It's because one day, you will change your code. Then you will try de-serialize the old object, and YOUR PROGRAM WILL CRASH.
If it crashes (or throws an exception) all you are left with is a bunch of binary data to try and sift through to recreate your objects.
If you are only persisting binary why not just save straight to disk. You also might want to look at using something like xml as, as has been mentioned, if you alter your object definition you may not be able to unserialise it without some hard work.
Related
I have a Android java project with quite complex datatypes including ArrayLists of other complex datatypes. I’m trying to store my data in a database using ORMLite (4.42).
I have major problems converting between ArrayLists and ForeignCollection, has anyone manage to do this?
I don’t want a continuous sync with the database, instead I have a save method that saves the entire structure and a read method that restores the entire method. After an object is restored from the database, the object is sent via interfaces where the receiver expects a ArrayLists and not orm lite ForeignCollections.
My approach is the following. For each ArrayList I have a complementary ForeignCollection. Before storing an object to the database, the content of the ArrayList is copied to the ForeignCollection, and when read back, vice versa.
When I read out a data structure from the database, I want the ensure that the entire structure is restored since later on, the object will be sent to users that are not aware about orm lite package, thus I set eager = true.
My complex datatype looks like this:
#DatabaseTable(tableName = "accounts")
public class Account {
#DatabaseField(generatedId = true)
private int id;
#ForeignCollectionField (eager = true, maxEagerLevel = 2)
private ForeignCollection<Order> ordersFoC;
private ArrayList<Order> ordersArL;
The Problem
Before storing the Account to the database, I tried to do the following. My hope was to copy the objects in the ArrayList to the ForeignCollection and the update the database with the Account object. The object myAccount has the same id as an object in the database so I expect an update of that object.
ordersFoC.addAll(orderArL);
myDao.createOrUpdate(myAccount);
No luck. The addAll method does directly write the orders to the database so I end up with duplicates of all orders since “addAll” adds and does not update any existing records.
So I tried to first remove all the orders from the database, but again with no luck:
ordersFoC.clear(); // Also tried: ordersFoC.removeAll(ordersFoC);
ordersFoC.addAll(orderArL);
myDao.createOrUpdate(myAccount);
The objects are not removed from the database. I read at groups.google.com (by Philip Fu) that this is a “known” problem with ormlite, that eager ForeignCollection does not play well with removeAll() or clear().
Is there a pattern that I have missed that allows me to work with objects that includes ArrayLists of complex types using the ormlite framework?
I've got my code working with ArrayLists ok. I declared my fields like this:
#ForeignCollectionField(eager = true, maxEagerForeignCollectionLevel = 99)
private Collection<PersonalNameStructure> personalnamestructures = new ArrayList<>();
and the getter as
public List<PersonalNameStructure> getPersonalNameStructures() {
return new ArrayList<>(personalnamestructures) ;
}
so it creates the ArrayList on the fly rather than trying to maintain it.
Obviously you may realise this already, but you also need to be careful around writing the objects as the eager setting affects reading foreign collections, it doesn't do anything for foreign fields or when writing a record.
I am looking for some guidance as to what is going on when using proto-buf net with obfuscation (Dotfuscator). One half of the project is a DLL and the other is an EXE elsewhere and using proto-buf NET they exchange data flawlessly. Until I obfuscate the DLL.
At that point P-BN fails without raising an exception, returning variously a 0 length byte array or a foreshortened one depending on what I have fiddled with. The class is fairly simple (VB):
<ProtoContract(Name:="DMailer")> _
Friend Class DMailer
Private _Lic As Cert
Private _Sys As Sys
Private _LList As List(Of LItem)
..
..
End Class
There are 3 props all decorated with ProtoMember to get/set the constituent class objects. Snipped for brevity.
Again, it works GREAT until I obfuscate the DLL. Then, Dotfuscator renames each of these to null, apparently since they are all Friend, and that seems to choke proto-buff. If I exempt the class from renaming (just the class name, not props/members), it seems to work again. It makes sense that P-BN would only be able to act on objects with a proper name, though when asked to serialize a null named object, it seems like an exception might be in order.
On the other hand, much of the charm of PB-N is supposed to be serialization independent of .NET names working from attributes - at least as I understand it. Yet in this case it only seems to work with classes with names. I tried using the Name qualifier or argument as shown above, to no avail - it apparently doesnt do what I thought it might.
So, I am curious if:
a) ...I have basically surmised the problem correctly
b) ...There is some other attribute or flag that might facilitate serializing
a null named object
c) ...if there are any other insights that would help.
If I exempt all 3 or 4 classes from Dotfuscator renaming (LList is not actually implemented yet, leaving DMailer, Cert and Sys), the DLL seems to work again - at least the output is the correct size. I can live with that, though obscured names would be better: Dotfuscator (CE) either exempts them or sets the names to Null - I cant seem to find a way to force them to be renamed.
Rather than exempt 3 or 4 classes from renaming, one alternative I am considering is to simply store the Serializer output for Cert and Sys as byte arrays or Base64 strings in DMailer instead of classes. Then have the receiver Deserialize each object individually. It is kind of nice to be able to unpack just one thing and have your toys right there as if by magic though.
(many)TIA
Interesting. I confess I have never tried this scenario, but if you can walk me through your process (or better: maybe provide a basic repro example with "run this, then this, then this: boom") I'll happily investigate.
Note: the Name on ProtoContract is mainly intended for GetProto() usage; it is not needed by the core serializer, and can be omitted to reduce your exposure. Also, protobuf-net isn't interested in fields unless those fields are decorated with the attributes, so that shouldn't be an issue.
However! there's probably a workaround here that should work now; you can pre-generate a static serialization dll; for example in a separate console exe (just as a tool; I really need to wrap this in a standalone utility!)
So if you create a console exe that references your unobfuscated library and protobuf-net.dll:
var model = RuntimeTypeModel.Create();
model.Add(typeof(DMailer), true); // true means "use the attributes etc"
// and other types needed, etc
model.Compile("MailSerializer", "MailSerializer.dll");
this should write MailSerializer.dll, which you can then reference from your main code (in addition to protobuf-net), and use:
var ser = new MailSerializer(); // our pre-genereated serializer
ser.Serialize(...); // etc
Then include MailSerializer.dll in your obfuscation payload.
(this is all v2 specific, btw)
If this doesn't work, I'll need to investigate the main issue, but I'm not an obfuscation expert so could do with your repro steps.
Since there were a few upticks of interest, here is what looks like will work:
a) No form of reflection will be able to get the list of properties for an obfuscated type.
I tried walking thru all the types to find the ones with ProtoContract on it, I could find them
but the property names are all changed to a,m, b, j, g.
I also tried Me.GetType.GetProperties with the same result.
You could implement a map from the output to indicate that Employee.FirstName is now a0.j, but distributing this defeats the purpose of obfuscation.
b) What does work to a degree is to exempt the class NAME from obfuscation. Since PB-N looks for the ProtoMember attributes to get the data, you CAN obfuscate the Property/Member names, just not the CLASS/type name. If the name is something like FederalReserveLogIn, your class/type has a bullseye on it.
I have had initial success doing the following:
1) Build a simple class to store a Property Token and value. Store everything as string using ConvertFromInvariantString. Taking a tip from PBN, I used an integer for the token:
<ProtoMember(propIndex.Foo)>
Property Foo As String
An enum helps tie everything together later. Store these in a Dictionary(Of T, NameValuePair)
2) add some accessors. these can perform the type conversions for you:
Public Sub Add(ByVal Key As T, ByVal value As Object)
If _col.ContainsKey(Key) Then
_col.Remove(Key)
End If
_col.Add(Key, New TValue(value))
End Sub
Public Function GetTItem(Of TT)(key As T) As TT
If _col.ContainsKey(key) Then
Return CType(_col(key).TValue, TT)
Else
Return Nothing
End If
End Function
T is whatever key type you wish to use. Integer results in the smallest output and still allows the subscribing code to use an Enum. But it could be String.
TT is the original type:
myFoo = props.GetTItem(Of Long)(propsEnum.Foo)
3) Expose the innerlist (dictionary) to PBN and bingo, all done.
Its also very easy to add converters for Point, Rectangle, Font, Size, Color and even bitmap.
HTH
I'm really sorry. This must seem like an incredibly stupid question, but unless I ask I'll never figure it out. I'm trying to write a program to read in a csv file in Visual Basic (tried and gave up on C#) and I asked a friend of mine who is much better at programming than I am. He said I should create a class to hold the data that I read in.
The problem is, I've never created a class before, not in VB, Java, or anything. I know all the terms associated with classes, I understand at a high level how classes work no problem. But I suck at the actual details of making one.
So here's what I did:
Public Class TsvData
Property fullDataSet() As Array
Get
Return ?????
End Get
Set(ByVal value As Array)
End Set
End Property
End Class
I got as far as the question marks and I'm stuck.
The class is going to hold a lot of data, so I made it an array. That could be a bad move. I don't know. All i know is that it can't be a String and it certainly can't be an Integer or a Float.
As for the Getter and Setter, the reason I put the question marks in is because I want to return the whole array there. The class will eventually have other properties which are basically permutations of the same data, but this is the full set that I will use when I want to save it out or something. Now I want to return the whole Array, but typing "Return fullDataSet()" doesn't seem like a good idea. I mean, the name of the property is "fullDataSet()." It will just make some kind of loop. But there is no other data to return.
Should I Dim yet another array inside the property, which already is an array, and return that instead?
Instead of writing your own class, you could get yourself familiar with the pre-defined class System.Data.DataTable and then use that for holding CSV data.
In the last few years that I've been programming, I've never actually used a multi-dimensional array, and I'd advise you not to use them, either. There's usually ways of achieving the same with a better data structure. For example, consider creating a class (let's call it CsvRecord) that holds only one record; that is, only one line from the CSV file. Then use any of the standard collection types from the System.Collections.Generic namespace (e.g. List(Of CsvRecord)) to hold the entire data (ie. all lines) in the CSV file. This effectively reduces the problem to, "How do I read in one line of CSV data?"
If you want to take suggestion #2 even further, do as cHao says and don't simply lay out the information you've read as a CsvRecord; instead, create an object that reflects the actual content. For example, if your CSV file contains product–price information, call your CSV record class ProductInfo or something more fitting.
If, however, you want to go on with your current approach, you will need a backing field for the property, as demonstrated by Philipp's answer. Your property then becomes a "façade" that only delegates to this backing field. This is not absolutely necessary: You could simply make the backing field Public and let the user of your class access it directly, though that is not considered a good practice.
Ideally, you ought to have a class representing the specific data you want to read in. Setting an entire array at once is asking for trouble; some programs that read {C,T}SV files will freak out if all rows don't have the same number of columns, which is exceedingly easy to do if you can set the data to be an array of arbitrary length.
If you're trying to represent arbitrary data, frankly, you'd do just as well to use a List(Of String). If it's meant to be a table, you could instead read in the first line and make it a list as above (let's call it "headers"), and then make each row a Dictionary(Of String, String). (Let's call each row "row", and the collection (a list of these dictionary objects) "rows".) Just read in the line, split it like you did the first, and say something like row(headers(column number)) = value for each column, and then stuff it into 'rows'.
Or, you could use the data classes (System.Data.DataTable and System.Data.DataSet would do wonders here).
Usually you use a private member to store the actual data:
Public Class TsvData
Private _fullDataSet As String()
Public Property FullDataSet() As String()
Get
Return _fullDataSet
End Get
Set(ByVal value As String())
_fullDataSet = value
End Set
End Property
Note that this is an instance of bad design since it couples a concept to a concrete representation and allows the clients of the class to modify the internals without any error checking. Returning a ReadOnlyCollection or some dedicated container would be better.
Ok so I'm trying to use the JavaScriptSerializer to work with this code. However it crashes when it reaches the last line;
Dim json As New String(sr.Serialize(dt))
I get this error message;
A circular reference was detected
while serializing an object of type
'System.Reflection.Module'.
I would really appreciate any insights that could help solve this problem.
Circular reference means that serialising the object would result in an infinite loop.
For example if you would try to serialize object "A" having a 1 to 1 reference to object "B".
Declare a class containg the data you want to serialize with JSON to solve this issue.
As hkda150 has already said, you could use a class specifically tailored for being serialized.
This will furthermore enable you to have foreign key values serialized instead of having related full objects serialized. Thus, if you are serializing object a which has a property a.SomeB of type B, then you will often want the ID of a.someB to be present in your webpage. Obviously I don't know enough to be able to say if this is relevant in your specific use case.
BTW, If you find yourself doing a lot of mapping between "business objects" and "objects meant for serialization" you may want to consider using AutoMapper.
I am learning linq to sql and I am trying to setup a lookup combo box in my web page. I was using a linq data source but I then moved my linqtosql code to it's own class library for my DAL dll (took out of app_code folder). So, I am converting the page code to be able to still have lookups driven now by a biz object.
Here's what I have done in my biz layer...
Public Class KosherTypes
Public Shared Function GetKosherTypes() As List(Of KosherTypeLookup)
Dim db As New DBDataContext
Dim types = From kt In db.KosherTypes _
Where kt.IsDeleted = False _
Order By kt.Name _
Select New KosherTypeLookup With {.Name = kt.Name, .ID = kt.KosherTypeID}
Return types.ToList
End Function
End Class
I then setup an object data source and mapped it to this class.
I have a few questions as when I was doing internet searches I didn't find anyone that seemed to be doing this and yet I know lookup tables / combo boxes are common...
Did I totally just miss something and there are better way(s) to do this?
I went with returning a list but I could have returned an IEnumerable or IQueryable. In my reading it seemed IQueryable had more functionality for linq to sql but since I am returning a simple two column list then I only need to return an IEnumerable or List. I went with a List since it's strongly typed. Did I make a good decision? AKA - Should I just have returned and IEnumerable or perhaps gone with IQueryable?
Thanks!
I'll answer in reverse order:
I wouldn't use IQueryable outside of your repos / DAL for the simple reason that since execution is deferred, you lose control of what exactly is executed (i.e., an aribtrary function could be assigned as a delegate for WHERE), making maintenance and testing much harder. I don't see an issue with you returning an IEnumberable(Of KosherTypeLookup) though.
If the lookup is a static lookup that never or rarely changes, you might want to look at a way to cache the lookup after the first use, to avoid having to hit the db every time that box is called. It really depends on your expected load, though. As always, performance/load testing will highlight where you need to optimize.