How can i serialize and deserialize a dictionary in vb.net? - vb.net

Question

I am working with an application that needs to store large dictionaries of 40 000+ keys and values in a file, and then load them back into dictionaries on startup... Right now i am using a simple character separation with split and a for each loop on startup like:
key1=value1|key2=value2|key3=value3 etc...
however, i am looking for a more efficient way of serializing and deserializing the dictionaries... also with size of the serialized data in mind as there is quite a lot of entries.

Answer 1

You could make use of BinaryFormatter
On my mid end machine: save took: 390ms load took: 359ms data saved was about 1500kb
'save
Dim dict = New Dictionary(Of String, String)
For i = 1 To 40000
dict.Add("key" & i, "value" & i)
Next
Dim fs As IO.FileStream = New IO.FileStream("d:\test\test.bin", IO.FileMode.OpenOrCreate)
Dim bf As New Runtime.Serialization.Formatters.Binary.BinaryFormatter()
bf.Serialize(fs, dict)
fs.Close()
'load
Dim fsRead As New IO.FileStream("d:\test\test.bin", IO.FileMode.Open)
Dim objTest As Dictionary(Of String, String) = bf.Deserialize(fsRead)
fsRead.Close()

Answer 2

This works well when using stacked dictionaries:
Dim DataDict as New Dictionary(Of String,Dictionary(Of String,String))
Here is a working example of embedded dictionaries being serialized for writing and reading:
Imports System.IO
Module Modules
Public Sub TestDict()
Dim DictsToSave As New Dictionary(Of String, Dictionary(Of String, String))
For DictsToHave = 1 To 10
Dim SingelDictData As New Dictionary(Of String, String)
For Values = 1 To 10000
SingelDictData.Add("Key " & Values.ToString(), "Value " & Values.ToString())
Next
DictsToSave.Add("Key " & DictsToHave.ToString(), SingelDictData)
Next
Dim WriteResult = WriteMultiSerializedDict("D:\TestDict.Bin", DictsToSave)
Dim ReadResult As Dictionary(Of String, Dictionary(Of String, String)) = ReadMultiSerializedDict("D:\TestDict.Bin")
End Sub
Public Function WriteMultiSerializedDict(ByVal FullPath As String, ByVal DataDict As Dictionary(Of String, Dictionary(Of String, String))) As Boolean
Try
Dim FileStream As IO.FileStream = New FileStream(FullPath, IO.FileMode.OpenOrCreate)
Dim BinFormatter As New Runtime.Serialization.Formatters.Binary.BinaryFormatter()
BinFormatter.Serialize(FileStream, DataDict)
FileStream.Close()
Return True
Catch ex As Exception
Return False
End Try
End Function
Public Function ReadMultiSerializedDict(ByVal FullPath As String) As Dictionary(Of String, Dictionary(Of String, String))
Try
Dim DataDict As New Dictionary(Of String, Dictionary(Of String, String))
Dim FileStream As IO.FileStream = New FileStream(FullPath, IO.FileMode.Open)
Dim BinFormatter As New Runtime.Serialization.Formatters.Binary.BinaryFormatter()
DataDict = BinFormatter.Deserialize(FileStream)
FileStream.Close()
Return DataDict
Catch ex As Exception
Return Nothing
End Try
End Function
On my mid end dev machine, this took about 260 ms for the write and 545 for the read, with a file that measured at 3.70 mb. Important note: If you change any of the dict's keys (not values), the load bombs! So, as a matter of programming, always assemble your dicts in the same order. I've used this approach before, using 5 100 key/value pairs in the sub dictionary's, without issue or problem.