Efficiently storing a GUID in text? - vb.net

Question

Our app requires us to write out a large list of key, GUID value pairs for export to a plain text file.
6004, aa20dc0b-1e10-4efa-b683-fc98820b0fba
There will be potentially millions of these GUIDs, and there may not be a lot of room on the device where the file will be written, so is there a more efficient way to print the GUID to keep the file size down?
I experimented with Hex encoding
Dim g As Guid = Guid.NewGuid
Dim sb As New Text.StringBuilder
For Each b As Byte In g.ToByteArray
sb.Append(String.Format("{0:X2}", b))
Next
Console.WriteLine(sb.ToString)
this got it down to 32 chars, each line is a bit shorter:
9870, EBB7EF914C29A6459A34EDCB61EB8C8F
Are there any other approaches to write the GUIDs to the file that are more efficient?

Answer 1

I agree with previous comment, SQL would be ideal (or another DB) text files can be very unstable. Just done some quick testing iterating over millions of GUIDs and storing in text files.
Here is the outcome, basically anything more than 1 million guids would start to cause issues. You can store safely 10 million, but the file would struggle to open (on an average PC) as its > 150mb see below:
The code I used is below for you if wanted to try out. I know its not perfect, but gives you an idea of where the time goes. Main conclusions are to append the files with a bulk append, don't try to append each GUID individually if you can help it. This saves a large amount of processing and time!
Even if you convert to other formats like base or binary, I think you will still get similar variations on the file size, don't forget, you are still just saving as a text file, binary will no doubt be a larger file size due to string length!
Hope this helps comment back if you need anything!
Chicken
Dim Report_List As List(Of String)
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Report_List = New List(Of String)
Dim Zeros As Integer = 2
Do While Zeros < 8
Dim sw As Stopwatch = New Stopwatch
sw.Start()
Dim CountName As String = "1" & Microsoft.VisualBasic.Left("00000000000000000000000", Zeros)
Dim CountNum As Integer = CInt(CountName)
Dim Master_String As System.Text.StringBuilder = New System.Text.StringBuilder
For i = 1 To CountNum
Dim g As Guid = Guid.NewGuid
'Dim sb As New System.Text.StringBuilder
'For Each b As Byte In g.ToByteArray
' sb.Append(String.Format("{0:X2}", b))
'Next
'Master_String.AppendLine(sb.ToString)
'Master_String.AppendLine(sb.ToString)
Master_String.AppendLine(Convert.ToBase64String(g.ToByteArray))
i += 1
Next
Using sr As StreamWriter = New StreamWriter("c:\temp\test-" & CountName & ".txt", False)
sr.Write(Master_String.ToString)
End Using
sw.Stop()
Report_List.Add(sw.Elapsed.ToString & " - " & CountName)
Zeros += 1
Loop
For Each lr In Report_List
Me.ListBox1.Items.Add(lr)
Next
End Sub