I am trying to Serialize an object to XML however my object is a generic list containing many records and causes the serializer to consume lots of memory. So I tried to serialize directly to a GZipStream with the following code:
Dim formatter As XmlSerializer = XmlSerializerFactory.GetSerializerForType(_type)
Using _ms As New MemoryStream()
Using gzStream As New GZipStream(_ms, CompressionMode.Compress, True)
_ms.Position = 0
formatter.Serialize(gzStream, obj)
_ms.Position = 0
gzStream.Flush()
gzStream.Close()
End Using
_ms.Position = 0
Dim decompressData() As Byte
Using gzStream As New GZipStream(_ms, CompressionMode.Decompress)
ReDim decompressData(9000 - 1) 'this number doesn't matter, the data in my test sample is small
Dim Len As Integer = gzStream.Read(decompressData, 0, decompressData.Length)
End Using
End Using
However I run into an InvalidDataException The magic number in GZip header is not correct. Make sure you are passing in a GZip stream. when trying to read the data into the decompressData array.
When I Serialize to a separate memory stream first and then compress that stream such as:
Dim formatter As XmlSerializer = XmlSerializerFactory.GetSerializerForType(_type)
Using _ms As New MemoryStream()
Dim uc_fileBytes() As Byte
Dim uc_len As Integer
Using _ms101 As New MemoryStream()
formatter.Serialize(_ms101, obj)
uc_fileBytes = _ms101.GetBuffer()
uc_len = _ms101.Length
End Using
Using gzStream As New GZipStream(_ms, CompressionMode.Compress, True)
_ms.Position = 0
gzStream.Write(uc_fileBytes, 0, uc_len)
gzStream.Flush()
gzStream.Close()
End Using
Dim decompressData() As Byte
Using gzStream As New GZipStream(_ms, CompressionMode.Decompress)
ReDim decompressData(9000 - 1)
Dim Len As Integer = gzStream.Read(decompressData, 0, decompressData.Length)
End Using
End Using
It works fine without error. But why does it fail when I serialize directly to the GZipStream?
The cause of the problem is because the GZipStream behaves differently (obviously) to the MemoryStream when writing to it. It doesn't handle paged writes very well.
Related
Please see part of the code used to save response as a string to resultData variable:
Using response As WebResponse = request.GetResponse()
Dim responseStream As IO.Stream = response.GetResponseStream()
Dim sr As New IO.StreamReader(responseStream)
resultData = sr.ReadToEnd()
It works correctly.
I have one case where the output is a binary file. How can I modify this code to save the reponse as a binary ResultData variable?
Thank you in advance for your support.
There is many ways to do that. This one, is one of those. (This one help you also to show the progress)
However (as advice) to monitoring progress there exists better approaches like Async methods etc.
In the example below I’m going to show you how to save e WebRequest as binary data.
Note that, I’m based on your code and what you want, but, as I said before there exists different better approaches.
'here the file you want to save
Dim LocalFilePath As String = "C:\Users\MyUser\Documents\FolderXYZ\yourfileName.extension"
Using reader As IO.Stream = request.GetResponse.GetResponseStream
Using writer As IO.Stream = New IO.FileStream(LocalFilePath, IO.FileMode.OpenOrCreate, IO.FileAccess.ReadWrite)
Dim b(1024 * 2) As Byte
Dim buffer As Integer = b.Length
Do While buffer <> 0
buffer = reader.Read(b, 0, b.Length)
writer.Write(b, 0, buffer)
writer.Flush()
Loop
End Using
End Using
I have two strings, each string is a PDF etiquette, I must to write this two etiquette into a PDF file. To do this I convert a each string to a byte array (I don't know if this is the best way) and each I Write into a PDF file. When I write one etiquette into PDF file all is good, I see the etiquette, but then I try to appending the second, the result is the same, into the file is only the first etiquette. For example this code write first etiquette and all working good:
Dim fs As FileStream = New FileStream(fullFileName, FileMode.CreateNew)
fs.Close()
fs = New FileStream(fullFileName, FileMode.Append)
Dim str As String = GetPDFString(27)
Dim binaryData As Byte() = ConvertStringToByte(str)
fs.Write(binaryData, 0, binaryData.Length)
fs.Close()
but if I want to append the second etiquette in the same PDF file using this code ... this not appending.
Dim fs As FileStream = New FileStream(fullFileName, FileMode.CreateNew)
fs.Close()
fs = New FileStream(fullFileName, FileMode.Append)
Dim str As String = GetPDFString(25)
Dim str1 As String = GetPDFString(27)
Dim binaryData As Byte() = ConvertStringToByte(str)
Dim binaryData1 As Byte() = ConvertStringToByte(str1)
fs.Write(binaryData, 0, binaryData.Length)
fs.Write(binaryData1, 0, binaryData1.Length)
fs.Close()
both have the same result, and I don't understand why the second etiquette isn't appending? Thank you a lot.
Your question title suggests that you are asking about how to append a byte to a FileStream, not about PDF, and not about Base64 string conversion (which you are using in your code).
Before asking a question on StackOverflow, you need to ensure you are conveying only one problem at a time. Remove everything that is not relevant, and prepare a code sample we can use in a brand new VS project, in order to reproduce your problem and help you solve it.
Now, if your question is really about appending a byte (or a byte array) to a file, it's as simple as one line of code (or two, if you keep FileStream approach). See below link:
C# Append byte array to existing file
Also copy-pasted for your convenience here (and converted from C# to VB.NET):
Dim appendMe As Byte() = New Byte(999) {}
File.AppendAllBytes("C:\test.exe", appendMe)
Or, to avoid memory overflow, if your byte array is expected to be large enough:
Public Shared Sub AppendAllBytes(path As String, bytes As Byte())
'argument-checking here.
Using stream = New FileStream(path, FileMode.Append)
stream.Write(bytes, 0, bytes.Length)
End Using
End Sub
With this line:
fs.Write(binaryData1, binaryData.Length + 1, binaryData1.Length)
Specifically the second argument (binaryData.Length + 1), you're telling it to start appending from the wrong position of binaryData1. If it's 3 bytes long, for example, and so is binaryData, it won't append anything. It should be similar to the first .Write line:
fs.Write(binaryData1, 0, binaryData1.Length)
So it appends all of binaryData1. It will still append it after binaryData - you don't need to specify the length of the preceeding binaryData in this line.
Alternatively, bypassing the above entirely, concatenate your two strings before encoding/writing them to the file:
Dim fs As FileStream = New FileStream(fullFileName, FileMode.CreateNew)
fs.Close()
fs = New FileStream(fullFileName, FileMode.Append)
Dim str As String = GetPDFString(id, token, depot, 25)
Dim str1 As String = GetPDFString(id, token, depot, 27)
Dim binaryData As Byte() = Convert.FromBase64String(str & str1) 'concatenate strings
fs.Write(binaryData, 0, binaryData.Length)
fs.Close()
I am able to write to the file perfectly... however I am having trouble reading from file and inserting read data into a listbox.
Public myData As New MySettings
Public saveFile As String = ("mysettings.ini")
'SAVE INFO TO SETTINGS FILE
Dim fs As Stream = New FileStream(saveFile, FileMode.Create)
Dim bf As BinaryFormatter = New BinaryFormatter()
For l_index As Integer = 0 To ListBox1.Items.Count - 1
Dim l_text As String = CStr(ListBox1.Items(l_index))
bf.Serialize(fs, l_text)
Next
fs.Close()
Return
This is the part where I am having trouble...
'LOAD INFO FROM SETTINGS FILE
Dim fs As Stream = New FileStream(saveFile, FileMode.Open)
Dim bf As BinaryFormatter = New BinaryFormatter()
For l_index As Integer = 0 To saveFile.Count - 1
Dim l_text As String = CStr(saveFile(l_index))
'myData = CType(bf.Deserialize(fs), CType(myData))
ListBox1.Items.Add(myData)
Next
fs.Close()
Return
Any help at all would be appreciated, even a point in the right direction.
Thanks in advance!!
The ListBox ObjectCollection (Items) is not marked as serializable, so you cant serialize the entire thing at once. You may have encountered this and used the loop to serialize each item. The loop does appear to serialize each item, but I don't know how you can deserialize in a loop - you wont know how many items there are, nor will the Serializer.
Rather than one item at a time, you could copy the ListBox items to an array and serialize the entire thing:
Dim ary(lb.Items.Count - 1) As Object
lb.Items.CopyTo(ary, 0)
' OpenOrCreate!
Using fs As New FileStream("C:\Temp\lbitems.bin", FileMode.OpenOrCreate)
Dim bf As New BinaryFormatter
bf.Serialize(fs, ary)
End Using ' close and dispose of stream
Deserialization is just the reverse:
Using fs As New FileStream("C:\Temp\lbitems.bin", FileMode.Open)
Dim bf As New BinaryFormatter
Dim myAry = bf.Deserialize(fs)
lb.Items.Clear()
lb.Items.Add(myAry)
End Using
You can make it a bit simpler using a List(of String) and assign it as the DataSource.
Private myLBItems As New List(Of String)
...
' fake items to add
myLBItems.Add("Foo")
myLBItems.Add("Bar")
myLBItems.Add("Option")
myLBItems.Add("Strict")
lb.DataSource = myLBItems
Using the List as the DataSource, you dont have to copy items from here to there - whatever is in the List will appear in the ListBox. Serialization is simple too:
bf.Serialize(fs, myLBItems)
Since Deserializing returns an Object, you need to cast it (Option Strict):
myLBItems = CType(bf.Deserialize(fs), List(Of String))
Note: This may not be doing what you want at all. Your code is serializing what is in the ListBox after converting it to String. Later, it looks like you want to Deserialize to a MySettings Type.
If MySettings is something like a Name and Value pair (or collection of them), probably half the data and all the Type information will have been lost.
I thought this would be an easy task but I seem to have gotten lost in a rabbit hole.
I am trying to convert a class to XML and then a byte array in order to send it as the content of a HTTPWebRequest for a WCF Restful webservice. My code will return the class as XML in the Byte Array but the format is incorrect. The XML looks like this:
<?xml version="1.0" encoding="utf-8"?><Person xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><FirstName>Richard</FirstName><LastName>Cranium</LastName></Person>
I was attempting to use the OmitXmlDeclaration method thinking this would eliminate the extra declarations but it does not work with the xmltextwriter object and when I use the xmlwriterobject I can't determine how convert the outcome to a Byte array.
Here is my code:
Private Shared Function GenerateXMLPersonAsByte(strFirstName As String, strLastName As String) As Byte()
' This should serialize this to a byte array
Dim p As New Person()
p.FirstName = strFirstName
p.LastName = strLastName
' Want to use this with the xmlTextWriter but it does not support the property it is to be used with the XMLWriter instead
'Dim settings As XmlWriterSettings = New XmlWriterSettings()
'settings.OmitXmlDeclaration = True
'settings.ConformanceLevel = ConformanceLevel.Fragment
'settings.CloseOutput = False
' Create the XmlWriter object and write some content.
Dim mStream As New MemoryStream()
Dim ser As New XmlSerializer(GetType(Person))
Dim xmlTW As New XmlTextWriter(mStream, Encoding.UTF8)
ser.Serialize(xmlTW, p)
mStream = DirectCast(xmlTW.BaseStream, MemoryStream)
mStream.Close()
Return mStream.ToArray()
End Function
What must I do in order to get the XML with the correct opening tag?
or
How do I get the output from the XmlTextWriter object to a byte array?
I can't answer the whole question, but convert to string first
Dim myXmlString As String
Dim myStringWriter As New StringWriter()
Dim myXmlTextWriter As XmlTextWriter = New XmlTextWriter(myStringWriter)
'Write XML, and then flush the writer...
myXmlTextWriter.Flush()
'Return text from string writer...
myXmlString = myStringWriter.ToString()
'Close the Objects...
myStringWriter.Close()
myXmlTextWriter.Close()
then convert to byte using (sorry, C#, but you get the .NET call)
byte[] data = ASCIIEncoding.UTF8.GetBytes(myXmlString);
I'm trying to set the user agent for a request with XmlRead. I googled a lot about this and couldn't find the answer. Here is my chunk of code:
Dim RssData As DataSet
Dim Title As String
Dim Url As String
Dim Stream As String
Dim buffer As Integer
RssData = New DataSet()
RssData.ReadXml("http://localhost/user_agent.php")
buffer = 0
For Each RssRow As DataRow In RssData.Tables("entry").Rows
Title = Microsoft.VisualBasic.Left(RssRow.Item("title").ToString, 30)
Stream += Title & vbCrLf
Next
LinkLabel3.Text = Stream
For Each RssRow As DataRow In RssData.Tables("entry").Rows
Title = Microsoft.VisualBasic.Left(RssRow.Item("title").ToString, 30)
Url = RssRow.Item("url").ToString
LinkLabel3.Links.Add(buffer, Title.Length, Url)
buffer = buffer + Title.Length + 2
Next
The part of the code that actually performs the web request is buried pretty deep so you'd have to inherit a bunch of code to do what you asked for. Instead, let me suggest a different path, download the XML on your own with code that's easy to set that header, and then load that into the dataset. The WebClient class lets you set arbitrary headers and has a simple DownloadString method. Once you've got that you can wrap it in a MemoryStream and pass that into ReadXml(). (I couldn't find a way to read the XML as a string, that's why I was forced to read it as Stream.)
''//Will hold our downloaded XML
Dim MyXml As String
''//Create a webclient to download our XML
Using WC As New System.Net.WebClient()
''//Manually set the user agent header
WC.Headers.Add("user-agent", "your user agent here")
''//Download the XML
MyXml = WC.DownloadString("http://localhost/user_agent.php")
End Using
''//Create our dataset object
Dim RssData As New DataSet()
''//There is no direct method to load XML as a string (at least that I could find) so we will
''// convert it to a byte array and load it into a memory stream
Dim Bytes As Byte() = System.Text.Encoding.UTF8.GetBytes(MyXml)
Using MS As New System.IO.MemoryStream(Bytes)
''//Load the stream into the reader
RssData.ReadXml(MS)
End Using
''//Your code continues normally here