Dim docu As New XmlDocument()
docu.load("C:\bigfile.xml")
Dim tempNode As XmlNode
tempNode = docu.SelectSingleNode("/header/type")
someArray(someIndex) = tempNode.innerText
...do something more...
I am using XmlDocument() to load a huge XML document (100~300MB)
When I open the document and read it as string, my application uses about 900MB of RAM.
I wonder why it happens and how can I prevent it ?
Note that : even, the XmlDocument does not have Dispose() to remove allocated things.
Although I need the whole string of the huge XML file in later part of the app, the /header/type.innerText is only a single word
More of source :
Private Sub setInfo(ByVal notePath As String)
Dim _NOTE As XDocument
_NOTE = XDocument.Load(notePath)
If (From node In _NOTE...<title> Select node).Value = "" Then
lvlist.Items.Add("No Title")
Else
lvlist.Items.Add((From node In _NOTE...<title> Select node).Value)
End If
lvlist.Items(lvlist.Items.Count - 1).SubItems.Add((From node In _NOTE...<group> Select node).Count)
End Sub
It reads XML document, counts tags and retrieves string value. That's all.
After having those values, _NOTE (XDocument) is of no use at that time.
XmlReader will probably solve your need. From MSDN:
Represents a reader that provides fast, noncached, forward-only access to XML data.
Right, XmlDocument does not have a Dispose method. It is put into memory. It does follow an object life cycle. So if you create the object in a Function and return the string you want then it will free itself as it losses scope from the Function.
Related
I have written a test application to modify a few hundred XMLs, basically I modify a few nodes and then save the XML again.
The input XML files are UTF-8-BOM but the output is UTF-8 (as viewed in Notepad++).
The code runs on a console application in VB.NET 4.7.2 and this is my basic code:
Dim myXML As XmlDocument = New XmlDocument
Dim nodelist As XmlNodeList
Dim node As XmlNode
myXML.Load(file)
nodelist = myXML.SelectNodes("//root/row")
For Each node In nodelist
'All my code goes here
Next
myXML.Save(file)
I tried things like:
myXML.CreateXmlDeclaration("1.0", "UTF-16", "")
But that didn't work. I have been searching and it seems everyone has the exact oposite issue and checking in the MSDOC I can't see any reference to specify BOM or not:
https://learn.microsoft.com/en-us/dotnet/api/system.xml.xmldeclaration.encoding?view=netframework-4.7
The issue is that when the files are imported to the DB without BOM some characters show corruption so I really need the same encoding.
After so much reading I could try to rewrite my app using StreamWriter instead XMLDocument but if there is a workaround about that I would pretty much prefer it :). Thanks!
As suggested by #JosefZ I used this strategy:
Using writer = New XmlTextWriter(file, New UTF8Encoding(True))
Dim xmlsettings As XmlWriterSettings = New XmlWriterSettings
xmlsettings.OmitXmlDeclaration = False
xmlsettings.ConformanceLevel = ConformanceLevel.Fragment
xmlsettings = writer.Settings
myXML.Save(writer)
End Using
I am trying to extract an application resource from My.Resources.FILE
I have discovered how to do this with DLL & EXE files, but I still need help with the code for extracting PNG & ICO files.
Other file types also. (If possible)
Here is my current code that works with DLL & EXE files.
Dim File01 As System.IO.FileStream = New System.IO.FileStream("C:\Users\" + Environment.UserName + "\Desktop\" + "SAMPLE.EXE", IO.FileMode.Create)
File01.Write(My.Resources.SAMPLE, 0, My.Resources.SAMPLE.Length)
File01.Close()
First things first, the code you have is bad. When using My.Resources, every time you use a property, you extract a new copy of the data. That means that your second line is getting the data to write twice, with the second time being only to get its length. At the very least, you should be getting the data only once and assigning it to a variable, then using that variable twice. You should also be using a Using statement to create and destroy the FileStream. Even better though, just call File.WriteAllBytes, which means that you don't have to create your own FileStream or know the length of the data to write. You should also not be constructing the file path that way.
Dim filePath = Path.Combine(My.Computer.FileSystem.SpecialDirectories.Desktop, "SAMPLE.EXE")
File.WriteAllBytes(filePath, My.Resources.SAMPLE)
As for your question, the important thing to understand here is that it really has nothing to do with resources. The question is really how to save data of any particular type and that is something that you can look up for yourself. When you get the value of a property from My.Resources, the type of the data you get will depend on the type of the file you embedded in first place. In the case of a binary file, e.g. DLL or EXE, you will get back a Byte array and so you save that data to a file in the same way as you would any other Byte array. In the case of an image file, e.g. PNG, you will get back an Image object, so you save that like you would any other Image object, e.g.
Dim filePath = Path.Combine(My.Computer.FileSystem.SpecialDirectories.Desktop, "PICTURE.PNG")
Using picture = My.Resources.PICTURE
picture.Save(filePath, picture.RawFormat)
End Using
For an ICO file you will get back an Icon object. I'll leave it to you to research how to save an Icon object to a file.
EDIT:
It's important to identify what the actual problem is that you're trying to solve. You can obviously get an object from My.Resources so that is not the problem. You need to determine what type that object is and determine how to save an object of that type. How to do that will be the same no matter where that object comes from, so the resources part is irrelevant. Think about what it is that you have to do and write a method to do it, then call that method.
In your original case, you could start like this:
Dim data = My.Resources.SAMPLE
Once you have written that - even as you write it - Intellisense will tell you that the data is a Byte array. Your actual problem is now how to save a Byte array to a file, so write a method that does that:
Private Sub SaveToFile(data As Byte(), filePath As String)
'...
End Sub
You can now which you want to do first: write code to call that method as appropriate for your current scenario or write the implementation of the method. There are various specific ways to save binary data, i.e. a Byte array, to a file but, as I said, the simplest is File.WriteAllBytes:
Private Sub SaveToFile(data As Byte(), filePath As String)
File.WriteAllBytes(filePath, data)
End Sub
As for calling the method, you need to data, which you already have, and the file path:
Dim data = My.Resources.SAMPLE
Dim folderPath = My.Computer.FileSystem.SpecialDirectories.Desktop
Dim fileName = "SAMPLE.EXE"
Dim filePath = Path.Combine(folderPath, fileName)
SaveToFile(data, filePath)
Simple enough. You need to follow the same steps for any other resource. If you embedded a PNG file then you would find that the data is an Image object or, more specifically, a Bitmap. Your task is then to learn how to save such an object to a file. It shouldn't take you long to find out that the Image class has its own Save method, so you would use that in your method:
Private Sub SaveToFile(data As Image, filePath As String)
data.Save(filePath, data.RawFormat)
End Sub
The code to call the method is basically as before, with the exception that an image object needs to be disposed when you're done with it:
Dim data = My.Resources.PICTURE
Dim folderPath = My.Computer.FileSystem.SpecialDirectories.Desktop
Dim fileName = "SAMPLE.EXE"
Dim filePath = Path.Combine(folderPath, fileName)
SaveToFile(data, filePath)
data.Dispose()
The proper way to create and dispose an object in a narrow scope like this is with a Using block:
Dim folderPath = My.Computer.FileSystem.SpecialDirectories.Desktop
Dim fileName = "SAMPLE.EXE"
Dim filePath = Path.Combine(folderPath, fileName)
Using data = My.Resources.PICTURE
SaveToFile(data, filePath)
End Using
Now it is up to you to carry out the same steps for an ICO file. If you are a hands on learner then get your hands on.
I'm writing some vb.net code to retrieve some data from middleware, do some stuff with it, and then save a dictionary of name/value pairs to cache for quick retrieval when needed.
It all seems to go well until I retrieve the object from cache and it is always Null/Nothing.
Yet in the watch window I can see my cached object and it has data in it.
Thinking maybe it's a serialization issue I made a new class that inherits Dictionary and is marked as serializable but I still get the same results.
So right now I have this class:
<Serializable()> Public Class SerialDict
Inherits Dictionary(Of String, String)
Public Sub New()
End Sub
End Class
I populate it and put it into cache like this:
Dim Licenses As New SerialDict
For Each r As DataRow In dtLicenses.Rows
Dim prikey As String = r("SettingID").ToString.Trim
Dim decryptionKey As String = GetHash((xx))
Dim licData As String = DecryptData(r("SettingVal"), decryptionKey)
Licenses.Add(r("SettingKey"), licData)
Next
If IsNothing(HttpContext.Current.Cache("Licenses")) Then
HttpContext.Current.Cache.Add("Licences", Licenses, Nothing, Cache.NoAbsoluteExpiration, Cache.NoSlidingExpiration, CacheItemPriority.Default, Nothing)
End If
Then elsewhere we need to check that data so I try to retrieve it like this:
Dim Licences As SerialDict = CType(HttpContext.Current.Cache("Licenses"), SerialDict)
At this point Licenses is always Nothing, but the watch window shows data in HttpContext.Current.Cache("Licenses").
Any suggestions? Thanks!
I have developed a Visual Basic.net application that uses serialization to save an object. I am wanting to open and save this object in two different Visual Basic.net applications.
What is the best way to do this? Do I need to create a class library to do this?
Can I please have some help for this?
EDIT
I am wanting to be able to open and save the object in both applications.
Depending on how complicated your data is, you should be able to simply mark your data's class with a <Serializable> attribute. You can then simply call the Serialize method in one application, save to disk, then read the file into your other application and call Deserialize.
You will need to define the class in both applications, which is easiest to do by sharing a common library.
See the MDSN example for basic serialization.
You can write/read to xml, then you would just have to check the folder where you save them from the other app to see if a new file has been created or updated.
Function to serialize object and write to xml
Public Function MyObjectFileGeneration()
Try
Dim strPath As String = System.IO.Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().CodeBase)
strPath = Replace(strPath, "file:\", "")
Dim myObj = Form1.MyObject
Dim objStreamWriter As New StreamWriter(strPath & "\MyFolder\MyObj.xml", False)
Dim x As New XmlSerializer(s.GetType)
x.Serialize(objStreamWriter, MyObj)
objStreamWriter.Close()
Return True
Catch ex As Exception
'Do something here if failure...
Return False
End Try
End Function
Function to read xml and de-serialize to object
Public Function GetMyObjFromXMLFile() As MyObj
'check if file exists first...
If xmlFileExists() Then
Dim strPath As String = System.IO.Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().CodeBase)
Dim objStreamReader As New StreamReader(Replace(strPath & "\MyFolder\MyObj.xml", "file:\", ""))
Dim MyObj As New MyObject
Dim x As New XmlSerializer(MyObj.GetType)
MyObj = x.Deserialize(objStreamReader)
objStreamReader.Close()
Return MyObj
Else
Return New MyObj
End If
End Function
I wish there was an easy way to do this, but unfortunately, I hit this wall also..
Serializable data can only be reread by the SAME application. (it gives you a lovely error message about this.) I tried using a serialized connection for simplified packet transfers, unsuccessfully..
Depending on how good you are at programming, I have a recommendation on this one..
Serialize your variables to a memorystream, then cut the header section out and shove it to another file stream, then when you reload it, save a variable to a memorystream to get the new header, then attach the remaining data, then read serialization..
haven't tried it yet, but when I get back to it, this is my next method.
I did see an XML method, but recommend using a compression/encryption library to keep your data safe.. did see some simple ways to possibly do that..
Sorry, I don't have code on this round, but when I investigate it, I will append this response..
Is there something that needs to be done with the following code to release the memory it uses?
Dim objImage As MemoryStream
Dim objwebClient As WebClient
Dim sURL As String = Trim(m_StationInterface.PicLocation)
objwebClient = New WebClient
objImage = New MemoryStream(objwebClient.DownloadData(sURL))
m_imgLiftingEye.Image = Image.FromStream(objImage)
The code is on a popup form that shouldn't ever get disposed. A new image is loaded onto the form every time it pops up. However, the process size for the application continues to grow each time it makes it through that code block.
I've tried objImage.Close() and .Flush(), objWebClient.Dispose(). The process size still grows by a good 4mb after every call. It's like the old image is kept in memory.
Image implements IDisposable, so you should Dispose the old image before replacing it with a new one.
Something like (bear with me, I haven't used VB in a while):
Dim objImage As MemoryStream
Dim objwebClient As WebClient
Dim sURL As String = Trim(m_StationInterface.PicLocation)
objwebClient = New WebClient
objImage = New MemoryStream(objwebClient.DownloadData(sURL))
If m_imgLiftingEye.Image Is Not Nothing Then
m_imgLiftingEye.Image.Dispose()
End If
m_imgLiftingEye.Image = Image.FromStream(objImage)
Try this:
Function GetImage() As Image
Using wc As New WebClient(), _
ms As New MemoryStream(wc.DownloadData(m_StationInterface.PicLocation.Trim())
GetImage = Image.FromStream(ms).Clone()
End Using
End Function
MemoryStream implements the IDisposable interface, so you should call Dispose on that object when you are done using it:
objImage = New MemoryStream(objwebClient.DownloadData(sURL))
m_imgLiftingEye.Image = Image.FromStream(objImage)
objImage.Dispose()
I would guess your conclusion was right; the image (in the memory stream) does remain in the memory.
Update: as Marc pointed out Image.FromStream requires the stream to remain open for the lifetime of the image. To resolve this the MemoryStream variable should be declared in same scope as the image (as a field in the form). When loading the image, there should first be a check whether the MemoryStream already is open and if so, close and dispose it before using the variable for a new stream (let's assume that we call it m_imageStream). Since the image also implements IDisposable, the same is true for that one:
If Not m_imageStream Is Nothing Then
m_imageStream.Dispose()
End If
If m_imgLiftingEye.Image Is Not Nothing Then
m_imgLiftingEye.Image.Dispose()
End If
m_imageStream = New MemoryStream(objwebClient.DownloadData(sURL))
m_imgLiftingEye.Image = Image.FromStream(m_imageStream)
I know I already gave one answer, but I've been thinking since then...
You said that this form should never be disposed. In that case, when exactly is this image load happening? My previous answer assumed it was during the form Shown event. However, if it's during the form Load event, it should only happen once total.
That is, unless more than one instance of the form is being created. If that's the case, and the previous form isn't being reused, you're ending up with multiple copies of the same form loaded in memory, each with its own copy of the image.
You could try
set objImage = nothing
set objwebClient = nothing
Often, like with ADO, if you don't explicitly set it to nothing it doesn't get released properly.