I hope you can help with what is probably a naive use of Streams in VB.net.
I have a program that batch processes images and am running out of memory making repeated use (in a loop) of the following:
Using str As Stream = File.OpenRead(file_stem + CStr(file_number) + "." + file_extension)
temp_img = Image.FromStream(str)
str.Close()
End Using
PictureBox1.Image = temp_img
bm = PictureBox1.Image.Clone
temp_img is globally declared Dim temp_img As Image. bm is declared in the same Sub routine as the loop Dim bm As Bitmap.
As the program runs I can see in Task Manager the memory usage rising and then it crashes with an out of memory error. It's as if each time I am using the Stream it is keeping the memory used. What am I doing wrong here?
EDIT:
This thread appears to have gone cold now, but I thought I would share my "work around" for this. It would seem that this is a VB.net bug as the way I have fixed it is to add a MsgBox which is shown only one, immediately prior to the first call to the subroutine that processes sets of 60 images. I ran a single job that processed 96 sets of 60 images and the memory usage didn't rise above about 45MB. The important thing that makes me think it is a bug is that I only show the MsgBox before the first set of 60 and all are run in series. Showing an MsgBox shouldn't fix anything in itself anyway!
Related
I'm trying to build an educational app for a child with severe disabilities. It is supposed to teach the child how to add two numbers ranging from 1 - 9. In order to do so, I want to make it play back a sound, consisting of n (the number the child clicked) times the sound "one.wav". I use the following code:
For i As Integer = 1 To buttonNumber
i = i + 1
My.Computer.Audio.Play("one.wav")
Threading.Thread.Sleep(1000)
Next
While it does play the sound, the number of times is highly irregular and does not equal "buttonNumber". What is the problem? (Note that I've tried the same code without the "i = i + 1".)
EDIT:
The following piece of code does NOT work:
Private Sub PlayButtonAudio(buttonNumber As Integer)
Dim bytes As Byte() = IO.File.ReadAllBytes("one.wav")
For i As Integer = 1 To buttonNumber
My.Computer.Audio.Stop()
My.Computer.Audio.Play(bytes, AudioPlayMode.Background)
Threading.Thread.Sleep(playDuration)
Next
End Sub
Quite curiously though, it works perfecty whenever "buttonNumber" is larger than or equal to four.
The 1000 milliseconds you've specified is from the start of the playing wav, so you'll need the time gap to add the length of the sound. The audio begins to play through the OS and your code continues while it's playing. If you attempt to play it a second time while the first is still playing you'll run into problems.
There may be other methods of playing audio which don't return to your code until the audio is complete, but this would freeze your UI.
[Edit: Unless you do it in a separate thread. But for your purposes it's possibly simpler to just increase the delay.]
The My.Computer.Audio component is using a shared system resource and the Play method is just a wrapper around the API PlaySound function.
I am assuming that you only want the sound to play for a maximum of the sleep duration. If it takes the system longer than specified sleep duration (or a significant portion thereof) to load the WAV file into memory, the first playback may be mishandled. This effect can be mitigated by loading WAV into a byte array and using the Play overload that takes a byte array instead of a file name.
As you are using a shared resource, other sounds may be playing; calling My.Computer.Audio.Stop() will cancel anything currently playing.
Const playDuration As Integer = 1000 ' milliseconds
' load the WAV file into a byte array
' this will minimize the latency caused by loading the file from disc
Dim bytes As Byte() = IO.File.ReadAllBytes("one.wav")
For i As Integer = 1 To buttonNumber
My.Computer.Audio.Stop() ' cancel any currently playing sound
My.Computer.Audio.Play(bytes, AudioPlayMode.Background)
Threading.Thread.Sleep(playDuration)
Next
My.Computer.Audio.Stop() ' cancel playing if wav file duration is longer than playDuration
It turns out that I had the first three buttons twice in the handler list. Now it works perfectly.
I am currently making a vb program that i plan to make very big. I have a decent knowledge of visual basic but today i came across something i do not understand. Because of the huge size of my program , i decided to try and keep the program as organized as possible by putting specific subs in modules. These subs consist of httprequest , webbrowsers(control), webclients and alot of loops. In order to prevent these subs from lagging my main application i thread them using threading.thread and i start them from my main form. But this leads to two problems.
Problem 1: The threads cannot in any way interact with the main form.
Once the a httprequest or webclient collects the information from my desired website, i am trying to make it add the info to a listbox in my main form, So what i did is it typed
Msgbox("Info Sent")
form1.listbox1.items.add(String)
The first messagebox will show but although the code right under it runs, nothing is added to the first forms listbox.I am not using delegates to transfer the information, instead, although its not a good habit, i am using checkforillegalcrossovers.
Problem 2: Threading with a webbrowser.
Threading with a webbrowser using threading.thread also does not work because it causes an active x error. After looking it up i found that a solution was to use a single threaded apartment but this would not work because i may need multiple threads running off the same sub at once.
One solution that i have found to this problem is creating another form completely and setting it invisible, and since the form is its own thread i do not need to use threading.thread , but the problem comes when i am trying to create multiple threads, or else i can somehow dynamically create the threads and put the subs inside of it programically this method wont work And even if it does i feel that it is sloppy so i will leave this for one of two last resorts.
The other solution is the most simple one in which i just put all of the code in the main form, but if i keep on doing that form1 is gonna become huge and sloppy, doing this wont solve the webbrowser problem either and even when using regions i still feel that something that 1000+ lines deserves its own class.
There must be some solution out there that solves these problems. Any help would be appreciated, Thanks.
I checked my code for updating the progress bar, and using a single thread with synclock will NOT work. They way I make it work is perform the step of the pbar each time after a thread is started as I have limited total threads (say less than 5 threads). Thus, even the progress bar steps before the threads are finished, but it will not progress further before new threads started. It is not 100% accurate but it more or less telling the progress
'update the progress bar
some_form.PBar1.PerformStep()
' This while loop is to count the existing running thread,
' and determine whether new thread should start
While 1
Dim t2 = New System.Threading.Thread(Sub() WaitForPermission())
t2.Start()
t2.Join()
If proceed_gen Then
Exit While
End If
End While
'Start doing what I need to do
Dim t1 = SomeSub()
t1.Start()
'End of code, as VB doest not have thread.detach()
Correct me if I am wrong, but you probably have to use a background worker. I know this is annoying, but this is the limitation of VB.net.
Or, you can have something like this (pseudo code, not tested)
structure some_struct
'define the strings you want to update, and their status such that
'main() knows if you need to update the stuff to the form
' You can also put how many threads are running, and the status of each thread,
'such that the main knows if all threads are completed
end structure
sub your_sub()
'Manipulate the website, and update the data structure with
'proper stuff you need
end sub
sub main(){
dim t1 = New System.Threading.Thread(Sub() your_sub())
t1.start()
' I am listing only one threads here, but start as many as you want
'check if there are strings that you need to update to the form
while 1
'check if there are any stuff you want to update from the data structure.
' Make sure you use synclock on the data structure, so each thread won't fight each other on accessing the data struct
dim should_update as boolean = false
should_update = 'Something thatyou should implement to judge whether you should update the form.
'You can start a thread and join it so the thread won't fight with other threads for accessing the data structure
dim some_string as string
if should_update
some_string = 'You may also need a thread to join to get the stuff you need. Dim the string as an array if it is required.
'You can also try pass by ref
'if you need to use thread to access the data structure to know if you need to update the form
form1.listbox1.items.add(some_string )
end if
end while
end sub
This is an ugly solution, but it will help you do the job...
I have the following code which I am using to populate a ImageList from a SQLite database with images stored as blobs.
Public Sub populateImagesStyles()
ShoeImages1.Images.Clear()
StyleImagesLView.Items.Clear()
Dim s As SQLiteDataReader
Dim rcount As Integer = 0
dbLocalQuery = New SQLiteCommand("SELECT id, image FROM tblImages", dbLocal)
s = dbLocalQuery.ExecuteReader()
While s.Read()
rcount += 1
ShoeImages1.Images.Add(CStr(s("id")), byte2img(s("image")))
StyleImagesLView.Items.Add(CStr(s("id")), CStr(s("id")))
End While
s.Close()
Here is the byte2img function...
Public Function byte2img(ByVal imgByte As Byte()) As Image
Dim imgMemoryStream As System.IO.MemoryStream = New System.IO.MemoryStream(imgByte)
byte2img = Drawing.Image.FromStream(imgMemoryStream)
End Function
The database contains over 250 images and this process is completed twice on load to populate two different ImageList, because I need the images displayed at two different sizes.
When the process runs on loading the form, it causes the process to consume between 800MB and 1GB of system memory, unless I manually run the process again from an form control, which seems to trigger garbage collection.
Stepping through the loading process, it is clear that it is the byte2img process that is causing the memory usage to escalate - what is the best way to mitigate this?
Also, if anyone can think of a more efficient process to execute this, i'm all ears. The images have to be stored in the database file because I need to be able to just package the .db file and send it to a remote location at a moments notice, so I can't mess with folders with images.
All help appreciated.
You are creating a lot of memory streams without disposing of them. Try this:
Public Function byte2img(ByVal imgByte As Byte()) As Image
Dim img As Image
Try
Using ms As New MemoryStream(imgbyte)
img = Drawing.Image.FromStream(ms)
End Using ' auto dispose of the MS
Catch ex As Exception
' report possibly bad/missing imgByte()
' resulting in an error in either place
End Try
Return img
End Function
An imprecise way to detect this kind of thing is to watch the HANDLES count in TaskManager.
Ok, I've found a solution/workaround that seems to work - call the PopulateImageStyles sub when a user visits the specific TabPage the ImageList resides on.
For some arbitrary reason, when run this way (as above, when called on the form), the process never proceeds to consume more than 50-60 MB of working memory.
I'll add a Background Worker so that the process can execute without hanging the form.
I'll try to create an zip file with the DotNetZip-Libary with 106 Images (675MB) with the following code:
Public Function GetZip() As Byte()
Dim zip As New Ionic.Zip.ZipFile(String.Format("{0}.zip", Me.GallerySystemName))
AddHandler zip.SaveProgress, AddressOf SaveProgress
For Each img In Me.Images
zip.AddFile(img.OriginalFile.FullName, "")
Next
Dim bytZip As Byte()
Using ms As New MemoryStream
zip.Save(ms)
bytZip = ms.ToArray
End Using
Return bytZip
End Function
When I run this code, the execution stops usally at image 40 (sometimes earlier) without any exeption. Nothing happens. I tried to save the zip directly to a file. It works.
Any ideas?
Jan
SET the zip object property ParallelDeflateThreshold to -1 just before saving the zip file
zip.ParallelDeflateThreshold = -1
REF: http://forums.codeguru.com/showthread.php?534177-Issue-with-DotNetZip-ionic.zip-class-hanging-on-save
It's been almost 2 years since your question, so I doubt this will help but I just encountered the same problem in v1.9.1.8.
I worked around it by increasing the BufferSize and CodecBufferSize ZipFile properties to 1MB each.
I can't download the DotNetZip source because of filters at work but here is a very-likely-related comment from http://dotnetzip.codeplex.com/releases/view/68268
There is a pretty major bug in the code. I am working to figure it out. Another chap logged it before me: Deadlock in ParallelDeflateOutputStream.EmitPendingBuffers The zip hangs. End-of-day I will have to rip this code out and start over with a new library. I need to call my last job and give them a head's up b/c I used this library at my last job. They will likely have to rip the code out too.
by jnarkiewicz on May 30 at 6:31 PM
So if this is indeed the problem, increasing the size of those buffers just lowers the likelihood of the deadlock occurring and is not an ideal solution.
This loop takes forever to run as the amount of items in the loop approach anything close to and over 1,000, close to like 10 minutes. This needs to run fast for amounts all the way up to like 30-40 thousand.
'Add all Loan Record Lines
Dim loans As List(Of String) = lar.CreateLoanLines()
Dim last As Integer = loans.Count - 1
For i = 0 To last
If i = last Then
s.Append(loans(i))
Else
s.AppendLine(loans(i))
End If
Next
s is a StringBuilder. The first line there
Dim loans As List(Of String) = lar.CreateLoanLines()
Runs in only a few seconds even with thousands of records. It's the actual loop that's taking a while.
How can this be optimized???
Set the initial capacity of your StringBuilder to a large value. (Ideally, large enough to contain the entire final string.) Like so:
s = new StringBuilder(loans.Count * averageExpectedStringSize)
If you don't specify a capacity, the builder will likely end up doing a large amount of internal reallocations, and this will kill performance.
You could take the special case out of the loop, so you wouldn't need to be checking it inside the loop. I would expect this to have almost no impact on performance, however.
For i = 0 To last - 1
s.AppendLine(loans(i))
Next
s.Append(loans(last))
Though, internally, the code is very similar, if you're using .NET 4, I'd consider replacing your method with a single call to String.Join:
Dim result as String = String.Join(Envionment.NewLine, lar.CreateLoanLines())
I can't see how the code you have pointed out could be slow unless:
The strings you are dealing with are huggggge (e.g. if the resulting string is 1 gigabyte).
You have another process running on your machine consuming all your clock cycles.
You haven't got enough memory in your machine.
Try stepping through the code line by line and check that the strings contain the data that you expect, and check Task Manager to see how much memory your application is using and how much free memory you have.
My guess would be that every time you're using append it's creating a new string. You seem to know how much memory you'll need, if you allocate all of the memory first and then just copy it into memory it should run much faster. Although I may be confused as to how vb.net works.
You could look at doing this another way.
Dim str As String = String.Join(Environment.NewLine, loans.ToArray)