How can I speed up VB copy - vb.net

I'm running the following loop successfully when the number of items is low. However, when run against a larger list on the ListView, it seems to be taking way too long. I tested it with a list of 8,700 files and it took about two hours to complete. Is there something I can do to speed this up? I guess that removing the check for the Cancel button would help but I would like to keep that there for usability. As I've mentioned in earlier posts, I'm pretty new to Visual Basic so please provide lots of explanation with your suggestions. Thanks. Here's the code:
For i As Integer = 0 To m_CountTo
' Has the background worker be told to stop?
If BackgroundWorker1.CancellationPending Then
' Set Cancel to True
e.Cancel = True
Exit For
End If
'Select the row from the LVFiles ListView, then move the first column (0) into strSourceFilePath and the last
' column (3) into strDestFilePath. Execute the CopyFile method to copy the file.
LVFiles.Items(i).Selected = True
strSourceFilePath = LVFiles.SelectedItems(i).SubItems(0).Text
strDestFilePath = LVFiles.SelectedItems(i).SubItems(3).Text
My.Computer.FileSystem.CopyFile(strSourceFilePath, strDestFilePath, overwrite:=False)
' Report The progress of the Background Worker.
BackgroundWorker1.ReportProgress(CInt((i / m_CountTo) * 100))
' Me.LabelStatus.Text = FormatPercent((i + 1) / (intLVIndex + 1), 2) ' Show Percentage in Label
SetLabelText_ThreadSafe(Me.LabelStatus, FormatPercent(i / m_CountTo, 2))
Next

The Backgroundworker encapsulates a new thread. You cannot directly access controls that are created in another thread. If you do you will get an InvalidOperationException because of a cross-thread operation. The Backgroundworker however offers some functionality to share data (or access to controls) between threads. You should use them.
Private Sub StartBGW_Click(sender As Object, e As EventArgs) Handles StartBGW.Click
Dim dict As New Dictionary(Of String, String)
For i As Integer = 0 To m_CountTo
dict.Add(Me.LVFiles.Items(i).SubItems(0).Text,
Me.LVFiles.Items(i).SubItems(3).Text)
Next
Me.BackgroundWorker1.RunWorkerAsync(dict)
End Sub
First we prepare a dictionary that contains the source as Key and the target as Value. This object is given to the BackgroundWorker as a parameter.
Now comes the essential part:
Private Sub BackgroundWorker1_DoWork(sender As Object, e As System.ComponentModel.DoWorkEventArgs) Handles BackgroundWorker1.DoWork
Dim counter As Integer = -1
Dim dict = DirectCast(e.Argument, Dictionary(Of String, String))
For Each kvp In dict
counter += 1
' Has the background worker be told to stop?
If Me.BackgroundWorker1.CancellationPending Then
' Set Cancel to True
e.Cancel = True
Exit For
End If
'Select the row from the LVFiles ListView, then move the first column (0) into strSourceFilePath and the last
' column (3) into strDestFilePath. Execute the CopyFile method to copy the file.
My.Computer.FileSystem.CopyFile(kvp.Key, kvp.Value, overwrite:=False)
' Report The progress of the Background Worker.
Me.BackgroundWorker1.ReportProgress(CInt((counter / m_CountTo) * 100), counter)
Next
End Sub
We don't access the ListView anymore. Instead we use the dictionary that is given to us as a parameter through e.Argument. Theres also a slight difference in the BackgroundWorker1.ReportsProgress line. There's a second parameter I have used to pass the current index to the ProgressChanged event which can be obtained via e.UserState.
Private Sub BackgroundWorker1_ProgressChanged(sender As Object, e As System.ComponentModel.ProgressChangedEventArgs) Handles BackgroundWorker1.ProgressChanged
Me.LVFiles.Items(Convert.ToInt32(e.UserState)).Selected = True
Me.LabelStatus.Text = e.ProgressPercentage.ToString
End Sub
This event is designed to be raised with a SynchronizationContext of the calling thread, in this case the UI thread. Here we can safely access any control and update them. The index is passed as e.UserState, so we can access the relevant item and set their Selected property to true.
The biggest improvement comes from the change of Me.LVFiles.SelectedItems(i).SubItems(0).Text to Me.LVFiles.Items(i).SubItems(0).Text. I'm not a professional, but it seems that SelectedItems isn't a real list. Instead it iterates through every item using the SendMessage API until the desired index is reached. This is why it takes longer the higher your index is. Everytime it starts with the first item and iterates through them. Lot of operations.
The second improvement is the separation of code that access UI controls. It's all done in one method now. More clear and readable.
Update: #Enigmativity mentioned that SelectedListViewItemCollection implements IList and therefore is a real list. Even though it has no underlying list containing all selected items like you have in ListViewItemCollection. My point was to say, that accessing a single element is more complicated.

Related

Secondary thread causes "Application has stopped working" crashes even when invoking

I have an application which has a form with a DataGridView bound to a BindingSource, which is bound to a DataTable:
bsList.DataSource = dsData
bsList.DataMember = "List"
dgvList.DataSource = bsList
The underlying data which populates dsData.Tables("List") can change whilst the user is working so to combat this I have a background thread which routinely checks the database for changes and updates dsData.Tables("List"). It also changes the colour of any row where another user is currently working.
However, users report that when this background updating functionality is enabled the application routinely CTDs with no application error message. I have been unable to reproduce this and my attempt to log the crashes via writing to a log file in Private Sub MyApplication_UnhandledException(sender As Object, e As UnhandledExceptionEventArgs) Handles Me.UnhandledException hasn't worked as the log file is never written to, suggesting this event is never triggered.
The thread is instantiated like this:
LiveUpdating = New Thread(AddressOf UpdateUserLocation) With {.IsBackground = True}
LiveUpdating.Start()
This is the UpdateUserLocation sub:
Public Sub UpdateUserLocation()
Do While My.Settings.customBackgroundUpdating = True And formLoaded = True
UserLocations.Clear()
dtUsers = CLS_USERS.GetUsersSequence(winUser)
dtProgress = DAC.GetProgress()
For Each CandRow As DataRow In dsHHData.Tables("List").Rows
Dim CandReadDate As Date
Dim CandRowNextRead As String = DBNull.Value.ToString
If Not (CandRow("NEXT READ").ToString = DBNull.Value.ToString) Then
If Date.TryParse(CandRow("NEXT READ").ToString, CandReadDate) Then
CandRowNextRead = CandReadDate.ToString("dd/MM/yyyy")
End If
End If
Dim CandRowSending As String = TryCast(CandRow("SENDING"), String)
Dim CandRowNotes As String = TryCast(CandRow("NOTES"), String)
For Each NewRow As DataRow In dtUsers.Rows
If CandRow("SQ").ToString = NewRow("SQ").ToString Then
UserLocations.Add(NewRow("SQ").ToString)
End If
Next
For Each ProgressRow As DataRow In dtProgress.Rows
If CandRow("SQ").ToString = ProgressRow("SQ").ToString Then
Dim NextReadDate As Date
Dim ProgressRowNextRead As String = DBNull.Value.ToString
If Not (ProgressRow("NEXT READ").ToString = DBNull.Value.ToString) Then
If Date.TryParse(ProgressRow("NEXT READ").ToString, NextReadDate) Then
ProgressRowNextRead = NextReadDate.ToString("dd/MM/yyyy")
End If
End If
Dim ProgressRowSending As String = TryCast(ProgressRow("SENDING"), String)
Dim ProgressRowNotes As String = TryCast(ProgressRow("NOTES"), String)
If CandRow("SQ").ToString = ProgressRow("SQ").ToString Then
If CandRowSending <> ProgressRowSending Then
BeginInvoke(New UpdateDataTableDelegate(AddressOf UpdateDataTableSending), CandRow, ProgressRowSending)
End If
If CandRowNextRead <> ProgressRowNextRead Then
BeginInvoke(New UpdateDataTableDelegate(AddressOf UpdateDataTableNextRead), CandRow, ProgressRowNextRead)
End If
If CandRowNotes <> ProgressRowNotes Then
BeginInvoke(New UpdateDataTableDelegate(AddressOf UpdateDataTableNotes), CandRow, ProgressRowNotes)
End If
End If
End If
Next
Next
dgv.BeginInvoke(
New MethodInvoker(
Sub()
For Each dgv_row As DataGridViewRow In dgv.Rows
If UserLocations.Contains(dgv_row.Cells("SQ").Value.ToString) Then
dgv.DefaultCellStyle.BackColor = My.Settings.customRowHighlight
Else
dgv.DefaultCellStyle.BackColor = Nothing
End If
Next
End Sub))
Thread.Sleep(My.Settings.customRefreshRate * 1000)
Loop
End Sub
The subs that do the DataTable update are like this:
Private Delegate Sub UpdateDataTableDelegate(ByVal CandRow As DataRow, ByVal ProgressRow As String)
Private Sub UpdateDataTableSending(ByVal CandRow As DataRow, ByVal ProgressRowSending As String)
CandRow("SENDING") = ProgressRowSending
End Sub
I know this is not the best way to handle a multi-user environment but the nature of this work requires that all people can access and see the same data. I could force them to refresh regularly but that seems very intrusive.
The crashes only occur when this thread is running and the crashes are regular (and not instant) but I cannot seem to reproduce them and the application is very stable otherwise.
There must be some cross-threading issue but I can't work how when all of the updates to the DataTable or DataGridView are done via a BeginInvoke on the main UI thread.
EDIT: I've just realised that even though I am doing the queries and most of the heavy lifting in the background thread, the updates are stilled called on the main UI thread which would lock the thread. This would be particularly noticeable if there were a lot of updates... Because each one is called individually.
If the UI lock up was long enough, and the user was clicking on stuff, would this cause Windows to treat the application as unresponsive and crash it? If so, is there a better way I could handle these updates?
Any help with resolving this would be enormously appreciated.

Using Filewatcher for a progress bar with subdirectories, Wont Update Properly

I'm trying to copy Files from a local Computer to a Network device. I'm trying to get a Progress Bar working for the File copy, and got it working for a Single Directory with no Subdirectory:
Private Sub CopyPictures()
Try
If Not Directory.Exists(DestinationPath) Then
My.Computer.FileSystem.CreateDirectory(DestinationPath)
End If
Dim counterLocalFiles = My.Computer.FileSystem.GetFiles(SourcePath)
UpdateProgressBarMaximum1(CInt(counterLocalFiles.Count))
UpdateLabelText2(CStr(counterLocalFiles.Count)) 'is a label which shows copied X files of Label2 Files
fsw1 = New IO.FileSystemWatcher(DestinationPath)
fsw1.EnableRaisingEvents = True
My.Computer.FileSystem.CopyDirectory(SourcePath, DestinationPath)
GetSettingsFromFile()
Catch Exec As System.IO.IOException
Dim dr As DialogResult = MessageBox.Show("Some Random Error Code", "Exception Title", MessageBoxButtons.OKCancel)
If (Not DialogResult.OK = dr) Then
Exit Sub
Return
End If
End Try
End Sub
Private Sub fsw1_Created(sender As Object, e As FileSystemEventArgs) Handles fsw1.Created
Dim counterRemoteFiles = My.Computer.FileSystem.GetFiles(DestinationPath)
UpdateProgressBar1(CInt(counterRemoteFiles.Count))
UpdateLabelText1(CStr(counterRemoteFiles.Count))
End Sub
The Update ObjectX Subs are just invoke Functions since the CopyPictures is raised by a backgroundworker as well looking all like this one for example
Private Sub UpdateProgressBar1(Value As Int32)
If ProgressBar1.InvokeRequired Then
ProgressBar1.Invoke(New Action(Of Integer)(AddressOf UpdateProgressBar1), Value)
Else
'We are on the UI thread so update the control.
ProgressBar1.Value = Value
End If
End Sub
This code works perfectly fine for me, but I have to deal with SubDirectories which contain the Images, and the names of the subs are random so i cant predetermine them so I came up with slight changes:
The Counter is looking now like this:
Dim counterLocalFiles = System.IO.Directory.GetFiles(SourcePath, "*.jpg*", SearchOption.AllDirectories).Length
UpdateProgressBarMaximum1(CInt(counterLocalFiles))
UpdateLabelText2(CStr(counterLocalFiles))
And this:
Dim counterRemoteFiles = IO.Directory.GetFiles(DestinationPath, "*.jpg", SearchOption.AllDirectories).Length
UpdateProgressBar1(CInt(counterRemoteFiles))
UpdateLabelText1(CStr(counterRemoteFiles))
And I added:
fsw1.IncludeSubdirectories = True
Now the weired Problems started: It would properly count the file in the source Directory setting label2 to the correct amount of files in all subdirectories and then start copying. It would NOT update the Progressbar though in real time. It just updated it once when it was done with the first directory and just adding the amount of files to it which it contained. After that it completly stoppedd nored the second directory and didn't add that at all to the progressbar. What am I doing wrong here? I hope my english is fine, If you have any question or If I was not clear enough, please let me know. Thank you
You don't have an event consumer that triggers your progressbar update routine - you call it once when your filesystemwatcher is instantiated.
You need to declare an event that handles the copy event and fires off your progress update code. Because Filesystemwatcher cannot monitor network drives, you may want to declare an event that fires off your progress update method when the counterRemoteFiles count increments.
Turns out I just made a mistake with correctly putting the
fsw1.IncludeSubdirectories = True
I was setting it to true in the Form Editor instead of doing it in the code. Once i actually put that in the code after initialising the fsw, it would work just fine

How do Settings/Options save their selves?

Not sure if my title makes much sense, so I will try to explain my question here. So basically I am expanding my program by allowing things to be customized within it.
Say for example I do this: I click on File -> Options, and a new form is opened with tabs. I have different settings that you can toggle via dropdown box and checkboxes. Now once a user sets the settings they want, or don't want, they click on a button that says either "OK" or "Cancel".
What is the method to saving these settings, or reverting back to the original settings? Do you save via txt file, or is this a default function within a certain line of code?
UPDATE:
So I fixed my previous issue. Now I am having another with the saves. The saves are working good, but I want to use them in selecting my CheckListBox Collection range and also have that range load on start as well. so these are the 2 things that I have been using to do so, that results in adding to the previous, set, collection.
Working for RNG:
Dim rand As New Random()
Dim winners = Enumerable.Range(1, My.Settings.numberSetting).OrderBy(Function(r) rand.Next()).Take(5).ToArray()
Not working for Onload CheckListBox:
Me.LotteryNumbers.Items.Add(1, My.Settings.numberSetting)
If I remove the 1 from Me.LotteryNumbers.Items.Add, the result is this:
This ought not compile:
LotteryNumbers.Items.Add(1, My.Settings.numberSetting)
The overload which takes a second argument expect a Boolean to set the item added to Checked or not. One way is to add items in a loop:
Dim maxNums = My.Settings.numberSetting
' make sure it is empty
clb.Items.Clear()
For n As Int32 = 1 To maxNums
clb.Items.Add(n.ToString)
Next
I don't like using items in Settings as variables, so it grabs the current value to use. Another way uses AddRange:
clb.Items.AddRange(Enumerable.Range(1, maxNums).Select(Function(s) s.ToString()).ToArray())
Items is an collection of Object, so the Select converts to string to add them.
NEVER ORDER BY RANDOM.NEXT()
Mostly you get lucky, but it's not guaranteed. It's only a matter of time before that code blows up on you at run time. The longer the sequence to be sorted, the more likely you are to get an exception.
What you should do instead is implement a Fisher-Yates sort method:
Private rand As New Random()
Public Function Shuffle(Of T)(ByVal items As IList(Of T)) As IList(Of T)
For i As Integer = items.Count - 1 To 1 Step -1
Dim j As Integer = rand.Next(i + 1)
Dim temp As T= items(i)
items(i) = items(j)
items(j) = temp
Next
Return items
End Function
Solution for the working code to update and save checklistbox box count.
Private Sub OptionOkButton_Click(sender As Object, e As EventArgs) Handles OptionOkButton.Click
Main.LotteryNumbers.Items.Clear()
My.Settings.numberSetting = CInt(NumberCombo.Text)
Dim maxNum = My.Settings.numberSetting
Main.LotteryNumbers.Items.AddRange(Enumerable.Range(1, maxNum).Select(Function(s) s.ToString()).ToArray())
My.Settings.Save()
Me.Close()
End Sub

Picturebox location change 20 times per second not redrawing

I'm trying to make a few images do the nice slidey thingy that I've seen lots of Microsoft applications use. The one where the movement starts slow speeds up half way there and then comes to a nice slow stop in it's new location. I've got all the calculations figured out, getting and setting the picture box locations, Confirmation using console.writeline that the image locations are correct, and even a test run that works in a simplified format.
But in the full blown version It's not repainting the image. In fact, it looks like nothing has happened at all while the script is running. I've tried Me.Refresh(), Invalidate(), Timer.Enabled = True/False, and Me.Update(). None of which have worked. The last step is the most frustrating: I'm calling my SetPanelLocation() method at the end to ensure that the panel ends up in the final location regardless of if the movement worked. Nothing happens on this call either, even though immediately after this routine fails I can call the same method from another user event and it starts working again like nothing was wrong.
I'm creating my own PictureBox class called clsFeedImageBox which inherits PictureBox that includes this functionality (along with other features). Each image is only 300x225 pixels so they're not massive images that take a lot of time to redraw. Each instance of this class is in a common Forms.SplitterPanel. I use a lot of comments out of habit so i left them in here, maybe they'll add some light.
Public Class clsFeedImgBox
Inherits PictureBox
Private iRank As Integer 'rank in whatever feed this file gets put in
Private iRankTarget As Integer 'rank to move to when rank feed event starts
Private iTopStart As Integer 'starting top location before feed event
Private iTopTarget As Integer 'final Top location after feed event
Private WithEvents tMyTimer As New System.Timers.Timer
Private WithEvents oParent As FeedBase 'splitter panel, all location info comes from the parent
Public Sub New(ByRef sender As FeedBase, ByVal rank as Integer)
'set objects
oParent = sender
'set .Image property to pre-made thumbnail
Image.FromFile(ThumbPath) 'ThumbPath is a property which is set by this point (some code has been removed)
'setup initial position
setPanelLocation(rank)
'set autosize
Me.SizeMode = PictureBoxSizeMode.StretchImage
'set Image Scroll timer interval to 20 fps (1000 / 20 = 50)
tMyTimer.Interval = 50
End Sub
Public Sub scroll(ByVal newRank As Integer)
'setPanelLocation(newRank) <== this works, timed movements don't
iRankTarget = newRank
iTopStart = Me.Top
iTopTarget = oParent.ImgTop(newRank) 'gets an integer for the new Top location
tMyTimer.Start()
End Sub
Private Sub myScrollStep() Handles tMyTimer.Elapsed
'tMyTimer.Enabled = False 'this idea with the enabled = True at the end didn't work
iTickCount += 1
Dim iScrollPerc As Integer 'scroll % between Start and End * 100
iScrollPerc = oParent.ScrollStep(iTickCount, Rank) 'this part works
Console.WriteLine(strThumbName & " scrollPerc: " & iScrollPerc.ToString)
If iScrollPerc >= 100 Then
'scroll event complete
Console.WriteLine(strThumbName & " SetFinalLocation")
Me.setPanelLocation(iRankTarget) '<== This line doesn't work here, but works when called by other means
'stop Feed updates
tMyTimer.Stop()
'reset iTickCount for next movement
iTickCount = 0
Else
'scrolling still going
Dim newTop As Integer
newTop = Math.Round(iTopTarget - (((100 - iScrollPerc) * (iTopTarget - iTopStart)) / 100)) 'this part works
'Console.WriteLine(strThumbName & " TopTarget: " & newTop)
Me.Top = newTop 'Nothing happens here
End If
'Me.Left = oParent.ImgLeft
'Me.Width = oParent.ImgWidth
'Me.Height = oParent.ImgHeight 'that didn't work
'Me.Refresh() 'this didn't work
'Invalidate() 'this didn't do much good either
'Me.Update() 'Aaaaand no cigar, time for StackOverflow
'tMyTimer.Enabled = True
End Sub
Public Sub setPanelLocation(ByVal rank As Integer)
iRank = rank
Me.MyRePaint()
End Sub
Public Sub MyRePaint()
'repaint image box with everything in it's current rank
Me.Left = oParent.ImgLeft
Me.Top = oParent.ImgTop(iRank)
Me.Width = oParent.ImgWidth
Me.Height = oParent.ImgHeight
End Sub
End Class
What gives? There must be some inner workings of VB.NET that will help me figure this out. I'm using VS 2012 and Win8
You could make a WPF application and use a Slider control instead of "manually" making a slider with planes, picture boxes, etc, etc.

Recursive search for all folders and subfolders in system

I wrote a Windows Forms script that searched for all non-hidden and non-readonly folders in my system. But the script itself, when run initially, runs for like 5 minutes. Subsequent opens take much less time. I was wondering if there is a logical error to it, so as to why its running so very slow.
Private Function FindSubFolders(ByVal dir As DirectoryInfo, ByVal node As TreeNode) As TreeNode
Dim subnode As New TreeNode
For Each folder As DirectoryInfo In dir.GetDirectories()
If (folder.Attributes And FileAttributes.Hidden) <> FileAttributes.Hidden Then
subnode = node.Nodes.Add(folder.FullName, folder.Name)
subnode = FindSubFolders(folder, subnode)
End If
Next
Return subnode
End Function
Private Sub SetFolders_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
'Is it possible to load this on 1st (initial) form load???
Try
Dim node As TreeNode
If TreeView1.Nodes.Count < 1 Then
For Each drive As String In Directory.GetLogicalDrives
Directory.GetLogicalDrives()
Dim folders As DirectoryInfo = New DirectoryInfo(drive)
If (folders.Attributes And FileAttributes.ReadOnly) <> FileAttributes.ReadOnly Then
node = TreeView1.Nodes.Add(drive, drive)
Try
node = FindSubFolders(folders, node)
Catch ex As Exception
Console.WriteLine(ex.Message)
Continue For
End Try
End If
Next
End If
If Not IsNothing(My.Settings.Folders) Then
If ListBox1.Items.Count < 1 Then
For Each col As String In My.Settings.Folders
ListBox1.Items.Add(col)
Next
End If
Else
My.Settings.Folders = New StringCollection
End If
Catch ex As Exception
Logs.Add("04", ex.Message)
End Try
Logs.Add("01", "Loaded.")
End Sub
Thanks for the help! :)
Here are a few tips:
One thing you can do to speed things up is to make sure the TreeView-control does not have to repaint itself every time you add an item to it.
Before adding any item, run Treeview1.BeginUpdate and after you have added all items run Treeview1.EndUpdate
If possible, get the directories as an array, and use the node.addrange to add a whole range of directiryes at once.
From MSDN:
To maintain performance while items
are added one at a time to the
TreeView, call the BeginUpdate method.
The BeginUpdate method prevents the
control from painting until the
EndUpdate method is called. The
preferred way to add items to a tree
view control is to use the AddRange
method to add an array of tree node
items to a tree view. However, if you
want to add items one at a time, use
the BeginUpdate method to prevent the
TreeView control from painting during
the add operations. To allow the
control to resume painting, call the
EndUpdate method when all the tree
nodes have been added to the tree
view.
Check out this question for a (maybe) more easy way to fetch the subfolders:
Get all folder / directories list in VB.net