GetFiles is incredibly slow vb.NET - vb.net

I need to process a large amount of files and scour thru a group of directories and subdirectories and get the file from the found directory. This is easy using a simple directory but when scouring a thousand directories, the program slows down to a crawl. I've heard of using enumeratefiles but I'm not how to approach this. thank you.
How should I approach this?
dim Filename_to_lookfor as string
dim filePaths As String()
for i = 0 to 1000
Filename_to_lookfor = array(i)
filePaths = Directory.GetFiles(sTargetPath, sFilename_to_lookfor.ToUpper,
IO.SearchOption.AllDirectories)
if filepath.length > 0 then
'copy file using Computer.FileSystem.CopyFile()
end if
next
Thank you.

Another coding would be using Linq:
Sub SearchFiles(pathAndfileNames_to_lookfor() As String, sTargetPath As String)
' Call just once Directory.GetFiles():
Dim FilePaths() As String = Directory.GetFiles(sTargetPath, "*.*", IO.SearchOption.AllDirectories)
Dim lookup1 = FilePaths.ToLookup(Function(x) x.ToLower)
Dim lookup2 = pathAndfileNames_to_lookfor.ToLookup(Function(x) x.ToLower)
Dim filesInBoth = lookup1.SelectMany(Function(x) x.Take(lookup2(x.Key).Count).ToArray)
For Each file In filesInBoth
'copy file using Computer.FileSystem.CopyFile()
Console.WriteLine(file)
Next
End Sub
Calling the procedure would be as follows:
Dim file1 As String = "C:\...\file1..."
Dim file2 As String = "C:\...\file2..."
Dim pathAndfileNames_to_lookfor() As String = New String() {file1, file2}
Dim sTargetPath = "..."
SearchFiles(pathAndfileNames_to_lookfor, sTargetPath)

Related

Select files from directory order of there names

I have 'N' number of files in a folder, the file names follows some common procedure(file0....fileN). the file names looks like:
file1.pdf
..
file7.pdf
..
file10.pdf
..
file15.pdf
..
fileN.pdf
Am collecting these files into a list of string using the following code:
Dim Files As String() = Directory.GetFiles(folderBase + "\files", "*.pdf")
here what is the problem am facing is that am getting in the list in the following order:
file1.pdf
file10.pdf
..
file2.pdf
..
file15.pdf
..
fileN.pdf
Getting file10.pdf after file1.pdf. i want to get file names in the sequential order(file1.pdf,file2.pdf...etc)
i have tried this also but it will not solve my problem:
Dim Files As List(Of String) = Directory.GetFiles(folderBase + "\files", "*.pdf").OrderBy(Function(f) New FileInfo(f).Name).ToList()
If you need to use an array then sort function can be used
Dim Files As String() = Directory.GetFiles(folderBase + "\files", "*.pdf")
System.Array.Sort(Of String)(Files)
Here is an approach that uses sorted dictionary. It assumes that the file names are letters followed by number(s). This works by making the file names the same length and using that as the key for the sorted dictionary.
'get the file name only
Dim fnames As IEnumerable(Of String) = From f In Files
Select IO.Path.GetFileNameWithoutExtension(f)
'get the longest file name
Dim max As Integer = fnames.Max(Function(fs) fs.Length)
'a palce for the results
Dim sd As New SortedDictionary(Of String, String)
Dim nums() As Char = "0123456789".ToCharArray
'create dictionary entries.
For ix As Integer = 0 To fnames.Count - 1
Dim fn As String = fnames(ix)
Dim idx As Integer = fn.IndexOfAny(nums)
If idx >= 0 Then fn = fn.Insert(idx, New String(" "c, max - fn.Length))
sd.Add(fn, Files(ix))
Next
Dim l As List(Of String) = sd.Values.Select(Function(f) IO.Path.GetFileName(f)).ToList
When working with paths use the methods in IO.Path, e.g.
Dim Files As String() = IO.Directory.GetFiles(IO.Path.Combine(folderBase, "files"), "*.pdf")
One other thing, use & for concatenation of strings not +.

Renaming all files in a folder

I'm wondering if it's possible to rename all the files in a folder with a simple program, using vb.NET
I'm quite green and not sure if this is even possible.
Lets say there is a folder containing the files:
Text_Space_aliens.txt, fishing_and_hunting_racoons.txt and mapple.txt.
Using a few credentials:
Dim outPut as String = "TextFile_"
Dim fileType as String = ".txt"
Dim numberOfFiles = My.Computer.FileSystem.GetFiles(LocationFolder.Text)
Dim filesTotal As Integer = CStr(numberOfFiles.Count)
Will it be possible to rename these, regardless of previous name, example:
TextFile_1.txt, TextFile_2.txt & TextFile_3.txt
in one operation?
I think this should do the trick. Use Directory.GetFiles(..) to look for specific files. Enumerate results with a for..each and move (aka rename) files to new name. You will have to adjust sourcePath and searchPattern to work for you.
Private Sub renameFilesInFolder()
Dim sourcePath As String = "e:\temp\demo"
Dim searchPattern As String = "*.txt"
Dim i As Integer = 0
For Each fileName As String In Directory.GetFiles(sourcePath, searchPattern, SearchOption.AllDirectories)
File.Move(Path.Combine(sourcePath, fileName), Path.Combine(sourcePath, "txtFile_" & i & ".txt"))
i += 1
Next
End Sub
In your title you state something about chronologically, but within your question you never mentioned it again. So I did another example ordering files by creationTime.
Private Sub renameFilesInFolderChronologically()
Dim sourcePath As String = "e:\temp\demo"
Dim searchPattern As String = "*.txt"
Dim curDir As New DirectoryInfo(sourcePath)
Dim i As Integer = 0
For Each fi As FileInfo In curDir.GetFiles(searchPattern).OrderBy(Function(num) num.CreationTime)
File.Move(fi.FullName, Path.Combine(fi.Directory.FullName, "txtFile_" & i & ".txt"))
i += 1
Next
End Sub
I've never done Lambdas in VB.net but tested my code and it worked as intended. If anything goes wrong please let me know.

VB.NET: GetFiles() in Specific Directory

Currently the line of code looks like this:
Dim files() As String = System.IO.Directory.GetFiles(path, filehead & ".*.*.fsi")
Dim seqfsi() As Integer
ReDim seqfsi(files.GetUpperBound(0))
Dim args() As String
Dim file As String = ""
For Each file In files
args = Split(file, ".")
If args.Length = 4 Then
seqfsi(System.Array.IndexOf(files, file)) = CInt(args(args.GetUpperBound(0) - 1))
End If
The problem is, sometimes, in my case, the path looks something like:
C:\Users\c.brummett\Downloads
and the split causes a split in the username. How can I avoid this problem but still split by periods? I'm sorry I don't how to make this more relatable.
My idea was to use a DirectoryInfo and do something like:
Dim di As DirectoryInfo
di = New DirectoryInfo(path)
Dim files() As String = di.GetFiles(filehead & ".*.*.fsi")
Edit: The problem with this second bit of code, is that it returns the error
Value of type '1-dimensional array of System.IO.FileInfo' cannot be converted to '1-> dimensional array of String' because 'System.IO.FileInfo' is not derived from 'String'.`
You can forget about getting an array of file names (you don't need that anyway) and iterate on the array of FileInfo:
Dim files() As FileInfo = New DirectoryInfo(path).GetFiles(filehead & ".*.*.fsi")
Dim seqfsi() As Integer
ReDim seqfsi(files.GetUpperBound(0))
Dim args() As String
For Each file As FileInfo In files
args = Split(file.Name, ".")
If args.Length = 4 Then
seqfsi(System.Array.IndexOf(files, file)) = CInt(args(args.GetUpperBound(0) - 1))
End If
Note AllDirectories and change in the line doing the splitting. I didn't look at your array structure stuff.
Dim files() As String = System.IO.Directory.GetFiles("C:\temp", "*.doc", IO.SearchOption.AllDirectories)
Dim seqfsi() As Integer
ReDim seqfsi(files.GetUpperBound(0))
Dim args() As String
Dim file As String = ""
For Each file In files
args = file.Substring(file.LastIndexOf("\") + 1).Split(".")
If args.Length = 4 Then
seqfsi(System.Array.IndexOf(files, file)) = CInt(args(args.GetUpperBound(0) - 1))
End If
Next file

Searching By File Extensions VB.NET [duplicate]

This question already has answers here:
What is a NullReferenceException, and how do I fix it?
(27 answers)
Closed 5 years ago.
Hi all i have been trying to search a specified directory and all sub directories for all files that have the specified file extension. However the inbuilt command is useless as it errors up and dies if you dont have access to a directory. So here's what i have at the moment:
Private Function dirSearch(ByVal path As String, Optional ByVal searchpattern As String = ".exe") As String()
Dim di As New DirectoryInfo(path)
Dim fi As FileInfo
Dim filelist() As String
Dim i As Integer = 0
For Each fi In di.GetFiles
If System.IO.Path.GetExtension(fi.FullName).ToLower = searchpattern Then
filelist(i) = fi.FullName
i += 1
End If
Next
Return filelist
End Function
However i get an "System.NullReferenceException: Object reference not set to an instance of an object." when i try to access the data stored inside the filelist string array.
Any idea's on what im doing wrong?
You didn't instantiate the Dim filelist() As String array. Try di.GetFiles(searchPattern)
Dim files() as FileInfo = di.GetFiles(searchPattern)
Use static method Directory.GetFiles that returns an array string
Dim files = Directory.GetFiles(Path,searchPattern,searchOption)
Demo:
Dim files() As String
files = Directory.GetFiles(path, "*.exe", SearchOption.TopDirectoryOnly)
For Each FileName As String In files
Console.WriteLine(FileName)
Next
Recursive directory traversal:
Sub Main()
Dim path = "c:\jam"
Dim fileList As New List(Of String)
GetAllAccessibleFiles(path, fileList)
'Convert List<T> to string array if you want
Dim files As String() = fileList.ToArray
For Each s As String In fileList
Console.WriteLine(s)
Next
End Sub
Sub GetAllAccessibleFiles(path As String, filelist As List(Of String))
For Each file As String In Directory.GetFiles(path, "*.*")
filelist.Add(file)
Next
For Each dir As String In Directory.GetDirectories(path)
Try
GetAllAccessibleFiles(dir, filelist)
Catch ex As Exception
End Try
Next
End Sub
Use System.IO.Directory.EnumerateFiles method and pass SearchOption.AllDirectories in to traverse the tree using a specific search pattern. Here is an example:
foreach (var e in Directory.EnumerateFiles("C:\\windows", "*.dll", SearchOption.AllDirectories))
{
Console.WriteLine(e);
}

Splitting a CSV Visual basic

I have a string like
Query_1,ab563372363_C/R,100.00,249,0,0,1,249,1,249,1e-132, 460
Query_1,ab563372356_C/R,99.60,249,1,0,1,249,1,249,5e-131, 455
in a file
in two separate lines. I am reading it from the textbox. I have to output ab563372363_C/R and ab563372356_C/R in a text box. I am trying to use the split function for that but its not working..
Dim splitString as Array
results = "test.txt"
Dim FileText As String = IO.File.ReadAllText(results) 'reads the above contents from file
splitString = Split(FileText, ",", 14)
TextBox2.text = splitString(1) & splitString(13)
for the above code, it just prints the whole thing.. What's wrong?
Try this
Private Function GetRequiredText() As List(Of String)
Dim requiredStringList As New List(Of String)
Dim file = "test.txt"
If FileIO.FileSystem.FileExists(file) Then
Dim reader As System.IO.StreamReader = System.IO.File.OpenText(file)
Dim line As String = reader.ReadLine()
While line IsNot Nothing
requiredStringList.Add(line.Split(",")(1))
line = reader.ReadLine()
End While
reader.Close()
reader.Dispose()
End If
Return requiredStringList
End Function
This will read the file line by line and add the item you require to a list of strings which will be returned by the function.
Returning a List(Of String) may be overkill, but it's quite simple to illustrate and to work with.
You can then iterate through the list and do what you need with the contents of the list.
Comments welcome!!
Also this might work...
Dim query = From lines In System.IO.File.ReadAllLines(file) _
Select lines.Split(",")(1)
this will return an IEnumerable(Of String)
Enjoy
First
Since you are reading the whole text, your FileText would be ending like this:
Query_1,ab563372363_C/R,100.00,249,0,0,1,249,1,249,1e-132,460
\r\n
Query_1,ab563372356_C/R,99.60,249,1,0,1,249,1,249,5e-131, 455
So when you are referencing to your splitStringwith those indexes (1, 13) your result might probably be wrong.
Second
Try to specify what kind of type your array is, Dim splitString as Array should be Dim splitString As String()
Third
Make your code more readable/maintainable and easy to edit (not only for you, but others)
The Code
Private const FirstIndex = 1
Private const SecondIndex = 12
Sub Main
Dim myDelimiter As Char
Dim myString As String
Dim mySplit As String()
Dim myResult1 As String
Dim myResult2 As String
myDelimiter = ","
myString += "Query_1,ab563372363_C/R,100.00,249,0,0,1,249,1,249,1e-132, 460"
myString += "Query_1,ab563372356_C/R,99.60,249,1,0,1,249,1,249,5e-131, 455"
mySplit = myString.Split(myDelimiter)
myResult1 = mySplit(FirstIndex)
myResult2 = mySplit(SecondIndex)
Console.WriteLine(myResult1)
Console.WriteLine(myResult2)
End Sub