recursive reading for files in folder using vb.net - vb.net

I wrote a program which is not checking the file update time but its not checking recursive folder files.kindly help for recursive folder files as well.
My code is here :
Sub getfilestat1()
Dim fileName As String
Dim CurrCyleTime As Date
Dim PrevCycleTime As Date
Dim DBCycleTime As Date
Dim connectionString As String, sql As String
Dim _SQLConnection As AseConnection
Dim _SQLCommand As AseCommand
Dim _SQLAdapter As AseDataAdapter
Dim _DataSet As DataSet
Dim _SQLReader As AseDataReader
_SQLConnection = New AseConnection
_SQLCommand = New AseCommand
_SQLConnection.ConnectionString = "Data Source='10.49.196.97';Port=9713;Database=db_print;Uid=kuat199;Pwd=testing1; "
_SQLCommand.Connection = _SQLConnection
_SQLCommand.CommandText = ""
_SQLCommand.CommandType = CommandType.Text
_SQLCommand.CommandTimeout = 900000000
_SQLConnection.Open()
Dim command As New AseCommand("select * from Kampachi_Cycle", _SQLConnection)
Dim reader As AseDataReader = command.ExecuteReader()
While reader.Read()
' Console.WriteLine(reader("pol_no").ToString() & " " & Convert.ToString(reader("image_return")) & " " & Convert.ToString(reader("no_of_images")))
DBCycleTime = reader("CYCLE").ToString()
End While
' Dim asSettings As AppSettingsSection = cAppConfig.AppSettings
'Dim fi As New System.IO.DirectoryInfo("D:\Vimal\test")
Dim fi As New System.IO.DirectoryInfo("\\kaip3r7ciwf01\BicorData\report\kam\")
Dim files = fi.GetFiles("*", SearchOption.AllDirectories).ToList()
'For Each filename As String In IO.Directory.GetFiles(Directory, "*", IO.SearchOption.AllDirectories)
'For Each file In files Select file Order By file.CreationTime Descending
''Dim first = (From file In files Select file Order By file.CreationTime Ascending).FirstOrDefault
'Count the number files in network path
Dim fcount = files.Count()
'Fetching the previous cycle run time from config file
PrevCycleTime = ConfigurationManager.AppSettings("PrevCycleTime")
CurrCyleTime = Now()
ConfigurationManager.AppSettings("PrevCycleTime") = CurrCyleTime
''''My.Settings.Save()
For i As Integer = 0 To fcount - 1
If files(i).LastWriteTime > DBCycleTime.AddMinutes(-20) Then
fileName = files(i).Name.ToString()
Dim insertCmd As New AseCommand("INSERT INTO Kampachi_FilesProcess " + " ( FILENAME, FileReadStatus) " + " VALUES( #file_name, #read_stat )", _SQLConnection)
Dim parm As New AseParameter("#file_name", AseDbType.VarChar, 1000)
insertCmd.Parameters.Add(parm)
parm = New AseParameter("#read_stat", AseDbType.VarChar, 12)
insertCmd.Parameters.Add(parm)
Dim recordsAffected As Integer
insertCmd.Parameters(0).Value = fileName
insertCmd.Parameters(1).Value = "Y"
recordsAffected = insertCmd.ExecuteNonQuery()
If i = 0 Then
fileName = files(i).Name.ToString()
Dim updCmd As New AseCommand("update Kampachi_Cycle set CYCLE = Getdate()", _SQLConnection)
Dim updparm As New AseParameter("#file_name", AseDbType.VarChar, 1000)
recordsAffected = updCmd.ExecuteNonQuery()
End If
End If
Next
End Sub
After these changes it looks fine and giving out properly.
It is giving recursive reading as well.

Change this line:
Dim files = fi.GetFileSystemInfos.ToList()
To:
Dim files = fi.GetFiles("*", SearchOption.AllDirectories).ToList()
To answer below question about the If not checking all of the files: You are correct, but your code explicitly used the FirstOrDefault method so it would only ever examine the first file. I don't know what you're doing with the rest of your program here, and your question didn't specify, but the above answered your question about recursive file searching.
To get a list of all the files that are older than 25 minutes use this code:
Dim files As List(Of FileInfo) = fi.GetFiles("*", SearchOption.AllDirectories).ToList
Dim oldFileTimeStamp As DateTime = DateTime.Now.AddMinutes(-25)
Dim olderFiles As List(Of FileInfo) = files.Where(Function(fi2) fi2.LastWriteTime > oldFileTimeStamp).ToList()
Please, if this answered this specific question, please click the accepted answer button. If you have additional questions, unrelated to the original question, please open a new Stackoverflow question, and do not add new questions to an existing Stackoverflow question. This makes it easier for future viewers to find answers to your follow up question(s) (ie: search won't find questions inside of question, it only finds the original question).

Related

How to convert encoding of FTP Getlisting array of strings?

I am using the following vb code to get the list of files in a ftp directory and populate a database table with it to be used in another integration process. Please forgive my bad bad programming skills (I am not a vb.net developer).
Public Sub Main()
Dim StrFolderArrary As String() = Nothing
Dim StrFileArray As String() = Nothing
Dim fileName As String
Dim RemotePath As String
RemotePath = Dts.Variables("User::FTPFullPath").Value.ToString()
Dim ADODBConnection As SqlClient.SqlConnection
ADODBConnection = DirectCast(Dts.Connections("DB_Connection").AcquireConnection(Dts.Transaction), SqlClient.SqlConnection)
Dim cm As ConnectionManager = Dts.Connections("FTP_Connection") 'FTP connection manager name
Dim ftp As FtpClientConnection = New FtpClientConnection(cm.AcquireConnection(Nothing))
ftp.Connect() 'Connecting to FTP Server
ftp.SetWorkingDirectory(RemotePath) 'Provide the Directory on which you are working on FTP Server
ftp.GetListing(StrFolderArrary, StrFileArray) 'Get all the files and Folders List
'If there is no file in the folder, strFile Arry will contain nothing, so close the connection.
If StrFileArray Is Nothing Then
ftp.Close()
'If Files are there, Loop through the StrFileArray arrary and insert into table
Else
For Each fileName In StrFileArray
'MessageBox.Show(fileName)
Dim SQLCommandText As String
SQLCommandText = "INSERT INTO dbo._FTPFileList ([DirName],[FileName]) VALUES (N'" + RemotePath + "', N'" + fileName + "')"
'MessageBox.Show(SQLCommandText)
Dim cmdDatabase As SqlCommand = New SqlCommand(SQLCommandText, ADODBConnection)
cmdDatabase.ExecuteNonQuery()
Next
ftp.Close()
End If
' Add your code here
'
Dts.TaskResult = ScriptResults.Success
End Sub
It works fine and I get the results in the database table. The problem is that the encoding of the strings coming from FTP makes the file names with accentuation to be written incorrectly as shown in the example below.
database table
The correct file name is Razão and I know that the db collation is correct since it can be written like this.
So I tried to convert the strings using this code for each file name in the string array but without any success.
For Each fileName In StrFileArray
Dim utf8 As UTF8Encoding = New UTF8Encoding(True, True)
Dim bytes As Byte() = New Byte(utf8.GetByteCount(fileName) + utf8.GetPreamble().Length - 1) {}
Array.Copy(utf8.GetPreamble(), bytes, utf8.GetPreamble().Length)
utf8.GetBytes(fileName, 0, fileName.Length, bytes, utf8.GetPreamble().Length)
Dim fileName2 As String = utf8.GetString(bytes, 0, bytes.Length)
I believe it is coming with different encoding from the FTP side so I would like to know how to convert the strings during the GetListing method.
Or do you have any ideas how to deal with this?
Thanks in advance.
edit:
I also tried the following code without success.
Dim utf8 As Encoding = Encoding.UTF8
Dim w1252 As Encoding = Encoding.GetEncoding(1252)
Dim w1252Bytes As Byte() = w1252.GetBytes(fileName)
Dim utf8Bytes As Byte() = Encoding.Convert(w1252, utf8, w1252Bytes)
Dim utf8Chars As Char() = New Char(utf8.GetCharCount(utf8Bytes, 0, utf8Bytes.Length) - 1) {}
utf8.GetChars(utf8Bytes, 0, utf8Bytes.Length, utf8Chars, 0)
Dim fileName2 As String = New String(utf8Chars)

vb.net Splitting string lines into new strings

I want to split a string - which includes multiple lines - into new strings.
As it seems that people dont understand my problem here some further informations:
I read out values into strings from a XML-file. Some of those strings countain multiple lines. Now I need every single value of that string on a new string(variable) so that I can tell Homer to drink a beer and tell Lenny to go to bed and not tell the whole Team to go to bed. (Hopefully this story helps you :D )
To keep this simple I'll define a "static" string for this sample.
I'll put 3 of my tries down below. I'd love to hear what's wrong with them. I also tried it with lists and enums where I could split the string but no define a new one..
But I assume that there is a much easier solution for my problem...
Dim team As String = "Simpson, Homer" & vbCrLf & "Leonard, Lenny" & vbCrLf & "Carlson, Carl"
1.
Dim objReader As New StringReader(team)
Dim tm() As String
Dim i As Integer = 1
Do While objReader.Peek() <> -1
tm(i) = objReader.ReadLine() & vbNewLine
i = i + 1
Loop
Dim i As Integer = 0
For Each Line As String In team.Split(New [Char]() {CChar(vbTab)})
Dim tm(i) As String = ReadLine(team, i)
i = i + 1
Next
3.
Dim tm() As String
Dim i As Integer = 0
Dim objReader As New StringReader(team)
Do While objReader.Peek() <> -1
tm(i) = ReadLine(team, i)
i = i + 1
Loop
And the function used in 2. and 3.
Public Function ReadLine(ByVal sFile As String, Optional ByVal nLine As Long = 1) As String
Dim sLines() As String
Dim oFSO As Object
Dim oFile As Object
On Error GoTo ErrHandler
oFSO = CreateObject("Scripting.FileSystemObject")
If oFSO.FileExists(sFile) Then
oFile = oFSO.OpenTextFile(sFile)
sLines = Split(oFile.ReadAll, vbCrLf)
oFile.Close()
Select Case Math.Sign(nLine)
Case 1
ReadLine = sLines(nLine - 1)
Case -1
ReadLine = sLines(UBound(sLines) + nLine + 1)
End Select
End If
ErrHandler:
oFile = Nothing
oFSO = Nothing
End Function
Thanks in advance for any shared thoughts.
There is in fact an easy solution for my problem. Sorry if I caused confusion.
Module Module1
Dim team As String = "Simpson, Homer" & vbCrLf & "Leonard, Lenny" & vbCrLf & "Carlson, Carl"
Sub Main()
Dim tm As String() = team.Split(vbLf)
'Test
Console.WriteLine(tm(0)) 'Homer
Console.WriteLine(tm(1)) 'Lenny
Console.WriteLine(tm(2)) 'Carl
End Sub
End Module

How can I improve the efficiency of my simple file-splitting program

I have a simple program that reads a .txt file, and then splits it up into many files of "pMaxRows" number of rows. These .txt files are huge - some are nearly 25Gb. Right now it is not running fast enough for my liking, I feel that there should be a way to improve the efficiency by maybe reading/writing multiple lines at once, but I am not very experienced with vb.net streamreader/streamwriter.
Code is below:
Public Sub Execute(ByVal pFileLocation As String, _
ByVal pMaxRows As Int32)
Dim sr As IO.StreamReader
Dim Row As String
Dim SourceRowCount As Int64
Dim TargetRowCount As int64
Dim TargetFileNumber As Int32
''Does the file exist in that location?
If IO.File.Exists(pFileLocation) = False Then
Throw New Exception("File does not exist at " & pFileLocation)
End If
''Split FileLocation into FileName and Folder Location
Dim arrFileLoc() As String = pFileLocation.Split("\")
Dim i As Integer = arrFileLoc.Length - 1
Dim FileName As String = arrFileLoc(i)
Dim FileLocationLength As Integer = pFileLocation.Length
Dim FileNameLength As Integer = FileName.Length
Dim Folder As String = pFileLocation.Remove(FileLocationLength - FileNameLength, FileNameLength)
''Read the file
sr = New IO.StreamReader(pFileLocation)
SourceRowCount = 0
TargetRowCount = 0
TargetFileNumber = 1
''Create First Target File Name
Dim TargetFileName As String
TargetFileName = TargetFileNumber & "_" & FileName
''Open streamreader and start reading lines
Do While Not sr.EndOfStream
''if it hits the target number of rows:
If (TargetRowCount = pMaxRows) Then
''Advance target file number
TargetFileNumber += 1
''Create New file with target file number
TargetFileName = TargetFileNumber & "_" & FileName
''Set target row count back to 0
TargetRowCount = 0
End If
''Read line
Row = sr.ReadLine()
''Write line
Using sw As New StreamWriter(Folder & TargetFileName, True)
sw.WriteLine(Row)
End Using
SourceRowCount += 1
TargetRowCount += 1
Loop
End Sub
Anyone have any suggestions? Even directing me to the right place if this has been answered before would be much appreciated

Combine multiple txt files alternately (vb.net)

Let me explain it on an excel sheet. I have few txt files in directory (f.txt, d.txt, s.txt, a.txt and q.txt). Each file has few lines of text. And I want to combine those files but in specific way - it is shown on screenshot.
and output should be:
I've already made a code but it doesn't work - I don't know why.
Dim fileEntries As String() = Directory.GetFiles("D:\dir\", "*.txt")
' Process the list of .txt files found in the directory. '
Dim i As Integer = 0
Dim filesCount As Integer = Directory.GetFiles("D:\dir\", "*.txt").Count
Do Until i = filesCount
'do it for every file in folder'
i = i + 1
Dim reader As New System.IO.StreamReader(fileEntries(i))
Dim files() As String = reader.ReadToEnd.Split({Environment.NewLine}, StringSplitOptions.RemoveEmptyEntries)
Dim lineCount = File.ReadAllLines(fileEntries(i)).Length
Dim w As Integer = 0
Dim dt As DataTable
dt.Columns.Add(i)
'add column "0" for file 1, "1" for file 2 etc.'
Do Until w = lineCount
dt.Rows.Add(files(w))
'write each line in file 1 to column 0, etc.'
w = w + 1
Loop
Loop
Can somebody help me?
Read/write
If your goal is as shown in the last image, write back to a file named output.txt, then this can be done in a single line of code.
My.Computer.FileSystem.WriteAllText("D:\dir\output.txt", String.Join(Environment.NewLine, (From path As String In Directory.GetFiles("D:\dir", "*.txt") Where IO.Path.GetFileNameWithoutExtension(path) <> "output" Select My.Computer.FileSystem.ReadAllText(path, Encoding.UTF8))), False, Encoding.UTF8)
You can of course make this a bit more readable if you don't like one-liners.
My.Computer.FileSystem.WriteAllText(
"D:\dir\output.txt",
String.Join(
Environment.NewLine,
(
From
path As String
In
Directory.GetFiles("D:\dir", "*.txt")
Where
IO.Path.GetFileNameWithoutExtension(path) <> "output"
Select
My.Computer.FileSystem.ReadAllText(path, Encoding.UTF8)
)
),
False,
Encoding.UTF8
)
Iterate
If you need to iterate each line and/or each file, store the result in a local variable.
Dim files As IEnumerable(Of String()) = (
From
path As String
In
Directory.GetFiles("D:\dir", "*.txt")
Select
My.Computer.FileSystem.ReadAllText(path, Encoding.UTF8).Split({Environment.NewLine}, StringSplitOptions.RemoveEmptyEntries)
)
For Each file As String() In files
For Each line As String In file
Next
Next
DataSet
If you need to create a DataSet from the result, then take advantage of anonymous types. This way you can store both the name of the file and its lines.
Dim files = (
From
path As String
In
Directory.GetFiles("D:\dir", "*.txt")
Select
New With {
Key .Name = IO.Path.GetFileNameWithoutExtension(path),
.Lines = My.Computer.FileSystem.ReadAllText(path, Encoding.UTF8).Split({Environment.NewLine}, StringSplitOptions.RemoveEmptyEntries)
}
)
Dim data As New DataSet()
With data
.BeginInit()
For Each item In files
With data.Tables.Add(item.Name)
.BeginInit()
.Columns.Add("Column1", GetType(String))
.EndInit()
.BeginLoadData()
For Each line As String In item.Lines
.Rows.Add(line)
Next
.EndLoadData()
End With
Next
.EndInit()
End With
There are few problems in your code:
Your datatable was not initialized
value of w is exceed than the size of files array
Note: I use DataSet to add each DataTable, However you can remove it if it's not required.
Try following code:
Dim fileEntries As String() = Directory.GetFiles("C:\dir\", "*.txt")
' Process the list of .txt files found in the directory. '
Dim filesCount As Integer = Directory.GetFiles("C:\dir\", "*.txt").Count()
Dim ds As New DataSet()
For i As Integer = 0 To filesCount - 1
'do it for every file in folder'
i = i + 1
Dim reader As New System.IO.StreamReader(fileEntries(i))
Dim files As String() = reader.ReadToEnd().Split(New String() {Environment.NewLine}, StringSplitOptions.RemoveEmptyEntries)
Dim lineCount = File.ReadAllLines(fileEntries(i)).Length
Dim w As Integer = 0
Dim dt As New DataTable()
dt.Columns.Add(i.ToString())
'add column "0" for file 1, "1" for file 2 etc.'
While w <> lineCount
If files.Length = w AndAlso w <> 0 Then
Continue While
End If
dt.Rows.Add(files(w))
'write each line in file 1 to column 0, etc.'
w = w + 1
End While
ds.Tables.Add(dt)
Next

streamwriter in windows store app - write text to file in VB

Does anybody have VB code to emulate Streamwriter for Windows Store?
I know it's been replaced by StorageFolder class but there is no VB sample in MSDN and I can't seem to translate properly from c# examples. Any help would be appreciated. I am just trying to write text (CSV) to a file and save it to the documents folder. In the code below windows store want a stream instead of strPath when I try dim-ing a streamwriter. (been playing with pickerdialog too, but that might be the next hurdle).
Dim strpath As String = Windows.Storage.Pickers.PickerLocationId.DocumentsLibrary & "\" & strFileName
'Build String for file*******************
Dim swExport As StreamWriter = New StreamWriter(strpath)
swExport.Flush()
For x = 0 To intCount - 1
strLine = "WriteSomeText"
swExport.WriteLine(strLine)
Next x
Possibly the simplest approach would be to use a MemoryStream if you like StreamWriter, so something like:
Dim sessionData As New MemoryStream()
' TODO: stage data in sessionData
Dim swExport As StreamWriter = New StreamWriter(sessionData)
swExport.Flush()
For x = 0 To intCount - 1
strLine = "WriteSomeText"
swExport.WriteLine(strLine)
Next x
Dim file As StorageFile = await ApplicationData.Current.RoamingFolder.CreateFileAsync("towns.json", CreationCollisionOption.ReplaceExisting)
Using (fileStream As Stream = await file.OpenStreamForWriteAsync())
sessionData.Seek(0, SeekOrigin.Begin)
await sessionData.CopyToAsync(fileStream)
await fileStream.FlushAsync()
End Using
I was making it too difficult. To write to a file I just needed to use storagefolder and storagefile. I have also included the FileSavePicker in the code (note that filetypechoices is mandatory)
Private Async Function btnExport_Click(sender As Object, e As RoutedEventArgs) As Task
'Calls Filepicker to determine location
'Calls Sqlite to select ALL
'Creates CSV file to be saved at location chosen
'save to file
Dim intCount As Integer = "5"
Dim x As Integer
Dim strLine As String 'hold each line for export file
'Create FileName based on date
Dim strDate As String = Date.Today.ToString("MMddyyyy")
Dim strFileName As String = "Export" & strDate & ".csv"
' Configure save file dialog box
Dim dlgPicker As New Windows.Storage.Pickers.FileSavePicker
'add types for picker (manditory field)
Dim types = New List(Of String)
types.Add(".csv")
types.Add(".txt")
'set picker parameters
dlgPicker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.Downloads
dlgPicker.SuggestedFileName = strFileName '"Document" '
dlgPicker.FileTypeChoices.Add("CSV/TXT", types) 'manditory
dlgPicker.DefaultFileExtension = ".csv" 'Filter files by extension
dlgPicker.CommitButtonText = "Save"
' Show save file dialog box
Dim SaveCSV = Await dlgPicker.PickSaveFileAsync()
'************************get data************
Dim sbExport As Text.StringBuilder = New Text.StringBuilder
sbExport.AppendLine(strHeader)
For x = 0 To intCount - 1
strLine = "Get the text you want to write here"
sbExport.AppendLine(strLine)
Next x
'************************************
'write data to file
Await FileIO.WriteTextAsync(SaveCSV, sbExport.ToString)
Dim mb As MessageDialog = New MessageDialog("Done")
Await mb.ShowAsync()
End Function