I should add a list of files into a ZIP. Procedure code is like this
Sub CreateZip
Dim FileList As New ArrayList 'List of File Paths to be added to the ZIP
For Each path in FileList
Try
AddFileToZip(ZipFilePath, path.ToString)
Catch (ex as New Exception)
....
End Try
Next
End Sub
And this is AddFileToZip
Public Sub AddFileToZip(ByVal zipFilename As String, ByVal fileToAdd As String)
Using zip As Package = System.IO.Packaging.Package.Open(zipFilename, FileMode.OpenOrCreate)
Dim destFilename As String = ".\" & Path.GetFileName(fileToAdd)
Dim uri As Uri = PackUriHelper.CreatePartUri(New Uri(destFilename, UriKind.Relative))
If zip.PartExists(uri) Then
zip.DeletePart(uri)
End If
Dim part As PackagePart = zip.CreatePart(uri, "", CompressionOption.Normal)
Using fileStream As New FileStream(fileToAdd, FileMode.Open, FileAccess.Read)
Using dest As Stream = part.GetStream()
CopyStream(fileStream, dest)
End Using
End Using
End Using
End Sub
At runtime, I get this error message
The process cannot access the file [ZipFilePath] because it is being used by another process
this error is raised randomly, maybe on adding small files into the Zip. If I place breakpoints and run procedure in debug mode, it works.
It seems clear that procedure thread is not synchronized with IO, so that my procedure loop continues even if IO is still adding processed file into Zip (ie VB.NET is faster than IO).
I also tried to place a Thread.Sleep(1000) before AddFileToZip, but this may be not enough to synchronize the two processes. Placing Thread.Sleep(2000) seems to make procedure work, but it may slow down dramaticly performances (I should pack more than 50 files into my Zip).
So, how can I force my loop to "wait" until IO Process has released ZIP file?
Related
I am using a legacy application where all the source code is in vb.net. I am checking if the file exists and if the condition is true replace all the " in the contents of the file. For instance "text" to be replaced as text. I am using the below code.
vb.net
Dim FileFullPath As String
FileFullPath = "\\Fileshare\text\sample.txt"
If File.Exists(FileFullPath) Then
Dim stripquote As String = FileFullPath
stripquote = stripquote.Replace("""", "").Trim()
Else
'
End If
I get no errors and at the same time the " is not being replaced in the content of the file.
Data:
ID, Date, Phone, Comments
1,05/13/2021,"123-000-1234","text1"
2,05/13/2021,"123-000-2345","text2"
3,05/13/2021,"123-000-3456","text2"
Output:
1,05/13/2021,123-000-1234,text1
2,05/13/2021,123-000-2345,text2
3,05/13/2021,123-000-3456,text2
You can read each line of the file, remove the double-quotes, write that to a temporary file, then when all the lines are done delete the original and move/rename the temporary file as the filename:
Imports System.IO
'...
Sub RemoveDoubleQuotes(filename As String)
Dim tmpFilename = Path.GetTempFileName()
Using sr As New StreamReader(filename)
Using sw As New StreamWriter(tmpFilename)
While Not sr.EndOfStream
sw.WriteLine(sr.ReadLine().Replace("""", ""))
End While
End Using
End Using
File.Delete(filename)
File.Move(tmpFilename, filename)
End Sub
Add error handling as desired.
The best way to go about this depends on the potential size of the file. If the file is relatively small then there's no point processing it line by line and certainly not using a TextFieldParser. Just read the data in, process it and write it out:
File.WriteAllText(FileFullPath,
File.ReadAllText(FileFullPath).
Replace(ControlChars.Quote, String.Empty))
Only if the file is potentially large and reading it all in one go would require too much memory should you consider processing it line by line. In that case, I'd go this way:
'Let the system create a temp file.
Dim tempFilePath = Path.GetTempFileName()
'Open the temp file for writing text.
Using tempFile As New StreamWriter(tempFilePath)
'Open the source file and read it line by line.
For Each line In File.ReadLines(FileFullPath)
'Remove double-quotes from the current line and write the result to the temp file.
tempFile.WriteLine(line.Replace(ControlChars.Quote, String.Empty))
Next
End Using
'Overwrite the source file with the temp file.
File.Move(tempFilePath, FileFullPath, True)
Note the use of File.ReadLines rather than File.ReadAllLines. The former will only read one line at a time where the latter reads every line before you can process any of them.
EDIT:
Note that this:
File.Move(tempFilePath, FileFullPath, True)
only works in .NET Core 3.0 and later, including .NET 5.0. If you're targeting .NET Framework then you have three other options:
Delete the original file (File.Delete) and then move the temp file (File.Move).
Copy the temp file (File.Copy) and then delete the temp file (File.Delete).
Call My.Computer.FileSystem.MoveFile to move the temp file and overwrite the original file in one go.
TextFieldParser is probably the way to go.
Your code with a few changes.
Static doubleQ As String = New String(ControlChars.Quote, 2)
Dim FileFullPath As String
FileFullPath = "\\Fileshare\text\sample.txt"
If IO.File.Exists(FileFullPath) Then
Dim stripquote As String = IO.File.ReadAllText(FileFullPath)
stripquote = stripquote.Replace(doubleQ, "").Trim()
Else
'
End If
Note the static declaration. I adopted this approach because it confused the heck out of me.
I have some ask to you to check and advice if my code is enough and correct. My application works on files sometimes its downloading multiple files by windows service and then by other windows service parser process those files are parsing. Unfortunately before process file has to be finished downloading before process parser should open the file and play with it therefore i prepared some code to check firstly whether the file is free - means not open by another process or some person, whatever. I use most cases streamreader to open the file and play with it but as i said i would like firstly check file if its ready.
This is my code, what do you think:
Public Shared Function FileInUse(ByVal sFile As String, ByVal log As String) As Boolean
Dim thisFileInUse As Boolean = True
Dim counter As Integer = 0
For i = 0 To 30
counter += 1
If System.IO.File.Exists(sFile) Then
Try
Using f As New IO.FileStream(sFile, FileMode.Open, FileAccess.ReadWrite, FileShare.None)
thisFileInUse = False
Exit For
End Using
Catch
System.Threading.Thread.Sleep(10000) '10 sec
End Try
End If
Next
Return thisFileInUse
End Function
I am making a class that is to help with saving some strings to a local text file (I want to append them to that file and not overwrite so that it is a log file). When I write with the streamwriter to find the end of the previous text, I get an error "the file is not available as it is being used by another process". I looked into this problem on MSDN and I got very little help. I tried to eliminate some variables so I removed the streamreader to check was that the problem and it was. When I tried to write to the file then it worked and I got no error so this made me come to the conclusion that the problem arose in the streamreader. But I could not figure out why?
Here is the code:
Public Sub SaveFile(ByVal Task As String, ByVal Difficulty As Integer, ByVal Time_Taken As String)
Dim SW As String = "C:/Program Files/Business Elements/Dashboard System Files/UserWorkEthic.txt"
Dim i As Integer
Dim aryText(3) As String
aryText(0) = Task
aryText(1) = Difficulty
aryText(2) = Time_Taken
Dim objWriter As System.IO.StreamWriter = New System.IO.StreamWriter(SW, True)
Dim reader As System.IO.StreamReader = New System.IO.StreamReader(SW, True)
reader.ReadToEnd()
reader.EndOfStream.ToString()
For i = 0 To 3
objWriter.WriteLine(aryText(reader.EndOfStream + i))
Next
reader.Close()
objWriter.Close()
End Sub
As Joel has commented on the previous answer it is possible to change the type of locking.
Otherwise building on what Neil has suggested, if to try to write to a file with a new reader it is difficult not to lose the information already within the file.
I would suggest you rename the original file to a temporary name, "UserWorkEthicTEMP.txt" for example. Create a new text file with the original name. Now; read a line, write a line, between the two files, before adding your new data onto the end. Finally Delete the temporary file and you will have the new file with the new details. If you have an error the temporary file will serve as a backup of the original. Some sample code below:
Change file names
Dim Line as string
line=Reader.readline
Do until Line=nothing
objwriter.writeline(line)
line=reader.readline
loop
add new values on the end and remove old file
You are trying to read and write to the same file and this is causing a lock contention. Either store the contents of the file into a variable and then write it back out including your new data to the file.
Psuedo
Reader.Open file
String content = Reader.ReadToEnd()
Reader.Close
Writer.Open file
Loop
Writer.Write newContent
Writer.Close
I have an SSIS package (SQL Server 2005) that loops through a bunch of flat files in a folder. I need to wait until the source application has finished writing the file before I can open it in my flat file import task.
I have a For Each loop container and within it a script task to execute before the Data Flow Task.
When I try to create the success connector between the Script Task and the Data Flow Task I get this error:
Could not create connector. Object reference not set to an instance of
an object.
I get that something is being set to nothing, but I can't see it. I have DelayValidation set to true on both the Script Task and the Data Flow Task. What else am I missing?
I'm a C# guy so maybe I'm missing something obvious in the VB. Here's the script I poached from the interwebs:
Public Sub Main()
Dim strFileName As String = CType(Dts.Variables("FileName").Value, String)
Dim objFS As System.IO.FileStream
Dim bolFinished As Boolean = False
Do
Try
objFS = System.IO.File.Open(strFileName, FileMode.Open, FileAccess.ReadWrite, FileShare.None)
bolFinished = True
objFS.Close()
Catch ex As Exception
System.Threading.Thread.Sleep(1000)
End Try
Loop
If bolFinished Then
Dts.TaskResult = Dts.Results.Success
Else
Dts.TaskResult = Dts.Results.Failure
End If
End Sub
Milen k is more than right. It looks like you have an infinite loop which is opening a file several times until it breaks down.
You could change your code with the below suggested code. This will help you to get out of the infinite loop.
Your current code:
Do
Try
objFS = System.IO.File.Open(strFileName, FileMode.Open, FileAccess.ReadWrite, FileShare.None)
bolFinished = True
objFS.Close()
Catch ex As Exception
System.Threading.Thread.Sleep(1000)
End Try
Loop
Suggested code:
Do While(true)
Try
objFS = System.IO.File.Open(strFileName, FileMode.Open, FileAccess.ReadWrite, FileShare.None)
bolFinished = True
objFS.Close()
Exit Do
Catch ex As Exception
System.Threading.Thread.Sleep(1000)
End Try
Loop
Make sure that you have created a Flat File Source for your Data Flow task. If you do not have an existing one, create a temporary one which act as a place-holder for the file paths you feed it through the For Each loop.
From what I understand, you should be passing the path to each file that you will be importing to your Flat File Connection. This can easily be done by adding the variable generated in your For Each loop as an expression in the Expression property of your Flat File Connection.
UPDATE:
You need to set a condition in your Do ... Loop. For example: Loop While Not bolFinished. Look at this document for more information.
I am using a VB script in SSIS Script Task to add header and Trailer to a flat file. The code was working fine until recently i came across a problem where the rows in the file are more than usual and resulting in a failure on script task with error`Error:
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.String.GetStringForStringBuilder(String value, Int32 startIndex, Int32 length, Int32 capacity)
at System.Text.StringBuilder.GetNewString(String currentString, Int32 requiredLength)
at System.Text.StringBuilder.Append(Char[] value, Int32 startIndex, Int32 charCount)
at System.IO.StreamReader.ReadToEnd()
at System.IO.File.ReadAllText(String path, Encoding encoding)
at System.IO.File.ReadAllText(String path)`
Can any one help me in fixing the problem please.I think instead of "String Builder" i need to use other string related method. I am getting error at
fileContents.Append(File.ReadAllText(Dts.Connections("DestinationConnection").ConnectionString))
Here is my code:
Public Sub Main()
Dim fileContents As New StringBuilder()
Dim finalfile As String
Dim firstline As String
Dim lastline As String
Dts.VariableDispenser.LockForRead("FirstLine")
Dts.VariableDispenser.LockForRead("LastLine")
Dts.VariableDispenser.LockForRead("FileName")
firstline = CType(Dts.Variables("FirstLine").Value, String)
finalfile = CType(Dts.Variables("FileName").Value, String)
lastline= CType(Dts.Variables("LastLine").Value, String)
'Write header, then append file contents and write back out.
fileContents.AppendLine(String.Format("{0}", firstline))
fileContents.Append(File.ReadAllText(Dts.Connections("DestinationConnection").ConnectionString))
fileContents.AppendLine(String.Format("{0}", lastline))
File.WriteAllText(finalfile, fileContents.ToString())
Dts.TaskResult = ScriptResults.Success
End Sub
Well, one simple way would be to just avoid the StringBuilder: open a TextWriter with File.CreateText, write the first line, then write File.ReadAllText(...), then write the final line.
However, that will only save you some memory - it will roughly halve the memory required, as you won't need it in both the StringBuilder and a string (which is what I think will happen now).
A much better alternative would be to:
Open the writer
Write the header line
Open the other file for reading
Loop over the file, reading a chunk of characters at a time and writing it to the new file, until you're done
Close the other file implicitly (use a Using statement for this)
Write the trailing line
Close the write implicitly (use a Using statement)
That way even if you've got huge files, you only need a small chunk of data in memory at a time.
The problem is File.ReadAllText has limitations when it comes to reading a large file because the entire file is read into memory.
What you will need to do is replace the File.ReadAllText with reading the file line by line and append it accordingly.
EDITED FOR AN EXAMPLE:
Option Explicit
Dim oFSO, sFile, oFile, sText
Set oFSO = CreateObject("Scripting.FileSystemObject")
sFile = "your text file"
If oFSO.FileExists(sFile) Then
Set oFile = oFSO.OpenTextFile(sFile, 1)
Do While Not oFile.AtEndOfStream
sText = oFile.ReadLine
If Trim(sText) <> "" Then
fileContents.AppendLine(sText)
End If
Loop
oFile.Close
Else
WScript.Echo "The file was not there."
End If
It's possible you may still have an issue with the fileContents StringBuilder. The original error shown though was thrown from the File.ReadAllText method. So hopefully, this does the trick.
If not, I would just forget about the fileContents StringBuilder all together and write out the header. Then read from the file line by line and write it out line by line, then finally write the footer.
An alternative (and much more SSIS-like) solution would be to create a Data Flow Task that reads your existing file, pipes it through a Script Component that adds the header and footer, and writes it to the file system. Here's what it might look like in SSIS 2005:
The Script Component will be a Transformation with the SynchronousInputID of its output set to False, so that it can generate header and footer rows:
And the VB source of the transform should look something like this:
Public Class ScriptMain
Inherits UserComponent
Dim headerWritten As Boolean = False
Public Overrides Sub IncomingRows_ProcessInputRow(ByVal Row As IncomingRowsBuffer)
If Not headerWritten Then
WriteHeader()
End If
OutgoingRowsBuffer.AddRow()
OutgoingRowsBuffer.theLine = Row.theLine
End Sub
Public Overrides Sub FinishOutputs()
MyBase.FinishOutputs()
WriteFooter()
End Sub
Private Sub WriteHeader()
OutgoingRowsBuffer.AddRow()
OutgoingRowsBuffer.theLine = "The First Header Line"
headerWritten = True
End Sub
Private Sub WriteFooter()
OutgoingRowsBuffer.AddRow()
OutgoingRowsBuffer.theLine = "Here's a footer line"
OutgoingRowsBuffer.AddRow()
OutgoingRowsBuffer.theLine = "Here's another one"
End Sub
End Class
This lets you use the streaming capabilities of SSIS to your advantage.