I am creating an application whereby I need to write files (many, many of them). However, I don't want to write files that are too big because the program that consumes them will crash :)
Essentially, I would like to create files that are no larger than a specific size (possibly 5MB). My data is of CSV format and I will be writing line by line or into some stringbuilder to create the file I need.
Question: Is there a way to write data and create a set of files that are no larger than a set size?
I assume you are creating CSV rows 1 at a time.
If you are using a StreamWriter on the file (hopefully you are, no need to hold it all in memory):
Create the new record (a string of CSV items).
Look at the StreamWriter's BaseStream.Length property (Caveat - You need to use AutoFlush to make that property correct with the data that's been written previously).
Add your stream position, new record's length, and your line separator's length (usually 1 or 2, depending on Cr, Lf, or CrLf), and see if it exceeds your threshold.
If no, write the line feed, the record, and continue. If yes, close your StreamWriter and open a new one for a new file, write the record, and continue.
Related
what I need to accomplish: I need to read a text file that gets updated every 10 sec and dump the new data into. I need to find the best way to do this. I've tried streamreader and file.readalllines, I can get the contents of the txt file, but I don't know how to compare and dump the added lines. Any input would be appreciated, thank you.
Under the assumptions that this is a log file you're reading, where the data is being appended to end, and you're reading the file every 10 seconds, why don't you keep track of the number of lines read or the last line number from the last read?
You can then check to see whether the current read has exceeded the previous line number or number of lines read. If it has exceeded, you should be able to work out what the new lines were added because you know where you read until previously. You can read from the (previous read line + 1) until the end of file.
This might be something that I am overlooking, as I am currently reading a book on VB.NET. I canceled a course in VB.Net recently because I was stuck on a problem and the lecturer did not get back to me with my questions that I had.
The objective was to create a business application with VB.NET that writes data from text boxes to a text file, seperated by a comma or a pipe. The user of the application must be able to select the directory where they want the file to be saved.
So I have for example 2 forms, one that captures the data for a client, and another form where you can select from a drop-down control, Now I know that streamwriter allows for the user to select self where they want the file to be saved, but how do I make the second form intelligent to know where the user saved the form and then reads the client_id, and fills the other data associated with the client_id to the text boxes in the form. I know streamreader is the one to use when you want to read data from a file, but how will streamreader know where the user will save the file to?
I am not doing the course any more, but I will keep on thinking what I could have done to actually get the project to work.
For writing to file
// Write single line to new file.
Using writer As New StreamWriter("C:\log.txt", True)
writer.WriteLine("Important data line 1")
End Using
For reading from file
// read from a file
Dim line As String
Using reader As New StreamReader("file.txt")
line = reader.ReadLine()
End Using
Console.WriteLine(line)
I am trying to code a program which reads a file, which will contain many words (one word per line), then selects a random line (word) from the file, so I am able to store it in a variable for me to use later on.
I don't really know where to start as I am not very experienced. Any help would be appreciated.
Well first you will need to open the file
file = open('filename.txt', 'w')
Then you need to read the file you can read each line into a list by doing words = file.readlines (this can also be done with a loop or in a number of other ways)
Then you can use the random module to generate a random number and get the word from that index in the words list. Then just store that word to a variable.
There are other ways of doing this but this is one of the easiest.
I have a large file which i am writing to a smaller file in VB, i just dont know how to select lines from an input file that i opened.
I would like to keep the first 12 lines of the large file and them copy every 3rd line into the new output.
Any help?!
You can do this with a StreamReader - just open one on the file, then you can call ReadLine() on it as many times as you need.
Easiest implementation would probably be a 0-to-11 (or 1-to-12) For loop, then a While where you read 2 and ignore them, then read the 3rd and write it.
Writing the new file can be done with StreamWriter, which just has a WriteLine() method to write the text.
The StreamReader and StreamWriter are generally the easiest ways to read and write text files.
in My.Settings i've alot of variables including a 36000 records Datatable and Couple Large Array Lists, now when i call My.Settings.Save() it saves all the variables which takes alot of time and it saves variables that haven't been changed, is there a way make it Save only a Specific Datatable or ArrayList?
My.Setting.Save()
'but what i want for example is something like
My.Settings.SomeDatatable.Save()
These settings are stored in an XML file. Since XML files do not store data in a fixed-width record format, there is no way to update a single node in an XML file without rewriting the whole thing. If it is too slow, I would recommend either:
Store these settings in multiple XML files, so that you only have to save the applicable file, but not the rest
Store the settings in the database where each row can be saved individually without rewriting the whole file
Databases store their data in fixed-width "pages" so that only the modified pages need to be rewritten, rather than the whole file.
What about saving your dataTable not within your Settings. You could make use of DataTable.WriteXml(). this would keep your settings much smaller and you can access them separately!
' save table
DataTable.WriteXml("yourFile.xml")
' load table
Dim newTable As New DataTable
newTable.ReadXml(fileName)
But if there is no other possibility this solution from SLaks will work too
' save table
Dim writer As New StringWriter()
table.WriteXml(writer)
My.MySettings.Default.TableXml = writer.ToString()
' load table
Dim reader As New StringReader(My.MySettings.Default.TableXml)
table.ReadXml(reader)