The way this file works is there is a null buffer, then a user check sum then a byte that gives you the user name letter count, then a byte for how many bytes to skip to the next user and a byte for which user file the user keeps their settings in.
the loop with the usersm variable in the IF statement sets up the whole file stream for extraction. However with almost the exact same code the else clause specifically the str.Read(xnl, 0, usn - 1) in the else code appears to be reading the very beginning of the file despite the position of the filestream being set earlier, anyone know whats happening here?
this is in vb2005
Private Sub readusersdata(ByVal userdatafile As String)
ListView1.BeginUpdate()
ListView1.Items.Clear()
Using snxl As IO.Stream = IO.File.Open(userdatafile, IO.FileMode.Open)
Using str As New IO.StreamReader(snxl)
str.BaseStream.Position = 4
Dim usersm As Integer = str.BaseStream.ReadByte()
Dim users As Integer = usersm
While users > 0
If usersm = users Then
Dim trailtouser As Integer = 0
str.BaseStream.Position = 6
Dim ust As Integer = str.BaseStream.ReadByte()
str.BaseStream.Position = 8
Dim snb(ust - 1) As Char
str.ReadBlock(snb, 0, ust)
Dim bst = New String(snb)
If usersm = 1 Then
str.BaseStream.Position = 16
Else
str.BaseStream.Position = 15
End If
cLVN(ListView1, bst, str.BaseStream.ReadByte)
str.BaseStream.Position = 8 + snb.Length
str.BaseStream.Position += str.BaseStream.ReadByte + 1
Else
Dim usn As Integer = str.BaseStream.ReadByte
str.BaseStream.Position += 2
Dim chrpos As Integer = str.BaseStream.Position
Dim xnl(usn - 1) As Char
str.Read(xnl, 0, usn - 1)
Dim skpbyte As Integer = str.BaseStream.ReadByte
str.BaseStream.Position += 3
Dim udata As Integer = str.BaseStream.ReadByte
End If
users -= 1
End While
End Using
End Using
ListView1.EndUpdate()
End Sub
When you change the position of the underlying stream, the StreamReader doesn't know you've done that. If it's previously read "too much" data (deliberately, for the sake of efficiency - it tries to avoid doing lots of little reads on the underlying stream) then it will have buffered data that it'll use instead of talking directly to the repositioned stream. You need to call StreamReader.DiscardBufferedData after repositioning the stream to avoid that.
Related
I searched everywhere on stackoverflow.com but still not have a solution:
My code is very simple :
'Get Full Path of File
Dim elements As String = Path.GetTempPath() & "file.txt"
'Create A new folder for outputs if not exist
If (Not System.IO.Directory.Exists(Path.GetTempPath() & "folder")) Then
System.IO.Directory.CreateDirectory(Path.GetTempPath() & "folder")
End If
I want divide the file.txt with contents to (x) number of files inside the new folder
example :
if x = 3
the output files will be created automatically:
/folder/file_1.txt
/folder/file_2.txt
/folder/file_3.txt
Read the data from the existing text file.
Divide the data into an arbitrary number of strings of arbitrary length.
If the new directory doesn't exist create it.
Create files in this directory to store the arbitrary sections of data until an arbitrary number of files have been created and all data has been stored.
Since you refuse to provide the logic (I guess it is a secret), I will have to make some assumptions.
The original file contains lines that are approximately the same length.
The intent is divide the original file into files of approximately equal size.
The number of files is based on the size of the original file.
I split the files based on lines so a word would not be split between 2 files.
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Dim OriginalFilePath = "C:/devlist.txt" '16KB file
Dim lines = File.ReadAllLines(OriginalFilePath)
Dim NumOfLines = lines.Length
Dim NumOfFiles As Integer = GetNumberOfFiles(OriginalFilePath, NumOfLines)
If NumOfFiles = -1 Then
MessageBox.Show("No data in file.")
Return
End If
'The "\" operator is integer division
Dim LinesPerFile = NumOfLines \ NumOfFiles
'Dim LeftoverLines = NumOfLines Mod NumOfFiles - didn't need this afterall
Directory.CreateDirectory("C:\Some Directory")
Dim StartIndex As Integer
Dim EndIndex As Integer
Dim sb As New StringBuilder
For i = 0 To NumOfFiles - 1
EndIndex = StartIndex + LinesPerFile
If EndIndex >= NumOfLines - 1 Then
EndIndex = NumOfLines - 1
End If
For index = StartIndex To EndIndex
sb.AppendLine(lines(index))
Next
Dim NewFilePath = $"C:\Some Directory\SplitFile{i.ToString}.txt"
'.WriteAllText will create the new file or overwrite it if it exists
File.WriteAllText(NewFilePath, sb.ToString)
StartIndex = EndIndex + 1
sb.Clear()
Next
End Sub
Private Function GetNumberOfFiles(FilePath As String, NumOfLines As Integer) As Integer
Dim OriginalFileLength = New FileInfo(FilePath).Length
Dim NumOfFiles As Integer
Select Case OriginalFileLength
Case 0
MessageBox.Show("No data in file")
Return -1
Case 1
NumOfFiles = 1
Case 2
If NumOfLines < 2 Then
NumOfFiles = 1
End If
NumOfFiles = 2
Case 3 To 10_000
If NumOfLines < 3 Then
NumOfFiles = NumOfLines
Else
NumOfFiles = 3
End If
'You can continue the If statements but I assumed
'a file of this size would have at least 4 lines
Case 10_001 To 100_000
NumOfFiles = 4
Case 100_001 To 500_000
NumOfFiles = 5
Case Else
NumOfFiles = 6
End Select
Return NumOfFiles
End Function
Result of splitting the 16KB file
This just sample
Dim Divider = 3
Dim myNewFile(Divider - 1) As String
Dim fileReader As String
fileReader = My.Computer.FileSystem.ReadAllText("C:\test.txt")
Dim myNewSize As Long = 0
Long.TryParse(fileReader.Length / Divider, myNewSize)
If myNewSize = 0 Then
MessageBox.Show("Can't Be Processed")
Exit Sub
End If
For myCnt As Int16 = 0 To divider - 1
myNewFile(myCnt) = fileReader.Substring(myCnt * myNewSize + 1, myNewSize)
Next
'Resize The Last For Include The Remain
Dim myNewLastSize As Long = myNewSize + fileReader.Length - myNewSize * Divider
myNewFile(Divider - 1) = fileReader.Substring((Divider - 1) * myNewSize + 1, myNewLastSize)
And then you should save each split data to each it's table
I am reading a SSE by using this method
Public Shared Sub ReadStreamForever(ByVal stream As Stream)
Dim encoder = New UTF8Encoding()
Dim buffer = New Byte(2047) {}
Dim counter As Integer = 0
While True
If stream.CanRead Then
Dim len As Integer = stream.Read(buffer, 0, 2048)
counter = counter + 1
If len > 0 Then
Dim text = encoder.GetString(buffer, 0, len)
SSEApplication.Push(text) 'Here I collect the text slices to a List(of string) object
Else
Exit While
End If
Else
Exit While
End If
End While
SSEApplication.writer() 'Here I write the content to a .txt file
End Sub
With my example data it takes about 2 seconds. I would prefer not to read the stream into memory though and tried this method
Public Shared Sub ReadStreamForever1(ByVal stream As Stream)
Dim output As FileStream = File.OpenWrite("C:\Users\mini_dataset.txt")
While True
If stream.CanRead Then
stream.CopyTo(output)
Else
Exit While
End If
End While
End Sub
But the process ends up in an endless loop (I guess) at least to me it looks like the end of the stream can not be found. I can break the process after a few seconds and all the data are in the .txt file. Any idea what I can do to get the direct stream to file method working?
Stream.CanRead tells you whether a stream supports reading. Since it's apparently readable, While True will go on forever.
Let's verify whether the output Stream.CanWrite instead.
Public Shared Sub ReadStreamForever1(ByVal stream As Stream)
Using output As FileStream = File.OpenWrite("[Output file path]")
If output.CanWrite Then
stream.CopyTo(output)
End If
End Using
End Sub
If the process takes some time and you need to report its progress, you could read the stream using a buffer (I didn't add any error checking but of course a try/catch block should be used):
(Here, with 100 parts division commonly used by a ProgressBar)
Public Sub ReadStreamForever1(ByVal stream As Stream)
Dim BufferLength As Integer = 81920 'As the default stream buffer
Dim Buffer(BufferLength) As Byte
Dim BytesRead As Long = 0L
Using output As FileStream = File.OpenWrite("[Output file path]")
If output.CanWrite Then
Dim Part As Long = stream.Length \ 100
Dim PartCount As Integer = 0
Dim read As Integer = 0
Do
read = stream.Read(Buffer, 0, BufferLength)
If read = 0 Then Exit Do
If (BytesRead / Part > PartCount) Then
PartCount += 1
'ReportWriteProgress(PartCount)
End If
output.Write(Buffer, 0, read)
BytesRead += read
Loop
End If
End Using
End Sub
so I am creating a piece of software that in short, has a list of original byte sequences and new sequences that those bytes need to be changed into, kinda like this in text form "original location(currently irrelevant as sequence can be in different places) $ 56,69,71,73,75,77 : 56,69,71,80,50,54"
I already have code that works fine, however there can be up to 600+ of these sequences to find and change and in some cases it is taking a really really long time 15 mins +, i think it is down to how long it is taking to find the sequences to them change so i am trying to find a better way to do this as currently it is unusable due to how long it takes.
I have copied the whole code for this function below in hopes one of you kind souls can have a look and help =)
Dim originalbytes() As Byte
Dim fd As OpenFileDialog = New OpenFileDialog()
fd.Title = "Select the file"
fd.Filter = "All files (*.*)|*.*|All files (*.*)|*.*"
fd.FilterIndex = 2
If fd.ShowDialog() = DialogResult.OK Then
TextBox2.Text = fd.FileName
originalbytes = File.ReadAllBytes(fd.FileName)
End If
Dim x As Integer = 0
Dim y As Integer = 0
Dim textbox1array() = TextBox1.Lines
Dim changedbytes() = originalbytes
Dim startvalue As Integer = 0
Dim databoxarray() As String
Dim databoxarray2() As String
While x < textbox1array.Length - 1
'for each change to make
databoxarray = textbox1array(x).Replace(" $ ", vbCr).Replace(" : ", vbCr).Split
databoxarray2 = databoxarray(1).Replace(",", vbCr).Split
Dim databox2bytes() As String = databoxarray2
'copy original bytes line to databox2 lines
y = 0
While y < (originalbytes.Length - databox2bytes.Length)
'repeat for all bytes in ori file - size of data to find
If originalbytes(y) = databox2bytes(0) Then
startvalue = y
Dim z As String = 1
Dim samebytecounter As Integer = 1
While z < databox2bytes.Length
'repeat for all ori bytes
If originalbytes(y + z) = databox2bytes(z) Then
samebytecounter = samebytecounter + 1
End If
z = z + 1
End While
If samebytecounter = databox2bytes.Length Then
'same original data found, make changes
Dim bytestoinsert() As String = databoxarray(2).Replace(",", vbCr).Split
Dim t As Integer = 0
While t < bytestoinsert.Length
changedbytes(startvalue + t) = bytestoinsert(t)
t = t + 1
End While
End If
End If
y = y + 1
End While
x = x + 1
End While
File.WriteAllBytes(TextBox2.Text & " modified", changedbytes)
Let 's take a look at that inner while loop in your code, there are some things that can be optimized:
There is no need to check the total length all the time
Dim length as Integer = originalbytes.Length - databox2bytes.Length
While y < length
'repeat for all bytes in ori file - size of data to find
If originalbytes(y) = databox2bytes(0) Then
startvalue = y
z is not necessary, samebytecounter does exactly the same
Dim samebytecounter As Integer = 1
This while loop is a real bottleneck, since you always check the full length of your databox2bytes, you should rather quit the while loop when they don't match
While samebytecounter < databox2bytes.Length AndAlso originalbytes(y + samebytecounter ) = databox2bytes(samebytecounter )
samebytecounter = samebytecounter + 1
End While
This seems fine, but you already splitted the data at the top of your while loop, so, no need to create another array that does the same operation again
If samebytecounter = databox2bytes.Length Then
'same original data found, make changes
Dim t As Integer = 0
While t < databoxarray2.Length
changedbytes(startvalue + t) = databoxarray2(t)
t = t + 1
End While
End If
End If
y = y + 1
End While
For the rest I would agree that the algorithm you created is hugely inefficient, theoretically your code could have been rewritten like eg: (didn't really test this code)
Dim text = System.Text.Encoding.UTF8.GetString(originalbytes, 0, originalbytes.Length)
dim findText = System.Text.Encoding.UTF8.GetString(stringToFind, 0, stringToFind.Length)
dim replaceWith = System.Text.Encoding.UTF8.GetString(stringToSet, 0, stringToSet.Length)
text = text.Replace( findText, replaceWith )
dim outbytes = System.Text.Encoding.UTF8.GetBytes(text)
which would probably be a huge time saver.
For the rest your code seems to be created in such a way that nobody will really understand it if it's laying around for a month or so, I would say, including yourself
I'm trying to figure out how to read a section of bytes (Say 16) starting at a specific address, say 0x2050. I'd like to get the 16 bits output in hex values into a label.
I've been trying to figure out BinaryReader, and FileStreams but I'm not entirely sure what the difference is, or which one I should be using.
*I've seen a lot of threads mentioning file size could be an issue, and I'd like to point out that some files I'll be checking may be up to 4gb in size.
I've tried the following:
Dim bytes() As Byte = New Byte(OpenedFile.Length) {}
ListBox1.Items.Add(Conversion.Hex(OpenedFile.Read(bytes, &H2050, 6)))
But this simply writes 6 bytes to the file, and I'm not sure why. There is no output in the listbox.
How about something like the following?:
Sub Main()
Dim pos As Long = 8272
Dim requiredBytes As Integer = 2
Dim value(0 To requiredBytes - 1) As Byte
Using reader As New BinaryReader(File.Open("File.bin", FileMode.Open))
' Loop through length of file.
Dim fileLength As Long = reader.BaseStream.Length
Dim byteCount As Integer = 0
reader.BaseStream.Seek(pos, SeekOrigin.Begin)
While pos < fileLength And byteCount < requiredBytes
value(byteCount) = reader.ReadByte()
pos += 1
byteCount += 1
End While
End Using
Dim displayValue As String
displayValue = BitConverter.ToString(value)
End Sub
Here is the code I used to store an image in my database (SQL Server 2008 R2) using VB 2010.
The images get stored but the problem is the clarity of the image is lost when retrieved and seen in a picture box.
Public Function InsertUpdateImage(ByRef _SqlConnection As System.Data.SqlClient.SqlConnection, ByVal _Image As System.Drawing.Image, ByVal _ImageFormat As System.Drawing.Imaging.ImageFormat) As Integer
Dim _SqlRetVal As Integer = 0
'System.IO.Path.GetFullPath(files(ListView1.SelectedIndices(0))) Give the path for the 'image from listview
Dim str As String = System.IO.Path.GetFullPath(files(ListView1.SelectedIndices(0)))
Dim i As Integer = Len(str)
Dim j As Integer = 0
Dim locstr(i + 10) As Char
i = 0
While i < Len(str)
If str(i) = "\" Then
locstr(j) = "\"
j = j + 1
Else
locstr(j) = str(i)
j = j + 1
End If
i = i + 1
End While
Dim loc As New String(locstr)
MsgBox(loc)
' lets add this record to database
Dim _SqlCommand As New System.Data.SqlClient.SqlCommand("insert into maindb(photo,name,location) values(#image,'" + System.IO.Path.GetFileName(files(ListView1.SelectedIndices(0))) + "','" + loc + "')", _SqlConnection)
' Convert image to memory stream
Dim _MemoryStream As New System.IO.MemoryStream()
_Image.Save(_MemoryStream, _ImageFormat)
' Add image as SQL parameter
Dim _SqlParameter As New System.Data.SqlClient.SqlParameter("#image", SqlDbType.Image)
_SqlParameter.Value = _MemoryStream.ToArray()
_SqlCommand.Parameters.Add(_SqlParameter)
' Executes a Transact-SQL statement against the connection
' and returns the number of rows affected.
_SqlRetVal = _SqlCommand.ExecuteNonQuery()
Console.Write(_SqlRetVal)
' Dispose command
_SqlCommand.Dispose()
_SqlCommand = Nothing
' Error occurred while trying to execute reader
' send error message to console (change below line to customize error handling)
Return _SqlRetVal
End Function
Your image.save() reduces the quality of the image (if saved as Jpeg) to the default compression level of aabout 75%.
Please see this MSDN article on increasing this quality level, when you call Save by passing in myEncoderParameters, containing a quality level at a much higher level (say 90%)
http://msdn.microsoft.com/en-us/library/system.drawing.imaging.encoder.quality.aspx
Or see the (untested) code below, that should do the trick
' Create a a single encoder parameter envelope
Dim EncoderParameters As New EncoderParameters(1)
' Create and add a single quality parameter to this envelope, specifying 95%
Dim QualityParam As New EncoderParameter(Encoder.Quality, CType(95L, Int32))
EncoderParameters.Param(0) = QualityParam
' Save the image with the encoder param specifying 95% quality
_image.Save(_MemoryStream, _ImageFormat, EncoderParameters)