I've been trying for 2 weeks to uncompress this user-defined TXXX string from an MP3 ID2,3 file.
000000B0789C6330377433D63534D575F3F737B570343767B02929CA2C4B2D4BCD2B29B6B301D376367989B9A976C519F9E50ACE1989452536FA60019B924C20696800017A10CA461F2C6AA30FD58A61427E5E72AA42228A114666E6F88CD047721100D5923799
Thanks to Dr. Adler for the correct answer when I converted the values to a string.
I have tried both MS DeflateStream and GZipstream with no success.
Every example I see uses a stream file. I am not using a file, I have the above zLib code in both an array or string variable.
GZipstream gives me 'no magic number' and Deflatestream gives me 'Block length does not match with its complement'.
I read this post:
http://george.chiramattel.com/blog/2007/09/deflatestream-block-length-does-not-match.html
tried removing bytes from the head, no luck. (I read trazillions of articles for sending a string to Deflatestream but again 'no luck'!
I have the above string, so how do I send it to Deflatestream? I'd post the two hundred different code examples I tried but that would be silly.
The funny thing is, I built my webAudio cue marker editor in less than two weeks and this is the last thing I have it do (my program must get the marker positions from a program that has worst audio editor known to man (they embedded them in the MP3 for some (bad) reason). Hence, I wrote my own to change audio cue marker so I could save hours of frustration at work. However, I'm not getting much sleep lately.
Help me get some sleep, please.
You can use a MemoryStream instead of a FileStream as they are both Streams:
Imports System.IO
Imports System.IO.Compression
Imports System.Text
Module Module1
Function HexStringToBytes(s As String) As Byte()
If (s.Length And 1) = 1 Then
Throw New ArgumentException("String is an odd number of characters in length - it must be even.")
End If
Dim bb As New List(Of Byte)
For i = 0 To s.Length - 1 Step 2
bb.Add(Convert.ToByte(s.Substring(i, 2), 16))
Next
Return bb.ToArray()
End Function
Sub Main()
Dim s = "000000B0789C6330377433D63534D575F3F737B570343767B02929CA2C4B2D4BCD2B29B6B301D376367989B9A976C519F9E50ACE1989452536FA60019B924C20696800017A10CA461F2C6AA30FD58A61427E5E72AA42228A114666E6F88CD047721100D5923799"
Dim result As String = ""
' trim off the leading zero bytes and skip the three bytes 0xB0 0x78 0x9C
Dim buffer = HexStringToBytes(s).SkipWhile(Function(b) b = 0).Skip(3).ToArray()
Using ms As New MemoryStream(buffer)
Using decompressedMemoryStream As New MemoryStream
Using decompressionStream As New DeflateStream(ms, CompressionMode.Decompress)
decompressionStream.CopyTo(decompressedMemoryStream)
result = Encoding.Default.GetString((decompressedMemoryStream.ToArray()))
End Using
End Using
End Using
Console.WriteLine(result)
Console.ReadLine()
End Sub
End Module
Outputs:
71F3-15-FOO58A77 <trivevents><event><name>show Chart</name><time>10000000.000000</time></event><event><name>show once a</name><time>26700000.000000</time></event></trivevents>
(There is a leading zero byte.)
P.S. It looks a bit strange that there is 71F3-15-FOO58A77 with letter Os instead of zeros.
P.P.S. If you could get the compressed data into a Base64 string instead of a hex string, you could pack more data into the same space.
Related
I have been trying to fix this for a number of days now without any success. I know I have created another post related to this issue but not sure if I should have continued with the other post rather than creating a new one as I am still quite new to how SO works so apologies if I have gone about this the wrong way.
Objective
Read a text file from disk, output the contents to a Textbox so I can then extract the last 3 lines from it. This is the only way I can think of doing this.
The text file is continuously been updated by another running program but I can still read it even though it is in use but cannot write to it.
I am probing this file through a Timer which ticks every 1 second in order to get the latest information.
Now to the issue...
I have noticed that after some time my app becomes sluggish which is noticeable when I try to move it across the screen or resize it and the CPU usage starts to creep up to over 33%
My Thought Process
As this reading the file is a continuous one, I was thinking that I could move it onto a BackgroundWorker which from my understanding would put it on a different thread and take some load off the main GUI.
Am I barking up the wrong tree on this one?
I am reaching out to more advanced users before I start to get all the text books out on learning how to use the BackgroundWorker.
Here is the code I am using to Read the txt file and output it to a text Box. I have not included the code for extracting the last 3 lines because I don't think that part is causing the issue.
I think the issue is because I am constantly probing the source files every second with a timer but not 100% sure to be honest.
Dim strLogFilePath As String
strLogFilePath = "C:\DSD\data.txt"
Dim LogFileStream As FileStream
Dim LogFileReader As StreamReader
'Open file for reading
LogFileStream = New FileStream(strLogFilePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)
LogFileReader = New StreamReader(LogFileStream)
'populate text box with the contents of the txt file
Dim strRowText As String
strRowText = LogFileReader.ReadToEnd()
TextBox1.text = strRowText
'Clean Up
LogFileReader.Close()
LogFileStream.Close()
LogFileReader.Dispose()
LogFileStream.Dispose()
Firstly, you should use the Using keyword instead of manually disposing objects, because that way you are guaranteed that the object will get disposed, even if an unexpected exception occurs, for example:
' You can initialize variables in one line
Dim strLogFilePath As String = "C:\DSD\data.txt"
Using LogFileStream As New FileStream(strLogFilePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)
' Everything goes in here
End Using
You don't need the reader for my solution. The reading will be done manually.
Next, you need to read the last n lines (in your case, 3) of the stream. Reading the entire file when you're only interested in a few lines at the end is inefficient. Instead, you can start reading from the end until you've reached three (or any number of) line seprators (based on this answer):
Function ReadLastLines(
NumberOfLines As Integer, Encoding As System.Text.Encoding, FS As FileStream,
Optional LineSeparator As String = vbCrLf
) As String
Dim NewLineSize As Integer = Encoding.GetByteCount(LineSeparator)
Dim NewLineCount As Integer = 0
Dim EndPosition As Long = Convert.ToInt64(FS.Length / NewLineSize)
Dim NewLineBytes As Byte() = Encoding.GetBytes(LineSeparator)
Dim Buffer As Byte() = Encoding.GetBytes(LineSeparator)
For Position As Long = NewLineSize To EndPosition Step NewLineSize
FS.Seek(-Position, SeekOrigin.End)
FS.Read(Buffer, 0, Buffer.Length)
If Encoding.GetString(Buffer) = LineSeparator Then
NewLineCount += 1
If NewLineCount = NumberOfLines Then
Dim ReturnBuffer(CInt(FS.Length - FS.Position)) As Byte
FS.Read(ReturnBuffer, 0, ReturnBuffer.Length)
Return Encoding.GetString(ReturnBuffer)
End If
End If
Next
' Handle case where number of lines in file is less than NumberOfLines
FS.Seek(0, SeekOrigin.Begin)
Buffer = New Byte(CInt(FS.Length)) {}
FS.Read(Buffer, 0, Buffer.Length)
Return Encoding.GetString(Buffer)
End Function
Usage:
Using LogFileStream As New FileStream(strLogFilePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)
' Depending on system, you may need to supply an argument for the LineSeparator param
Dim LastThreeLines As String = ReadLastLines(3, System.Text.Encoding.UTF8, LogFileStream)
' Do something with the last three lines
MsgBox(LastThreeLines)
End Using
Note that I haven't tested this code, and I'm sure it can be improved. It may also not work for all encodings, but it sounds like it should be better than your current solution, and that it will work in your situation.
Edit: Also, to answer your question, IO operations should usually be performed asynchronously to avoid blocking the UI. You can do this using tasks or a BackgroundWorker. It probably won't make it faster, but it will make your application more responsive. It's best to indicate that something is loading before the task begins.
If you know when your file is being written to, you can set a flag to start reading, and then unset it when the last lines have been read. If it hasn't changed, there's no reason to keep reading it over and over.
For my current project in need a way to use ä,ö etc. in a datatable that is written to a .csv
It is the same project as in: VB Reading data from SQL Server to Array, writing into .CSV
I know that I need UTF-8 but how do I use it ?
Unlike VB6/VBScript/VBA, VB.Net strings already use full Unicode internally. You can already put accented characters in your string variables (and string members for other objects), and you don't need to do anything special.
There are three things you do need to watch for, though.
First, you must be sure to use NVARCHAR rather than VARCHAR for your Sql Server columns, as well as your ADO.Net parameters. You may also need to be careful about what collation you have (but the default is almost certainly fine here).
Second, when you open your StreamWriter, you need to use unicode-capable correct Encoding. System.Text.UTF8Encoding is one option. You could also do System.Text.UnicodeEncoding (which is UTF16) or System.Text.UTF32Encoding and get accurate output.
Finally, just because you successfully create a unicode CSV file, this does not mean your downstream consumers will handle the file correctly. A lot of text editors and other tools like to assume csv data is ASCII. But that's really outside of your scope. All you can is give them valid data. If they don't process it, that's on them :)
So assuming the database is correct, and based on the other question, you have this code:
Sub WriteCsvFiles(destPath As String, headings As String(), dt As DataTable)
Dim separator As Char = ";"c
Dim header = String.Join(separator, headings)
For Each r As DataRow In dt.Rows
Dim destFile = Path.Combine(destPath, r(0).ToString().Trim() & ".csv")
Using sw As New StreamWriter(destFile)
sw.WriteLine(header)
sw.WriteLine(CsvLine(r.ItemArray, separator))
End Using
Next
End Sub
This is close. However, take a look at the remarks in the documentation for the StreamWriter constructor:
This constructor creates a StreamWriter with UTF-8 encoding without a Byte-Order Mark (BOM), so its GetPreamble method returns an empty byte array. The default UTF-8 encoding for this constructor throws an exception on invalid bytes. This behavior is different from the behavior provided by the encoding object in the Encoding.UTF8 property.
So we kind of already have UTF-8 data, but to really have a correct UTF-8 file, including correct byte-order handling for certain wide characters, we need to change things just a little bit. Where you have this right now:
Using sw As New StreamWriter(destFile)
should become:
Using sw As New StreamWriter(destFile, False, Encoding.UTF8)
It also seems very odd to create a separate file for every row that will all have the same structure. I know it's in your original question, but I'd really push back on that, or find out why, and the maybe re-write the method as so:
Sub WriteCsvFile(destFile As String, headings As IEnumerable(Of String), dt As DataTable)
Dim separator As Char = ";"c
Dim header As String = String.Join(separator, headings)
Using sw As New StreamWriter(destFile, False, Encoding.UTF8)
sw.WriteLine(header)
For Each r As DataRow In dt.Rows
sw.WriteLine(CsvLine(r.ItemArray, separator))
Next
End Using
End Sub
Following code should sum two numbers from file "input.txt" and write the sum to "output.txt". Compilation is succesfull, but "output.txt" is still empty after running program. What am I doing wrong?
Imports System.IO
Public Class test
Public Shared Sub Main()
Dim scan as StreamReader = new StreamReader("input.txt")
Dim writer as StreamWriter = new StreamWriter("output.txt", True)
Dim input as String
input = scan.ReadLine()
Dim ab() as String = Split(input)
Dim res as Integer = Val(ab(0))+Val(ab(1))
writer.writeLine(res)
writer.close()
End sub
End class
Your code works properly for me, so as long as your input file is formatted properly (i.e. a single line with two numbers separated by spaces, like "1 2") and you have the necessary OS permissions to read and write to those files, then it should work for you too. However, it's worth mentioning that there are several issues with your code that would be good to correct, since the fly in the face of typical best-practices.
First, you should, as much as possible, turn Option Strict On. I know that you have it Off because your code won't compile with it On. The following line is technically misleading, and therefore fails with Option Strict On:
Dim res As Integer = Val(ab(0)) + Val(ab(1))
The reason if fails is because the Val function returns a Double, not an integer, so, technically, depending on the contents of the file, the result could be fractional or could be too large to fit in an Integer. With Option Strict Off, the compiler is essentially automatically fixing your code for you, like this:
Dim res As Integer = CInt(Val(ab(0)) + Val(ab(1)))
In order to set the res variable equal to the result of the calculation, the more capable Double value must be converted down to an Integer. When you are forced to put the CInt in the code yourself, you are fully aware that the conversion is taking place and what the consequences of it might be. When you have Option Strict Off and it inserts the conversion behind-the-scenes, then you may very well miss a potential bug.
Secondly, the Val function is old-school VB6 syntax. While it technically works fine, it's provided mainly for backwards compatibility. The new .NET equivalent would be to use Integer.Parse, Integer.TryParse or Convert.ToInt32.
Thirdly, you never close the scan stream reader. You could just add scan.Close() to the end of your method, but is better, when possible, to create Using blocks for any disposable object, like this:
Using scan As StreamReader = New StreamReader("test.txt")
Using writer As StreamWriter = New StreamWriter("output.txt", True)
Dim input As String
input = scan.ReadLine()
Dim ab() As String = Split(input)
Dim res As Integer = Integer.Parse(ab(0)) + Integer.Parse(ab(1))
writer.WriteLine(res)
End Using
End Using
Lastly, as Hans pointed out, it's not good to rely on the current directory. It's always best to specify full paths for your files. There are different methods in the framework for getting various folder paths, such as the user's desktop folder, or the download folder, or the temp folder, or the application folder, or the current application's folder, or the folder of the current running assembly. You can use any such method to get your desired folder path, and then use Path.Combine to add the file name to get the full file path. For instance:
Dim desktopFolderPath As String = Environment.GetFolderPath(Environment.SpecialFolder.DesktopDirectory)
Dim inputFilePath As String = Path.Combine(desktopFolderPath, "input.txt")
Dim outputFilePath As String = Path.Combine(desktopFolderPath, "output.txt")
Hey guys I'm trying to make a program that helps people encrypt messages and decrypt messages using the Caesar shift cipher, I know it's probably already been done, I want to have a go myself though.
The problem I've been having is when it comes to encrypting the text. The user selects a number (between 1-25) and then the application will change the letters corresponding to the number chosen, e.g. if the user inputs "HI" and selects 2, both characters are moved two places down the alphabet outputting "JK". My main problem is the replacing characters though, mostly because I've set up the program to be able to encrypt large blocks of text, because my code is:
If cmbxKey.Text = "1" Then
If txtOutput.Text.Contains("a") Then
sOutput = txtOutput.Text.Replace("a", "b")
txtOutput.Text = sOutput
End If
If txtOutput.Text.Contains("b") Then
sOutput = txtOutput.Text.Replace("b", "c")
txtOutput.Text = sOutput
End If
End If
This means if the user inputs "HAY" it will change it to "HBY" and then because of the second if statement it will change it to "HCY" but I only want it to be changed once. Any suggestions to avoid this???? Thanks guys
Since you want to shift all characters, start out by looping though the characters using something like ToArray:
For each s as string in txtOutput.Text.ToArray
'This will be here for each character in the string, even spaces
Next
Then, rather than having cases for every letter, look at it's ascii number:
ACS(s)
...and shift it by the number you want to. Keep in mind that if the number is greater than (I don't know if you want upper/lower case) 122, you want to subtract 65 to get you back to "A".
Then you can convert it back into a character using:
CHR(?)
So this might look something like this:
Dim sb as new text.StringBuilder()
For each s as string in txtOutput.Text.ToArray
If asc(s) > 122 Then
sb.append(CHR(ASC(s) + ?YourShift? - 65)
Else
sb.append(CHR(ASC(s) + ?YourShift?)
END IF
Next
txtOutput.Text = sb.ToString
A very simple method of changing your application while keeping your strategy is to replace the lower case characters with upper case characters. Then they won't be recognized by the Replace method anymore.
Obviously, the problem is that you want to implement an algorithm. In general, an algorithm should be smart in the sense that you don't have to do the grunt work. That's why a method such as the one presented by Steve is smarter; it doesn't require you to map each character separately, which is tedious, and - as most tedious tasks - error prone.
One big issue arise when you're facing a String that the basic Alphanumeric table can't handle. A String that contains words like :
"Déja vu" -> The "é" is going to be what ?
And also, how about encoding the string "I'm Aaron Mbilébé" if you use .ToUpper().
.ToUpper returns "I'M AARON MBILÉBÉ".
You've lost the casing, and how do you handle the shifting of "É" ?
Of course, a code should be smart as pointed above, and I was used to deal with strings just by using the System.Text.ASCIIEncoding to make things easier. But from the moment I started to use large amount of textual datas, sources from the web, files (...) I was forced to dig deeper, and seriously consider string encoding (and System Endianness by the way, when coding and decoding string to/from array of bytes)
Re-think of what do you really want in the end. If you're the only one to use your code, and you're certain that you'll only use A..Z, 0..9, a..z, space and a fixed amount of allowed characters (like puntuation) then, just build a Table containing each of those chars.
Private _AllowedChars As Char() = { "A"c, "B"c, ... "0"c, "1"c, .. "."c, ","c ... }
or
Private _AllowedChars As Char() = "ABCDEF....012...abcd..xyz.;,?:/".ToCharArray()
Then use
Private Function ShiftChars(ByVal CurrentString As String, ByVal ShiftValue As Integer) As String
Dim AllChars As Char() = CurrentString.ToCharArray()
Dim FinalChars As Char()
Dim i As Integer
FinalChars = New Char(AllChars.Length - 1) {} ' It's VB : UpperBound is n+1 item.
' so n items is UpperBound - 1
For i = 0 To AllChars.Length - 1
FinalChars(i) = _AllowedChars((Array.IndexOf(_AllowedChars, AllChars(i)) + ShiftValue) Mod _AllowedChars.Length)
Next
Return New String(FinalChars)
End Function
And
Private Function UnShiftChars(ByVal CurrentString As String, ByVal ShiftValue As Integer) As String
' ... the same code until :
FinalChars(i) = _AllowedChars((Array.IndexOf(_AllowedChars, AllChars(i)) - ShiftValue + _AllowedChars.Length) Mod _AllowedChars.Length)
' ...
End Function
^^ Assuming ShiftValue is always positive (defined once)
But again, this only works when you have a predefined set of allowed characters. If you want a more flexible tool, you ought to start dealing with encodings, array of byte, BitConverter and have a look at system endianness. That's why I asked if someone else is goind to use your application : let's try this string :
"Xin chào thế giới" ' which is Hello World in vietnamese (Google Trad)
In that case, you may give up..? No ! You ALWAYS have a trick in your cards !
Just create your allowed chars on the fly
Private _AllowedChars As New SortedList(Of Char, Char)
-> get the string to encode (shift)
Private Function ShiftChars(ByVal CurrentString As String, ByVal ShiftValue As Integer) As String
Dim AllChars As Char() = CurrentString.ToCharArray()
Dim FinalChars As Char()
Dim i As Integer
' Build your list of allowed chars...
_AllowedChars.Clear()
For i = 0 To AllChars.Length - 1
If Not _AllowedChars.ContainsKey(AllChars(i)) Then
_AllowedChars.Add(AllChars(i), AllChars(i))
End If
Next
' Then, encode...
FinalChars = New Char(AllChars.Length - 1) {}
For i = 0 To AllChars.Length - 1
FinalChars(i) = _AllowedChars.Keys.Item((_AllowedChars.IndexOfKey(AllChars(i)) + ShiftValue) Mod _AllowedChars.Count)
Next
Return New String(FinalChars)
End Function
The same for Unshift/decode.
Note : in foreing languages, the resulting string is pure garbage and totally unreadable, unless you (un)shift the chars again.
However, the main limitation of this workaround is the same as the fixed chars array above : Once you encode your string, and add a char in your encoded string that doesn't exists in the initial generated allowed chars, then you've nuked your data and you won't be able to decode your string. All you'll have is pure garbage.
So one day... one day maybe, you'll have to dig deeper at the byte level of the thing, in a defined extended encoding (Unicode/UTF8/16) to secure the integrity of your data.
this is my code to convert string to hex
Function StringToHex(ByVal text As String) As String
Dim xhex As String
For i As Integer = 0 To text.Length - 1
xhex &= Asc(text.Substring(i, 1)).ToString("x").ToUpper
Next
Return xhex
End Function
I convert string file to hex with this code, but if size file more than 1MB my program is not responding
how to make this code more efficient for size file more than 1MB sorry my english is bad
As I said in my initial comment, your current approach is creating a new string each time you go through the For loop. Strings are immutable (can't be changed) in .NET - so for example if you have 3000 characters in the string, xHex = &a is going to create 3,000 strings, and that's just for the first part. Then you have a Substring, then a ToString and finally a ToUpper - so if my math is right, you're creating 4 strings for every character in the input string (so if you have 3,000 characters that 12,000 additional strings).
The call to Substring is unnecessary - you can treat the string as an array and access each character in the string as an array index, so now you would have:
xhex &= Asc(text(i)).ToString("x").ToUpper
You can also get rid of the call .ToUpper() by using an uppercase "X" in the call to .ToString() - so now you have:
xhex &= Asc(text(i)).ToString("X")
You could also make xhex a StringBuilder, and then you'd only be creating one additional string each time through the loop (the call to .ToString()). Putting that all together gives you this:
Dim xhex As StringBuilder = New StringBuilder()
For i As Integer = 0 To text.Length - 1
xhex.Append(Asc(text(i).ToString("X"))
Next
Return xhex.ToString()
That may help with the process, but if the string is really large you may still run into memory issues. IF the file is really large I'd recommend reading it using a Stream and processing the Stream one byte at a time (or several bytes at time, your choice).
I would also suggest Googling for VB.NET convert string to hex, as there are many examples of other ways to do this.