VB .NET 2010: Save a large bitmap as file [duplicate] - vb.net

This question already has answers here:
A Generic error occured in GDI+ in Image.Save() method
(2 answers)
Closed 6 years ago.
I have an application which creates a Bitmap object consisting of all the 16 million colors. The final bitmap would be measuring 4096 × 4096 pixels.
When I try to call the Bitmap's Save() method, it causes an error.
This is the error message that comes up:
A generic error occurred in GDI+.
Please try to help me on this thing.
Thanks in advance!
Note: Look at the last few commented lines of the source code below. I have explained the doubt there.
SOURCE CODE:
Public Sub CreateAllColorImage()
Dim BMP As New Bitmap(4096, 4096) 'This BMP variable is where the image will be created.'
Dim CurrX = 0
Dim CurrY = 0
Dim ExitFors As Boolean = False
For R = 0 To 255
For G = 0 To 255
For B = 0 To 255
BMP.SetPixel(CurrX, CurrY, Color.FromArgb(R, G, B))
CurrY += 1 'Increment the Y axis to move to next pixel.'
If CurrY > 4095 Then CurrX += 1 : CurrY = 0 'Move to next row, or increment X axis, if the last pixel on the Y axis is reached'
If CurrX > 4095 Then ExitFors = True 'Set the variable to exit the FOR loops if the very last pixel on the whole image is reached.'
If ExitFors Then Exit For 'Exit the FOR loop if the variable is true.'
Next
If ExitFors Then Exit For 'Exit the FOR loop if the variable is true.'
Next
If ExitFors Then Exit For 'Exit the FOR loop if the variable is true.'
Next
'So therefore, the final image is the BMP variable.'
'Here, I try to save the Bitmap as a file by calling this:'
BMP.Save("C:\TEST.BMP")
'This is when the error occurs. I think so because of the image is too large. If so, is there any way to do anything?'
'And by the way, I already have the rights to access the C:\ drive because I am working from an Administrator account...'
BMP.Dispose()
BMP = Nothing
End Sub

As a normal user one usually does not have access to the root C:\ drive. Even if you are an administrator your application will run under the normal privileges unless otherwise specified.
Unfortunately GDI+ hides most exceptions so it's hard to know the exact cause of what's happening, but my guess is that your problem is due to that your application isn't allowed to save directly in the C:\ drive.
Try saving the image in a path you are guaranteed to have access to, like C:\Users\YourUserName\Desktop.

It may likely be a permission problem. To test it, find your compiled .exe and run it as an administrator. If it works, you know it's a permission problem.
If you are trying to save it to your C: drive because you don't know the user name, you can instead use SpecialDirectory.
e.g.
My.Computer.FileSystem.SpecialDirectories.MyDocuments
will let you save it in the user's documents folder. Generally, saving to C: drive isn't the best thing to do.
I would also advise using the other constructor in which you can specify the image format.
e.g.
bitmapToSave.Save(saveLocation, Imaging.ImageFormat.Bmp)

Related

Read bytes from a file, but they might be anywhere in the file (VB.NET) [duplicate]

This question already exists:
Read specific bytes from a file, but they might be anywhere in the file (VB.NET) [closed]
Closed 2 years ago.
I have a file that changes size depending on the amount of data it contains. With that, the location of the bytes that I want to read moves back and forth every time the file is saved by its main application. I am using the string inside the file "This is the data" to get close to the bytes I want to read 31 38 33 34. They're always on the same position after the string, regardless of the size of the file. The only consistent thing is the string, the bytes will be different every time.
Try
TextBuffer = File.ReadAllText("C:\test.txt")
Catch ex As Exception
Exit Sub
End Try
Dim indexTar As Integer = TextBuffer.IndexOf("This is the data")
If indexTar >= 0 Then
ListView1.Items.Add("This is the data")
End If
I use the code above to read the whole file and end up near the location where the bytes I want to read are.
How do I read those bytes 31 38 33 34?
I'm not quite sure why you're talking about bytes, when this seems to be a text file - it would be easier to read and treat it as such, but simplistically you can read the whole thing into memory, find the index of what you know and then add some amount to get to the thing you don't:
Dim s = File.ReadAllText("C:\test.txt")
Dim indexTar = s.IndexOf("This is the data")
If indexTar >= 0 Then
Dim tIdx = indexTar + "This is the data".Length + 4 'seems to be 4 bytes after the end of the string
Dim iWantText = s.Substring(tIdx, 4)
End If
Now, iWantText contains 1834. If you want it as a byte array, Dim bytes = Encoding.ASCII.GetBytes(iWantText) will give it you.
It might be better, if the file is huge, to read it char by char (it will be buffered elsewhere, don't worry about inefficiency of reading one char at a time) looking for T and if you find it, see if his is my data follows...

inserting data from a text file to textboxes and setting combobox indecies

so I am writing a program and trying to setup the save/open features of the program. I have the Save feature working just fine, but can't get the open feature to work.
The issue I'm running into is pulling the data from the text file to the form to fill in the multiple fields and controls. my example code is below
Imports System.IO
Main 1
Sub openFile_Click(sender, e) handles openFile.Click
Dim lineIndex As Integer = 12 'this is my total lines in my file
ofdRead.ShowDialog()
If ofdRead.FileName <> "" then
sr = New StreamReader(ofdRead.FileName)
For i As Integer = 0 To lineIndex -1
sr.ReadLine()
Next
txtField1.Text = sr.ReadLine
cboBox1.SelectedIndex = sr.ReadLine
'this continues through all fields til complete
sr.Close()
End If
End Sub
End Class
I keep getting an error for anything that is being returned as not being a string, and it seems as though the data is being read in reverse as well according to my error output.
Any help would be much appreciated (been searching and pouring over forums for 2 days looking for help)
Thanks Tim Schmelter for your insight. I was calling the wrong data type for the cboBox1 variable. Here is my corrected code (without the For-Loop, turns out i didn't need it)
cboBox1.SelectedItem = sr.ReadLine
so everytime I run into something like that I just have to tell it to put it in as a string instead of an integer

Why do I get stack overflow BEFORE all the possible combinations have been reached?

So.
I am making a bruteforcer in visual basic.
It has a charset as shown below:
Dim charset as string
charset = "abcdefghijklmnopqrstuvwxyz1234567890."
There is 37 different chars right? The program is made to search for all the different combinations made with this charset, with a maximum of 3 different letters. For example
This is a combination that can be made: ac6
So since there is 37 letters and 3 slots the possible combinations are 37^3
But I wanted my program not to try the same combination twice.
So it saves every single combination tried in this location (Desktop)
Dim filex As System.IO.StreamWriter
filex = My.Computer.FileSystem.OpenTextFileWriter("c:\Users\" + Environment.UserName + "\desktop\alreadytested.txt", True)
filex.WriteLine(combination)
filex.Close()
And, at the start of the Sub that checks for new combinations, I have this
text = File.ReadAllText("c:\Users\" + Environment.UserName + "\desktop\alreadytested.txt")
index = text.IndexOf(combination) 'checks if it has been generated already
If index >= 0 Then
keyword() 'The sub
End If
But after some combinations (in this case the max 37^3 ~= 50.000 and I the program tried around 5200 times) I get this error
An unhandled exception of type 'System.StackOverflowException' occurred in mscorlib.dll
And this error points in this line of code
filex = My.Computer.FileSystem.OpenTextFileWriter("c:\Users\" + Environment.UserName + "\desktop\alreadytested.txt", True)
But why? at 5200 there is still 44800 possible random combinations, why do I get an overflow?
It would make sense if I got it when I had 50000 combinations out of 50000 possible tested, but now I have 10% only, so why do I get an overflow?
You keep on recursively calling the keyword() method. Every time you call a method its return address and possibly its arguments is/are added to the call stack. The callstack can only take a certain amount of calls before it overflows, and for your computer and that specific method of yours, that amount seems to be about 5200.
You should switch to using for example a While-loop instead, and whenever you want to block the rest of the execution and go back to the beginning of the loop you'd just call Continue While.
A little side note is also that you shouldn't open and close the file every time you read/write to it. Store the contents of the file in a long string (or even better, in a HashSet(Of T)) instead and check that every time you need to, then at the end of the loop you may write all the contents to a file.
If you still wish to write to the file during the process then do so. But instead open a stream before your loop which you keep writing to until the loop is finished, then close the stream.

VB.NET (2013) - Check string against huge file

I have a text file that is 125Mb in size, it contains 2.2 million records. I have another text file which doesn't match the original but I need to find out where it differs. Normally, with a smaller file I would read each line and process it in some way, or read the whole file into a string and do likewise, however the two files are too big for that and so I would like to create something to achieve my goal. Here's what I currently have.. excuse the mess of it.
Private Sub refUpdateBtn_Click(sender As Object, e As EventArgs) Handles refUpdateBtn.Click
Dim refOrig As String = refOriginalText.Text 'Original Reference File
Dim refLatest As String = refLatestText.Text 'Latest Reference
Dim srOriginal As StreamReader = New StreamReader(refOrig) 'start stream of original file
Dim srLatest As StreamReader = New StreamReader(refLatest) 'start stream of latest file
Dim recOrig, recLatest, baseDIR, parentDIR, recOutFile As String
baseDIR = vb.Left(refOrig, InStrRev(refOrig, ".ref") - 1) 'find parent folder
parentDIR = Path.GetDirectoryName(baseDIR) & "\"
recOutFile = parentDIR & "Updated.ref"
Me.Text = "Processing Reference File..." 'update the application
Update()
If Not File.Exists(recOutFile) Then
FileOpen(55, recOutFile, OpenMode.Append)
FileClose(55)
End If
Dim x As Integer = 0
Do While srLatest.Peek() > -1
Application.DoEvents()
recLatest = srLatest.ReadLine
recOrig = srOriginal.ReadLine ' check the original reference file
Do
If Not recLatest.Equals(recOrig) Then
recOrig = srOriginal.ReadLine
Else
FileOpen(55, recOutFile, OpenMode.Append)
Print(55, recLatest & Environment.NewLine)
FileClose(55)
x += 1
count.Text = "Record No: " & x
count.Refresh()
srOriginal.BaseStream.Seek(0, SeekOrigin.Begin)
GoTo 1
End If
Loop
1:
Loop
srLatest.Close()
srOriginal.Close()
FileClose(55)
End Sub
It's got poor programming and scary loops, but that's because I'm not a professional coder, just a guy trying to make his life easier.
Currently, this uses a form to insert the original file and the latest file and outputs each line that matches into a new file. This is less than perfect, but I don't know how to cope with the large file sizes as streamreader.readtoend crashes the program. I also don't need the output to be a copy of the latest input, but I don't know how to only output the records it doesn't find. Here's a sample of the records each file has:
doc:ARCHIVE.346CCBD3B06711E0B40E00163505A2EF
doc:ARCHIVE.346CE683B29811E0A06200163505A2EF
doc:ARCHIVE.346CEB15A91711E09E8900163505A2EF
doc:ARCHIVE.346CEC6AAA6411E0BEBB00163505A2EF
The program I have currently works... to a fashion, however I know there are better ways of doing it and I'm sure much better ways of using the CPU and memory, but I don't know this level of programming. All I would like is for you to take a look and offer your best answers to all or some of the code. Tell me what you think will make it better, what will help with one line, or all of it. I have no time limit on this because the code works, albeit slowly, I would just like someone to tell me where my code could be better and what I could do to get round the huge file sizes.
Your code is slow because it is doing a lot of file IO. You're on the right track by reading one line at a time, but this can be improved.
Firstly, I've created some test files based off the data that you provided. Those files contain three million lines and are about 130 MB in size (2.2 million records was less than 100 MB so I've increased the number of lines to get to the file size that you state).
Reading the entire file into a single string uses up about 600 MB of memory. Do this with two files (which I assume you were doing) and you have over 1GB of memory used, which may have been causing the crash (you don't say what error was shown, if any, when the crash occurred, so I can only assume that it was an OutOfMemoryException).
Here's a few tips before I go through your code:
Use Using Blocks
This won't help with performance, but it does make your code cleaner and easier to read.
Whenever you're dealing with a file (or anything that implements the IDisposable interface), it's always a good idea to use a Using statement. This will automatically dispose of the file (which closes the file), even if an error happens.
Don't use FileOpen
The FileOpen method is outdated (and even stated as being slow in its documentation). There are better alternatives that you are already (almost) using: StreamWriter (the cousin of StreamReader).
Opening and closing a file two million times (like you are doing inside your loop) won't be fast. This can be improved by opening the file once outside the loop.
DoEvents() is evil!
DoEvents is a legacy method from back in the VB6 days, and it's something that you really want to avoid, especially when you're calling it two million times in a loop!
The alternative is to perform all of your file processing on a separate thread so that your UI is still responsive.
Using a separate thread here is probably overkill, and there are a number of intricacies that you need to be aware of, so I have not used a separate thread in the code below.
So let's look at each part of your code and see what we can improve.
Creating the output file
You're almost right here, but you're doing some things that you don't need to do. GetDirectoryName works with file names, so there's no need to remove the extension from the original file name first. You can also use the Path.Combine method to combine a directory and file name.
recOutFile = Path.Combine(Path.GetDirectoryName(refOrig), "Updated.ref")
Reading the files
Since you're looping through each line in the "latest" file and finding a match in the "original" file, you can continue to read one line at a time from the "latest" file.
But instead of reading a line at a time from the "original" file, then seeking back to the start when you find a match, you will be better off reading all of those lines into memory.
Now, instead of reading the entire file into memory (which took up 600 MB as I mentioned earlier), you can read each line of the file into an array. This will use up less memory, and is quite easy to do thanks to the File class.
originalLines = File.ReadAllLines(refOrig)
This reads all of the lines from the file and returns a String array. Searching through this array for matches will be slow, so instead of reading into an array, we can read into a HashSet(Of String). This will use up a bit more memory, but it will be much faster to seach through.
originalLines = New HashSet(Of String)(File.ReadAllLines(refOrig))
Searching for matches
Since we now have all of the lines from the "original" line in an array or HashSet, searching for a line is very easy.
originalLines.Contains(recLatest)
Putting it all together
So let's put all of this together:
Private Sub refUpdateBtn_Click(sender As Object, e As EventArgs)
Dim refOrig As String
Dim refLatest As String
Dim recOutFile As String
Dim originalLines As HashSet(Of String)
refOrig = refOriginalText.Text 'Original Reference File
refLatest = refLatestText.Text 'Latest Reference
recOutFile = Path.Combine(Path.GetDirectoryName(refOrig), "Updated.ref")
Me.Text = "Processing Reference File..." 'update the application
Update()
originalLines = New HashSet(Of String)(File.ReadAllLines(refOrig))
Using latest As New StreamReader(refLatest),
updated As New StreamWriter(recOutFile, True)
Do
Dim line As String
line = latest.ReadLine()
' ReadLine returns Nothing when it reaches the end of the file.
If line Is Nothing Then
Exit Do
End If
If originalLines.Contains(line) Then
updated.WriteLine(line)
End If
Loop
End Using
End Sub
This uses around 400 MB of memory and takes about 4 seconds to run.

Open and save problems

I have a function in my VB project where i scan an image and then I can change the contrast.
I scan it and saves it C:\temp\my_img.tif.
In the winform the image is displayed in a PictureBox.
If I in the contrast function set like img.Save("C:\temp\my_img.tif", ImageFormat.Tiff) I get "A generic error occurred in GDI+.". If I however set the filename to something else, it works just fine.
So, how do I release the used image before saving it?
The whole function, in short:
Sub setContrast(ByVal C As Single)
'filename(1) ia a "global" variable that stores the used file path, in this case "C:\temp\my_img.tif"
Dim img As Image = Image.FromFile(filename(1)) '<--- I get the image
'A bunch of contrast stuff in some rows.....
'Here, i should release the image...
img.Save(filename(1), ImageFormat.Tiff) '<---Tries to save
PictureBox1.Refresh()
End Sub
Save it using a different file name, and then, if necessary, delete the old file and rename the new file to match the old, having Disposed of the Image beforehand.
From Image.FromFile:
The file remains locked until the Image is disposed.
There's no wording anywhere else that says that this is somehow worked around if the same Image instance is trying to Save back to the file.