The following VB .net code gives me an out of memory exception. Does anybody knows why?
Dim vArray As ILArray(Of Double) = ILMath.rand(10000000)
Using ILScope.Enter(vArray)
For i As Integer = 1 To 100
vArray = ILMath.add(vArray, vArray)
Next
End Using
Thank you very much.
In this toy example you can simply remove the artificial scope and it will run fine:
Dim vArray As ILArray(Of Double) = ILMath.rand(10000000)
For i As Integer = 1 To 100
vArray = ILMath.add(vArray, vArray)
Next
Console.WriteLine("OK: " + vArray(0).ToString())
Console.ReadKey()
However, in a more serious situation, ILScope will be your friend. As stated on the ILNumerics page an artificial scope ensures a deterministic memory management:
All arrays created inside the scope are disposed once the block was
left.
Otherwise one had to rely on the GC for cleanup. And, as you know, this involves a gen 2 collection for large objects – with all disadvantages in terms of performance.
In order to be able to dispose the arrays they need to be collected and tracked somehow. Whether or not this qualifies for the term 'memory leak' is rather a philosophical question. I will not go into it here. The deal is: after the instruction pointer runs out of the scope these arrays are taken care of: their memory is put into the memory pool and will be reused. As a consequence, no GC will be triggered.
The scheme is especially useful for long running operations and for large data. Currently, the arrays are released only AFTER the scope block was left. So if you create an algorithm/ loop which requires more memory than available on your machine you need to clean up DURING the loop already:
Dim vArray As ILArray(Of Double) = ILMath.rand(10000000)
For i As Integer = 1 To 100
Using ILScope.Enter
vArray.a = ILMath.add(vArray, vArray)
' ...
End Using
Next
Here, the scope cleans up the memory after each iteration of the loop. This affects all local arrays assigned within the loop body. If we want an array value to survive the loop iteration we can assign to its .a property as shown with vArray.a.
Related
I'm in the process of reviewing many of our applications. One application came back with quite a few null deference issues. It is worth mentioning is that I am quite unfamiliar with VB.NET so bear with any issues I may relay with my code.
Public Sub SomeMethod()
Try
PerformLog("Entering SomeMethod")
Dim intTemp As Short
Erase gstrFieldWhatever //Defined earlier: Public gstrFieldWhatever() As String
...
intTemp = intTemp + 1
ReDim Preserve gstrFieldWhatever(intTemp)
gstrFieldWhatever(intTemp) = "Z*L-"
The analyzer, Fortify in this case, is reporting that Erase gstrFieldWhatever will cause a potential null exception when we try to call it later at gstrFieldWhatever(intTemp) = "Z*L-". However, we using ReDim Preserve gstrFieldWhatever(intTemp) should create a new array. If this is the case, why would the analyzer indicate a potential null dereference issue?
The gstrFieldWhatever is a public field. There's a risk if this object is used in multi-threaded code, because the field could be accessed between when it is freed and when it is ReDimmed.
In most cases, the use of ReDim Preserve indicates a place where you really want a collection like List(Of String), however, I understand if that change might cascade into other areas of the code.
Ok so I was in college and I was talking to my teacher and he said my code isn't good practice. I'm a bit confused as to why so here's the situation. We basically created a for loop however he declared his for loop counter outside of the loop because it's considered good practice (to him) even though we never used the variable later on in the code so to me it looks like a waste of memory. We did more to the code then just use a message box but the idea was to get each character from a string and do something with it. He also used the Mid() function to retrieve the character in the string while I called the variable with the index. Here's an example of how he would write his code:
Dim i As Integer = 0
Dim justastring As String = "test"
For i = 1 To justastring.Length Then
MsgBox( Mid( justastring, i, 1 ) )
End For
And here's an example of how I would write my code:
Dim justastring As String = "test"
For i = 0 To justastring.Length - 1 Then
MsgBox( justastring(i) )
End For
Would anyone be able to provide the advantages and disadvantages of each method and why and whether or not I should continue how I am?
Another approach would be, to just use a For Each on the string.
Like this no index variable is needed.
Dim justastring As String = "test"
For Each c As Char In justastring
MsgBox(c)
Next
I would suggest doing it your way, because you could have variables hanging around consuming(albeit a small amount) of memory, but more importantly, It is better practice to define objects with as little scope as possible. In your teacher's code, the variable i is still accessible when the loop is finished. There are occasions when this is desirable, but normally, if you're only using a variable in a limited amount of code, then you should only declare it within the smallest block that it is needed.
As for your question about the Mid function, individual characters as you know can be access simply by treating the string as an array of characters. After some basic benchmarking, using the Mid function takes a lot longer to process than just accessing the character by the index value. In relatively simple bits of code, this doesn't make much difference, but if you're doing it millions of times in a loop, it makes a huge difference.
There are other factors to consider. Such as code readability and modification of the code, but there are plenty of websites dealing with that sort of thing.
Finally I would suggest changing some compiler options in your visual studio
Option Strict to On
Option Infer to Off
Option Explicit to On
It means writing more code, but the code is safer and you'll make less mistakes. Have a look here for an explanation
In your code, it would mean that you have to write
Dim justastring As String = "test"
For i As Integer = 0 To justastring.Length - 1 Then
MsgBox( justastring(i) )
End For
This way, you know that i is definitely an integer. Consider the following ..
Dim i
Have you any idea what type it is? Me neither.
The compiler doesn't know what so it defines it as an object type which could hold anything. a string, an integer, a list..
Consider this code.
Dim i
Dim x
x = "ab"
For i = x To endcount - 1
t = Mid(s, 999)
Next
The compiler will compile it, but when it is executed you'll get an SystemArgumenException. In this case it's easy to see what is wrong, but often it isn't. And numbers in strings can be a whole new can of worms.
Hope this helps.
I am wondering what would be the most correct way to deal with memory when using VBScript. Should declare all variables right before I use them? The beginning of the program? I understand global vs local, however in my script all variables are local. I know that memory leaks will never be a problem when writing in VBScript 99.9% of the time, but I am also curious as to the 'best' way to clear and release memory within a script. By 'best' I mean, the timing of clearing variables/objects (right after you are done using them vs the end of the script), etc.
An example:
Dim fso: Set fso = CreateObject("Scripting.FileSystemObject")
Dim arrList : Set arrList = CreateObject("System.Collections.ArrayList")
Dim objDict: Set objDic = CreateObject( "Scripting.Dictionary")
Dim objEmail : Set objEmail = CreateObject("CDO.Message")
Dim someArray(), x, y, z, item
It's best practice to declare all variables, but not for the reason you assume. VBScript is sufficiently good at cleaning up after itself, so memory leaks usually aren't an issue. Most of the time you don't even need to release objects (Set var = Nothing) because they're automatically destroyed when leaving the context.
The reason why you still want to declare your variables is that you want to use Option Explicit in your scripts (which enforces variable declarations), so that you can avoid problems due to mistyped or otherwise uninitialized variables. Without Option Explicit VBScript would automagically create missing variables and initialize them with an empty/zero value. Silly example:
Dim foo : foo = 3
Dim bar : bar = 1
Do
bar = bar + fo 'mistyped variable, initilized as empty/0
Loop Until bar > 10
WScript.Echo bar
Running the above would create an infinite loop. If you add Option Explicit the script will instead immediately terminate with a runtime error:
C:\path\to\your.vbs(5, 3) Microsoft VBScript runtime error: Variable is undefined: 'fo'
The VBScript garbage collector runs at the end of every line to clear implicit variables and at the end of every procedure (end sub, end function, and end property) to clear explicit variables. Objects are similar but have added constraints. It works similar to VBA's garbage collector. By contrast JScript waits until 30,000 objects have gone out of scope before running and freeing memory.
An implicit variable is an unnamed variable - msgbox LCase(UCase("String") has two implicit variables - the result of UCase("String") and that is passed to LCase(implicitVar1) which returns implicitVar2 which is passed to Msgbox. An Explict variable is declared either by DIM or just by using it as in A=5 which creates an explicit variable called A.
VBScript, on the other hand, has a much simpler stack-based garbage collector. Scavengers are added to a stack when they come into scope, removed when they go ou t of scope, and any time an object is discarded it is immediately freed.
https://blogs.msdn.microsoft.com/ericlippert/2003/09/17/how-do-the-script-garbage-collectors-work/
VBScript’s garbage collector is completely different. It runs at the end of every statement and procedure, and does not do a search of all memory. Rather, it keeps track of everything allocated in the statement or procedure; if anything has gone out of scope, it frees it immediately
https://blogs.msdn.microsoft.com/ericlippert/2004/12/22/t4-vbscript-and-the-terminator/
Also
https://blogs.msdn.microsoft.com/ericlippert/2004/04/28/when-are-you-required-to-set-objects-to-nothing/
https://blogs.msdn.microsoft.com/ericlippert/2004/03/01/syntax-semantics-micronesian-cults-and-novice-programmers/
The CPU is a stack based machine (and VBScript a stack based virtual machine). When the CPU calls a function the calling program puts the parameters on the stack and the return address, adjust the stack frame and does a jump. The callee function creates local variables on the stack and also places the return value on it. When it returns the stack pointer is adjusted back to where it was which automatically frees all the above.
I am retrieving a byte blob from an SQLite database/record set. I am not experienced with garbage collection yet.
When I say:
Dim bt() As Byte
bt = r.fields("mybyteblob").value
... is that okay or unsafe?
I would like to make a copy of the byte array in the record set field, and I am not sure if I am simply referencing the byte array here instead of copying it.
In your code you are only referencing the byte array.
If you want to copy it you need
Dim bt() As Byte
if r.fields("mybyteblob").Value Is not Nothing then
dim lenArray = r.fields("mybyteblob").Length
bt = new Byte(lenArray)
Array.Copy(r.fields("mybyteblob").value, bt, lenArray)
end if
There is another alternative.
The Buffer class is faster than Array and more appropriate because you are using a byte array
Dim bt() As Byte
if r.fields("mybyteblob").Value Is not Nothing then
dim lenArray = r.fields("mybyteblob").Length
bt = new Byte(lenArray)
Buffer.BlockCopy(r.fields("mybyteblob").value, 0, bt, 0, lenArray)
end if
Here a good question on the two methods
It is very unusual to run into garbage collection problems if you only write managed code (i.e. no P/Invoke).
Many smart people has put a lot of effort into making garbage collection work without you having to worry about it, so do just that: don't worry about it. Just write your code, and if you run into a specific behavior you don't understand, then ask about that particular behavior (and I can assure you that in 99.9967% [1] of the cases it will not be the garbage collector).
[1] This is not a random number. I've ran into a garbage collection gotcha once in ~10 years of programming. Assuming 10 bugs a day and 300 days of work per year, that makes 29999/30000 bugs which are not garbage-collection related = 99.9967%.
This loop takes forever to run as the amount of items in the loop approach anything close to and over 1,000, close to like 10 minutes. This needs to run fast for amounts all the way up to like 30-40 thousand.
'Add all Loan Record Lines
Dim loans As List(Of String) = lar.CreateLoanLines()
Dim last As Integer = loans.Count - 1
For i = 0 To last
If i = last Then
s.Append(loans(i))
Else
s.AppendLine(loans(i))
End If
Next
s is a StringBuilder. The first line there
Dim loans As List(Of String) = lar.CreateLoanLines()
Runs in only a few seconds even with thousands of records. It's the actual loop that's taking a while.
How can this be optimized???
Set the initial capacity of your StringBuilder to a large value. (Ideally, large enough to contain the entire final string.) Like so:
s = new StringBuilder(loans.Count * averageExpectedStringSize)
If you don't specify a capacity, the builder will likely end up doing a large amount of internal reallocations, and this will kill performance.
You could take the special case out of the loop, so you wouldn't need to be checking it inside the loop. I would expect this to have almost no impact on performance, however.
For i = 0 To last - 1
s.AppendLine(loans(i))
Next
s.Append(loans(last))
Though, internally, the code is very similar, if you're using .NET 4, I'd consider replacing your method with a single call to String.Join:
Dim result as String = String.Join(Envionment.NewLine, lar.CreateLoanLines())
I can't see how the code you have pointed out could be slow unless:
The strings you are dealing with are huggggge (e.g. if the resulting string is 1 gigabyte).
You have another process running on your machine consuming all your clock cycles.
You haven't got enough memory in your machine.
Try stepping through the code line by line and check that the strings contain the data that you expect, and check Task Manager to see how much memory your application is using and how much free memory you have.
My guess would be that every time you're using append it's creating a new string. You seem to know how much memory you'll need, if you allocate all of the memory first and then just copy it into memory it should run much faster. Although I may be confused as to how vb.net works.
You could look at doing this another way.
Dim str As String = String.Join(Environment.NewLine, loans.ToArray)