I am retrieving a byte blob from an SQLite database/record set. I am not experienced with garbage collection yet.
When I say:
Dim bt() As Byte
bt = r.fields("mybyteblob").value
... is that okay or unsafe?
I would like to make a copy of the byte array in the record set field, and I am not sure if I am simply referencing the byte array here instead of copying it.
In your code you are only referencing the byte array.
If you want to copy it you need
Dim bt() As Byte
if r.fields("mybyteblob").Value Is not Nothing then
dim lenArray = r.fields("mybyteblob").Length
bt = new Byte(lenArray)
Array.Copy(r.fields("mybyteblob").value, bt, lenArray)
end if
There is another alternative.
The Buffer class is faster than Array and more appropriate because you are using a byte array
Dim bt() As Byte
if r.fields("mybyteblob").Value Is not Nothing then
dim lenArray = r.fields("mybyteblob").Length
bt = new Byte(lenArray)
Buffer.BlockCopy(r.fields("mybyteblob").value, 0, bt, 0, lenArray)
end if
Here a good question on the two methods
It is very unusual to run into garbage collection problems if you only write managed code (i.e. no P/Invoke).
Many smart people has put a lot of effort into making garbage collection work without you having to worry about it, so do just that: don't worry about it. Just write your code, and if you run into a specific behavior you don't understand, then ask about that particular behavior (and I can assure you that in 99.9967% [1] of the cases it will not be the garbage collector).
[1] This is not a random number. I've ran into a garbage collection gotcha once in ~10 years of programming. Assuming 10 bugs a day and 300 days of work per year, that makes 29999/30000 bugs which are not garbage-collection related = 99.9967%.
Related
I am trying to program a way to read a text file and match all the values and their quantites. For example if the text file is like this:
Bread-10 Flour-2 Orange-2 Bread-3
I want to create a list with the total quantity of all the common words. I began my code, but I am having trouble understanding to to sum the values. I'm not asking for anyone to write the code for me but I am having trouble finding resources. I have the following code:
Dim query = From data In IO.File.ReadAllLines("C:\User\Desktop\doc.txt")
Let name As String = data.Split("-")(0)
Let quantity As Integer = CInt(data.Split("-")(1))
Let sum As Integer = 0
For i As Integer = 0 To query.Count - 1
For j As Integer = i To
Next
Thanks
Ok, lets break this down. And I not seen the LET command used for a long time (back in the GWBASIC days!).
But, that's ok.
So, first up, we going to assume your text file is like this:
Bread-10
Flour-2
Orange-2
Bread-3
As opposed to this:
Bread-10 Flour-2 Orange-2 Bread-3
Now, we could read one line, and then process the information. Or we can read all lines of text, and THEN process the data. If the file is not huge (say a few 100 lines), then performance is not much of a issue, so lets just read in the whole file in one shot (and your code also had this idea).
Your start code is good. So, lets keep it (well ok, very close).
A few things:
We don't need the LET for assignment. While older BASIC languages had this, and vb.net still supports this? We don't need it. (but you will see examples of that still floating around in vb.net - especially for what we call "class" module code, or "custom classes". But again lets just leave that for another day.
Now the next part? We could start building up a array, look for the existing value, and then add it. However, this would require a few extra arrays, and a few extra loops.
However, in .net land, we have a cool thing called a dictionary.
And that's just a fancy term of for a collection VERY much like an array, but it has some extra "fancy" features. The fancy feature is that it allows one to put into the handly list things by a "key" name, and then pull that "value" out by the key.
This saves us a good number of extra looping type of code.
And it also means we don't need a array for the results.
This key system is ALSO very fast (behind the scene it uses some cool concepts - hash coding).
So, our code to do this would look like this:
Note I could have saved a few lines here or there - but that would make this code hard to read.
Given that you look to have Fortran, or older BASIC language experience, then lets try to keep the code style somewhat similar. it is stunning that vb.net seems to consume even 40 year old GWBASIC type of syntax here.
Do note that arrays() in vb.net do have some fancy "find" options, but the dictionary structure is even nicer. It also means we can often traverse the results with out say needing a for i = 1 to end of array, and having to pull out values that way.
We can use for each.
So this would work:
Dim MyData() As String ' an array() of strings - one line per array
MyData = File.ReadAllLines("c:\test5\doc.txt") ' read each line to array()
Dim colSums As New Dictionary(Of String, Integer) ' to hold our values and sum them
Dim sKey As String
Dim sValue As Integer
For Each strLine As String In MyData
sKey = Split(strLine, "-")(0)
sValue = Split(strLine, "-")(1)
If colSums.ContainsKey(sKey) Then
colSums(sKey) = colSums(sKey) + sValue
Else
colSums.Add(sKey, sValue)
End If
Next
' display results
Dim KeyPair As KeyValuePair(Of String, Integer)
For Each KeyPair In colSums
Debug.Print(KeyPair.Key & " = " & KeyPair.Value)
Next
The above results in this output in the debug window:
Bread = 13
Flour = 2
Orange = 2
I was tempted here to write this code using just pure array() in vb.net, as that would give you a good idea of the "older" types of coding and syntax we could use here, and a approach that harks all the way back to those older PC basic systems.
While the dictionary feature is more advanced, it is worth the learning curve here, and it makes this problem a lot easier. I mean, if this was for a longer list? Then I would start to consider introduction of some kind of data base system.
However, without some data system, then the dictionary feature is a welcome approach due to that "key" value lookup ability, and not having to loop. It also a very high speed system, so the result is not much looping code, and better yet we write less code.
Ok so I was in college and I was talking to my teacher and he said my code isn't good practice. I'm a bit confused as to why so here's the situation. We basically created a for loop however he declared his for loop counter outside of the loop because it's considered good practice (to him) even though we never used the variable later on in the code so to me it looks like a waste of memory. We did more to the code then just use a message box but the idea was to get each character from a string and do something with it. He also used the Mid() function to retrieve the character in the string while I called the variable with the index. Here's an example of how he would write his code:
Dim i As Integer = 0
Dim justastring As String = "test"
For i = 1 To justastring.Length Then
MsgBox( Mid( justastring, i, 1 ) )
End For
And here's an example of how I would write my code:
Dim justastring As String = "test"
For i = 0 To justastring.Length - 1 Then
MsgBox( justastring(i) )
End For
Would anyone be able to provide the advantages and disadvantages of each method and why and whether or not I should continue how I am?
Another approach would be, to just use a For Each on the string.
Like this no index variable is needed.
Dim justastring As String = "test"
For Each c As Char In justastring
MsgBox(c)
Next
I would suggest doing it your way, because you could have variables hanging around consuming(albeit a small amount) of memory, but more importantly, It is better practice to define objects with as little scope as possible. In your teacher's code, the variable i is still accessible when the loop is finished. There are occasions when this is desirable, but normally, if you're only using a variable in a limited amount of code, then you should only declare it within the smallest block that it is needed.
As for your question about the Mid function, individual characters as you know can be access simply by treating the string as an array of characters. After some basic benchmarking, using the Mid function takes a lot longer to process than just accessing the character by the index value. In relatively simple bits of code, this doesn't make much difference, but if you're doing it millions of times in a loop, it makes a huge difference.
There are other factors to consider. Such as code readability and modification of the code, but there are plenty of websites dealing with that sort of thing.
Finally I would suggest changing some compiler options in your visual studio
Option Strict to On
Option Infer to Off
Option Explicit to On
It means writing more code, but the code is safer and you'll make less mistakes. Have a look here for an explanation
In your code, it would mean that you have to write
Dim justastring As String = "test"
For i As Integer = 0 To justastring.Length - 1 Then
MsgBox( justastring(i) )
End For
This way, you know that i is definitely an integer. Consider the following ..
Dim i
Have you any idea what type it is? Me neither.
The compiler doesn't know what so it defines it as an object type which could hold anything. a string, an integer, a list..
Consider this code.
Dim i
Dim x
x = "ab"
For i = x To endcount - 1
t = Mid(s, 999)
Next
The compiler will compile it, but when it is executed you'll get an SystemArgumenException. In this case it's easy to see what is wrong, but often it isn't. And numbers in strings can be a whole new can of worms.
Hope this helps.
The following VB .net code gives me an out of memory exception. Does anybody knows why?
Dim vArray As ILArray(Of Double) = ILMath.rand(10000000)
Using ILScope.Enter(vArray)
For i As Integer = 1 To 100
vArray = ILMath.add(vArray, vArray)
Next
End Using
Thank you very much.
In this toy example you can simply remove the artificial scope and it will run fine:
Dim vArray As ILArray(Of Double) = ILMath.rand(10000000)
For i As Integer = 1 To 100
vArray = ILMath.add(vArray, vArray)
Next
Console.WriteLine("OK: " + vArray(0).ToString())
Console.ReadKey()
However, in a more serious situation, ILScope will be your friend. As stated on the ILNumerics page an artificial scope ensures a deterministic memory management:
All arrays created inside the scope are disposed once the block was
left.
Otherwise one had to rely on the GC for cleanup. And, as you know, this involves a gen 2 collection for large objects – with all disadvantages in terms of performance.
In order to be able to dispose the arrays they need to be collected and tracked somehow. Whether or not this qualifies for the term 'memory leak' is rather a philosophical question. I will not go into it here. The deal is: after the instruction pointer runs out of the scope these arrays are taken care of: their memory is put into the memory pool and will be reused. As a consequence, no GC will be triggered.
The scheme is especially useful for long running operations and for large data. Currently, the arrays are released only AFTER the scope block was left. So if you create an algorithm/ loop which requires more memory than available on your machine you need to clean up DURING the loop already:
Dim vArray As ILArray(Of Double) = ILMath.rand(10000000)
For i As Integer = 1 To 100
Using ILScope.Enter
vArray.a = ILMath.add(vArray, vArray)
' ...
End Using
Next
Here, the scope cleans up the memory after each iteration of the loop. This affects all local arrays assigned within the loop body. If we want an array value to survive the loop iteration we can assign to its .a property as shown with vArray.a.
This loop takes forever to run as the amount of items in the loop approach anything close to and over 1,000, close to like 10 minutes. This needs to run fast for amounts all the way up to like 30-40 thousand.
'Add all Loan Record Lines
Dim loans As List(Of String) = lar.CreateLoanLines()
Dim last As Integer = loans.Count - 1
For i = 0 To last
If i = last Then
s.Append(loans(i))
Else
s.AppendLine(loans(i))
End If
Next
s is a StringBuilder. The first line there
Dim loans As List(Of String) = lar.CreateLoanLines()
Runs in only a few seconds even with thousands of records. It's the actual loop that's taking a while.
How can this be optimized???
Set the initial capacity of your StringBuilder to a large value. (Ideally, large enough to contain the entire final string.) Like so:
s = new StringBuilder(loans.Count * averageExpectedStringSize)
If you don't specify a capacity, the builder will likely end up doing a large amount of internal reallocations, and this will kill performance.
You could take the special case out of the loop, so you wouldn't need to be checking it inside the loop. I would expect this to have almost no impact on performance, however.
For i = 0 To last - 1
s.AppendLine(loans(i))
Next
s.Append(loans(last))
Though, internally, the code is very similar, if you're using .NET 4, I'd consider replacing your method with a single call to String.Join:
Dim result as String = String.Join(Envionment.NewLine, lar.CreateLoanLines())
I can't see how the code you have pointed out could be slow unless:
The strings you are dealing with are huggggge (e.g. if the resulting string is 1 gigabyte).
You have another process running on your machine consuming all your clock cycles.
You haven't got enough memory in your machine.
Try stepping through the code line by line and check that the strings contain the data that you expect, and check Task Manager to see how much memory your application is using and how much free memory you have.
My guess would be that every time you're using append it's creating a new string. You seem to know how much memory you'll need, if you allocate all of the memory first and then just copy it into memory it should run much faster. Although I may be confused as to how vb.net works.
You could look at doing this another way.
Dim str As String = String.Join(Environment.NewLine, loans.ToArray)
the message was variable j is used before it have been assigned value.If in php language, it's not a problem at all.
Dim j() As String
Dim i As Integer
If (i = 1) Then
j(0) = "x"
Else
j(0) = "d"
End If
in php language,it's not a problem at all.
php doesn't use real arrays. A real array is a contiguous block of memory. Php uses a collection type with array-like semantics. They call it an array, but it's not really. Collections seldom make the same kind of guarantees about continuity because of performance problems caused when the collection grows at runtime.
If you want php-style "arrays" in .Net, you need to do the same thing php does and use a collection. The System.Collections.Generics.List<T> type works pretty well. Then you can append to the List by using its .Add() method or using the collection initializer syntax demonstrated below (requires Visual Studio 2010):
Dim i As Integer = 1
Dim j As New List(Of String) From { If(i = 1, "x","d") }
We can forgive php it's naming lapse, though, as real arrays should be used sparingly. A collection is almost always more appropriate.