Free up Memory: How to delete variables once am don with them- VBA VB ACCESS - vba

How do i free up Memory?
Say I have a string
Dim TestStri As String
TestStri = "Test"
' What do i have to type up to get rid of the variable?
' I know
TestStri = Nothing
' will give it the default value, but the variable is still there.
Can I use the same Method for other variables i.e. Long, int etc.

I'm assuming you are referring to VB6 and VBA as indicated by your title, not VB.Net, as indicated by a keyword.
In VB6 and VBA the memory consumption of a string variable consists of a fixed part for the string's length and a terminator and a variable length part for the string contents itself. See http://www.aivosto.com/vbtips/stringopt2.html#memorylayout for a good explanation of this.
So, when you set the string variable to an empty string or vbNullString, you will be freeing up the variable part of the string but not the fixed part.
Other types like long, int, bool and date consume a fixed amount of memory.
You can't "free" local variables in VB completely (come to think of it, is there ANY programming language where you can do that?), and for the most part, you wouldn't care because the local variables themselves (the fixed portion) is usually very small.
The only case I can think of where the memory consumption of local varibles could get big is if you have recursive function calls with deep recursion/wide recursion.

I went a differs route :
I was hoping MemoryUsage would be useful. It wasn't, apparently...
I run a vba script that goes through multiple files (since access cannot handle anything too large); and append them to a table, transform it and then spit out a summary.
The script loops through files and runs macros against each of them.
The quick answer is to pull the memory usage from the task manager and then if it exceeds 1 GB; pause the subroutine so no corrupt records get in.
How do we do this?
Insert this memory usage Function with the readfile function.
You will need to create an if statement in your code that says:
dim memory as long
memory = memory_usage
' 1000000 ~ 1 GB
If memory > 1000000 then
End Sub
end if
=================================================
[path to file] = "C:\….\ShellOutputfile.txt"
Function Memory_Usage() as Long
Dim lines As Long
Dim linestring As String
Shell "tasklist /fi " & """IMAGENAME EQ MSACCESS.EXE""" & ">" & """[path to file]"""
'get_list_data
lines = CInt(get_listing_data("[path to file]", 1, 0))
linestring = get_listing_data("[path to file]", 2, 4)
linestring = Right(linestring, 11)
linestring = Replace(linestring, " K", "") ' K
linestring = Replace(linestring, " ", "")
lines = CLng(linestring)
Memory_Usage = lines
End Function
=============================
Public Function get_listing_data(PATH As String, Choice As Integer, typeofreading As Integer) As String
' parse in the variable, of which value you need.
Const ForReading = 1, ForWriting = 2, ForAppending = 8
Dim tmp_var_str As String
Dim fso, ts, fileObj, filename
Dim textline As String
Dim tmp_result As String
Dim TMP_PATH As String
Dim tmpchoice As Integer
Dim tor As Integer
Dim counter As Integer
' type of reading determines what loop is used
' type of reading = 0; to bypass; > 0, you are choosing a line to read.
counter = 0
TMP_PATH = PATH
tmp_var_str = var_str
tmp_result = ""
tor = typeofreading
' choice = 1 (count the lines)
' choice = 2 (read a specific line)
tmpchoice = Choice
' Create the file, and obtain a file object for the file.
If Right(PATH, 1) = "\" Then TMP_PATH = Left(PATH, Len(PATH) - 1)
filename = TMP_PATH '& "\Profit_Recognition.ini"
Set fso = CreateObject("Scripting.FileSystemObject")
Set fileObj = fso.GetFile(filename)
' Open a text stream for output.
Set ts = fileObj.OpenAsTextStream(ForReading, TristateUseDefault)
Do While ts.AtEndOfStream <> True
If tmpchoice = 1 Then
counter = counter + 1
textline = ts.ReadLine
tmp_result = CStr(counter)
End If
If tmpchoice = 2 Then
counter = counter + 1
tmp_result = ts.ReadLine
If counter = tor Then
Exit Do
End If
End If
Loop
get_listing_data = tmp_result
End Function

Related

Append a variant of strings to an empty variant array

I would like to repeatedly append an array of string values to a master array, which is initially empty. I cannot get it to append.
Sub main()
Dim num As Integer, root As String, pathToFile As String, allOf As Variant, someOf As Variant
Dim i As Integer, opts() As String, val As Integer
root = Application.ActiveWorkbook.Path
pathToFile = root & "\" & "name" & ".txt"
num = 5 ' the number of files I may have
For i = 0 To num - 1 ' loop over all the files
ReDim Preserve opts(i)
someOf = read_whole_file(pathToFile) ' read the file into variant
For val = LBound(someOf) To UBound(someOf) ' run through the array
' -- append someOf to allOf and loop
Dim Nmbr As Integer
On Error Resume Next
Err.Clear
If allOf.Value = "Empty" Then
Nmbr = UBound(allOf)
allOf(0) = someOf(0)
Else
ReDim Preserve allOf(UBound(allOf) + 1)
allOf(UBound(allOf)) = someOf(val)
End If
Next val
Next i
End Sub
Function read_whole_file(filePath As String) As Variant
Dim sWhole As String
Open filePath For Input As #1
sWhole = Input$(LOF(1), 1)
Close #1
read_whole_file = Split(sWhole, vbNewLine)
End Function
Contents of the text file :
"
Hello
This
Is a
Text
File
"
This is VBA. DOn't use an array use a collection.
Sub main()
Dim num As Integer, root As String, pathToFile As String, allOf As Collection, someOf As Variant
Set allof = new collection
Dim i As Integer, opts() As String, val As Variant
root = Application.ActiveWorkbook.Path
pathToFile = root & "\" & "name" & ".txt"
num = 5 ' the number of files I may have
For i = 0 To num - 1 ' loop over all the files
someOf = read_whole_file(pathToFile) ' read the file into variant
For Each val In someOf ' run through the array
alloff.Add val
Next
Next i
End Sub
In your code, you say you have 5 files:
num = 5 ' the number of files I may have
but you set the path_to_file immediately before this. More importantly, you do not change path_to_file within the loop, nor do you pass any modifier to read_whole_file. So you code will read from the same file 5 times.
You also don't set any value to allOf before you use it. You don't even identify what type it is (apart from Variant), so checking .Value is meaningless and should result in a compile error. Because of your On Error statement, that section is probably being ignored, thus the intended action is not occurring.
How to fix:
Add Option Explicit at the top of the module. Always.
Remove the On Error handling - if you think you may have some out of bounds issues, then cater for it in the program logic.
Bonus: Instead of ReDimming opts(I) each time in the loop, you already know how many iterations, so only ReDim it once before entering the loop.

Array Out of bounds error VB

Sorry for the terrible wording on my last question, I was half asleep and it was midnight. This time I'll try to be more clear.
I'm currently writing some code for a mini barcode scanner and stock manager program. I've got the input and everything sorted out, but there is a problem with my arrays.
I'm currently trying to extract the contents of the stock file and sort them out into product tables.
This is my current code for getting the data:
Using fs As StreamReader = New StreamReader("The File Path (Is private)")
Dim line As String = "ERROR"
line = fs.ReadLine()
While line <> Nothing
Dim pos As Integer = 0
Dim split(3) As String
pos = products.Length
split = line.Split("|")
productCodes(productCodes.Length) = split(0)
products(products.Length, 0) = split(1)
products(products.Length, 1) = split(2)
products(products.Length, 2) = split(3)
line = fs.ReadLine()
End While
End Using
I have made sure that the file path does, in fact, go to the file. I have looked through debug to find that all the data is going through into my "split" table. The error throws as soon as I start trying to transfer the data.
This is where I declare the two tables being used:
Dim productCodes() As String = {}
Dim products(,) As Object = {}
Can somebody please explain why this is happening?
Thanks in advance
~Hydro
By declaring the arrays like you did:
Dim productCodes() As String = {}
Dim products(,) As Object = {}
You are assigning size 0 to all your arrays, so during your loop, it will eventually try to access a position that haven't been previously declared to the compiler. It is the same as declaring an array of size 10 Dim MyArray(10) and try to access the position 11 MyArray(11) = something.
You should either declare it with a proper size, or redim it during execution time:
Dim productCodes(10) As String
or
Dim productCodes() As String
Dim Products(,) As String
Dim Position as integer = 0
'code here
While line <> Nothing
Redim Preserve productCodes(Position)
Redim Preserve products(2,Position)
Dim split(3) As String
pos = products.Length
split = line.Split("|")
productCodes(Position) = split(0)
products(0,Position) = split(1)
products(1,Position) = split(2)
products(2,Position) = split(3)
line = fs.ReadLine()
Position+=1
End While

Creating Fixed Width files from strings

I have searched high and low on the internet and I can't find a straight answer to this !
I have a file that has approx 100,000 characters in one long line.
I need to read this file in and write it out again in its entirety, in lines 102 character long ending with VbCrLf. There are no delimiters.
I thought there were a number of ways to tackle issues like this in VB Script... but
apparently not !
Can anyone please provide me with a pointer ?
Here's something (off the top of my head - untested!) that should get you started.
Const ForReading = 1
Const ForWriting = 2
Dim sNewLine
Set fso = CreateObject("Scripting.FileSystemObject")
Set tsIn = fso.OpenTextFile("OldFile.txt", ForReading) ' Your input file
Set tsOut = fso.OpenTextFile("NewFile.txt", ForWriting) ' New (output) file
While Not tsIn.AtEndOfStream ' While there is still text
sNewLine = tsIn.Read(102) ' Read 120 characters
tsOut.Write sNewLine & vbCrLf ' Write out to new file + CR/LF
Wend ' Loop to repeat
tsIn.Close
tsOut.Close
I won't cover the reading of files, since that is stuff you can find everywhere. And since it's been years I've coded in vb or vbscript, I hope that .net code will suffice.
pseudo: read line from file, put it in for example a string (performance issues anyone?).
A simple algorithm would be and this might have performance issues (multithreading, parallel could be a solution):
Public Sub foo()
Dim strLine As String = "foo²"
Dim strLines As List(Of String) = New List(Of String)
Dim nrChars = strLine.ToCharArray.Count
Dim iterations = nrChars / 102
For i As Integer = 0 To iterations - 1
strLines.Add(strLine.Substring(0, 102))
strLine = strLine.Substring(103)
Next
'save it to file
End Sub

Is there any way I can speed up this VBA algorithm?

I am looking to implement a VBA trie-building algorithm that is able to process a substantial English lexicon (~50,000 words) in a relatively short amount of time (less than 15-20 seconds). Since I am a C++ programmer by practice (and this is my first time doing any substantial VBA work), I built a quick proof-of-concept program that was able to complete the task on my computer in about half a second. When it came time to test the VBA port however, it took almost two minutes to do the same -- an unacceptably long amount of time for my purposes. The VBA code is below:
Node Class Module:
Public letter As String
Public next_nodes As New Collection
Public is_word As Boolean
Main Module:
Dim tree As Node
Sub build_trie()
Set tree = New Node
Dim file, a, b, c As Integer
Dim current As Node
Dim wordlist As Collection
Set wordlist = New Collection
file = FreeFile
Open "C:\corncob_caps.txt" For Input As file
Do While Not EOF(file)
Dim line As String
Line Input #file, line
wordlist.add line
Loop
For a = 1 To wordlist.Count
Set current = tree
For b = 1 To Len(wordlist.Item(a))
Dim match As Boolean
match = False
Dim char As String
char = Mid(wordlist.Item(a), b, 1)
For c = 1 To current.next_nodes.Count
If char = current.next_nodes.Item(c).letter Then
Set current = current.next_nodes.Item(c)
match = True
Exit For
End If
Next c
If Not match Then
Dim new_node As Node
Set new_node = New Node
new_node.letter = char
current.next_nodes.add new_node
Set current = new_node
End If
Next b
current.is_word = True
Next a
End Sub
My question then is simply, can this algorithm be sped up? I saw from some sources that VBA Collections are not as efficient as Dictionarys and so I attempted a Dictionary-based implementation instead but it took an equal amount of time to complete with even worse memory usage (500+ MB of RAM used by Excel on my computer). As I say I am extremely new to VBA so my knowledge of both its syntax as well as its overall features/limitations is very limited -- which is why I don't believe that this algorithm is as efficient as it could possibly be; any tips/suggestions would be greatly appreciated.
Thanks in advance
NB: The lexicon file referred to by the code, "corncob_caps.txt", is available here (download the "all CAPS" file)
There are a number of small issues and a few larger opportunities here. You did say this is your first vba work, so forgive me if I'm telling you things you already know
Small things first:
Dim file, a, b, c As Integer declares file, a and b as variants. Integer is 16 bit sign, so there may be risk of overflows, use Long instead.
DIM'ing inside loops is counter-productive: unlike C++ they are not loop scoped.
The real opportunity is:
Use For Each where you can to iterate collections: its faster than indexing.
On my hardware your original code ran in about 160s. This code in about 2.5s (both plus time to load word file into the collection, about 4s)
Sub build_trie()
Dim t1 As Long
Dim wd As Variant
Dim nd As Node
Set tree = New Node
' Dim file, a, b, c As Integer : declares file, a, b as variant
Dim file As Integer, a As Long, b As Long, c As Long ' Integer is 16 bit signed
Dim current As Node
Dim wordlist As Collection
Set wordlist = New Collection
file = FreeFile
Open "C:\corncob_caps.txt" For Input As file
' no point in doing inside loop, they are not scoped to the loop
Dim line As String
Dim match As Boolean
Dim char As String
Dim new_node As Node
Do While Not EOF(file)
'Dim line As String
Line Input #file, line
wordlist.Add line
Loop
t1 = GetTickCount
For Each wd In wordlist ' for each is faster
'For a = 1 To wordlist.Count
Set current = tree
For b = 1 To Len(wd)
'Dim match As Boolean
match = False
'Dim char As String
char = Mid$(wd, b, 1)
For Each nd In current.next_nodes
'For c = 1 To current.next_nodes.Count
If char = nd.letter Then
'If char = current.next_nodes.Item(c).letter Then
Set current = nd
'Set current = current.next_nodes.Item(c)
match = True
Exit For
End If
Next nd
If Not match Then
'Dim new_node As Node
Set new_node = New Node
new_node.letter = char
current.next_nodes.Add new_node
Set current = new_node
End If
Next b
current.is_word = True
Next wd
Debug.Print "Time = " & GetTickCount - t1 & " ms"
End Sub
EDIT:
loading the word list into a dynamic array will reduce load time to sub second. Be aware that Redim Preserve is expensive, so do it in chunks
Dim i As Long, sz As Long
sz = 10000
Dim wordlist() As String
ReDim wordlist(0 To sz)
file = FreeFile
Open "C:\corncob_caps.txt" For Input As file
i = 0
Do While Not EOF(file)
'Dim line As String
Line Input #file, line
wordlist(i) = line
i = i + 1
If i > sz Then
sz = sz + 10000
ReDim Preserve wordlist(0 To sz)
End If
'wordlist.Add line
Loop
ReDim Preserve wordlist(0 To i - 1)
then loop through it like
For i = 0 To UBound(wordlist)
wd = wordlist(i)
I'm out of practice with VBA, but IIRC, iterating the Collection using For Each should be a bit faster than going numerically:
Dim i As Variant
For Each i In current.next_nodes
If i.letter = char Then
Set current = i
match = True
Exit For
End If
Next node
You're also not using the full capabilities of Collection. It's a Key-Value map, not just a resizeable array. You might get better performance if you use the letter as a key, though looking up a key that isn't present throws an error, so you have to use an ugly error workaround to check for each node. The inside of the b loop would look like:
Dim char As String
char = Mid(wordlist.Item(a), b, 1)
Dim node As Node
On Error Resume Next
Set node = Nothing
Set node = current.next_nodes.Item(char)
On Error Goto 0
If node Is Nothing Then
Set node = New Node
current.next_nodes.add node, char
Endif
Set current = node
You won't need the letter variable on class Node that way.
I didn't test this. I hope it's all right...
Edit: Fixed the For Each loop.
Another thing you can do which will possibly be slower but will use less memory is use an array instead of a collection, and resize with each added element. Arrays can't be public on classes, so you have to add methods to the class to deal with it:
Public letter As String
Private next_nodes() As Node
Public is_word As Boolean
Public Sub addNode(new_node As Node)
Dim current_size As Integer
On Error Resume Next
current_size = UBound(next_nodes) 'ubound throws an error if the array is not yet allocated
On Error GoTo 0
ReDim next_nodes(0 To current_size) As Node
Set next_nodes(current_size) = new_node
End Sub
Public Function getNode(letter As String) As Node
Dim n As Variant
On Error Resume Next
For Each n In next_nodes
If n.letter = letter Then
Set getNode = n
Exit Function
End If
Next
End Function
Edit: And a final optimization strategy, get the Integer char value with the Asc function and store that instead of a String.
You really need to profile it, but if you think Collections are slow maybe you can try using dynamic arrays?

Function to count number of lines in a text file

Need a function that will accept a filename as parameter and then return the number of lines in that file.
Should be take under 30 seconds to get the count of a 10 million line file.
Currently have something along the lines of - but it is too slow with large files:
Dim objFSO, strTextFile, strData, arrLines, LineCount
CONST ForReading = 1
'name of the text file
strTextFile = "sample.txt"
'Create a File System Object
Set objFSO = CreateObject("Scripting.FileSystemObject")
'Open the text file - strData now contains the whole file
strData = objFSO.OpenTextFile(strTextFile,ForReading).ReadAll
'Split by lines, put into an array
arrLines = Split(strData,vbCrLf)
'Use UBound to count the lines
LineCount = UBound(arrLines) + 1
wscript.echo LineCount
'Cleanup
Set objFSO = Nothing
If somebody still looking for faster way, here is the code:
Const ForAppending = 8
Set fso = CreateObject("Scripting.FileSystemObject")
Set theFile = fso.OpenTextFile("C:\textfile.txt", ForAppending, Create:=True)
WScript.Echo theFile.Line
Set Fso = Nothing
Of course, the processing time depend very much of the file size, not only of the lines number. Compared with the RegEx method TextStream.Line property is at least 3 times quicker.
The only alternative I see is to read the lines one by one (EDIT: or even just skip them one by one) instead of reading the whole file at once. Unfortunately I can't test which is faster right now. I imagine skipping is quicker.
Dim objFSO, txsInput, strTemp, arrLines
Const ForReading = 1
Set objFSO = CreateObject("Scripting.FileSystemObject")
strTextFile = "sample.txt"
txsInput = objFSO.OpenTextFile(strTextFile, ForReading)
'Skip lines one by one
Do While txsInput.AtEndOfStream <> True
txsInput.SkipLine ' or strTemp = txsInput.ReadLine
Loop
wscript.echo txsInput.Line-1 ' Returns the number of lines
'Cleanup
Set objFSO = Nothing
Incidentally, I took the liberty of removing some of your 'comments. In terms of good practice, they were superfluous and didn't really add any explanatory value, especially when they basically repeated the method names themselves, e.g.
'Create a File System Object
... CreateObject("Scripting.FileSystemObject")
Too large files...
The following is the fastest-effeciently way I know of:
Dim oFso, oReg, sData, lCount
Const ForReading = 1, sPath = "C:\file.txt"
Set oReg = New RegExp
Set oFso = CreateObject("Scripting.FileSystemObject")
sData = oFso.OpenTextFile(sPath, ForReading).ReadAll
With oReg
.Global = True
.Pattern = "\r\n" 'vbCrLf
'.Pattern = "\n" ' vbLf, Unix style line-endings
lCount = .Execute(sData).Count + 1
End With
WScript.Echo lCount
Set oFso = Nothing
Set oReg = Nothing
You could try some variation on this
cnt = 0
Set fso = CreateObject("Scripting.FileSystemObject")
Set theFile = fso.OpenTextFile(filespec, ForReading, False)
Do While theFile.AtEndOfStream <> True
theFile.SkipLine
c = c + 1
Loop
theFile.Close
WScript.Echo c,"lines"
txt = "c:\YourTxtFile.txt"
j = 0
Dim read
Open txt For Input As #1
Do While Not EOF(1)
Input #1, read
j = j + 1
Loop
Close #1
If it adds an empty last line the result is (j - 1).
It works fine for one column in the txt file.
How to count all lines in the notepad
Answers:
=> Below is the code -
Set t1=createObject("Scripting.FileSystemObject")
Set t2=t1.openTextFile ("C:\temp\temp1\temp2_VBSCode.txt",1)
Do Until t2.AtEndOfStream
strlinenumber = t2.Line
strLine = t2.Readline
Loop
msgbox strlinenumber
t2.Close
I was looking for a faster way than what I already had to determine the number of lines in a text file. I searched the internet and came across 2 promising solution. One was a solution based on SQL thew other the solution I found here based on Fso by Kul-Tigin. I tested them and this is part of the result:
Number of lines Time elapsed Variant
--------------------------------------------------------
110 00:00:00.70 SQL
110 00:00:00.00 Vanilla VBA (my solution)
110 00:00:00.16 FSO
--------------------------------------------------------
1445014 00:00:17.25 SQL
1445014 00:00:09.19 Vanilla VBA (my solution)
1445014 00:00:17.73 FSO
I ran this several times with large and small numbers. Time and again the vanilla VBA came out on top. I know this is far out of date, but for anyone still looking for the fastest way to determine the number of lines in a csv/text file, down here's the code I use.
Public Function GetNumRecs(ASCFile As String) As Long
Dim InStream As Long
Dim Record As String
InStream = FreeFile
GetNumRecs = 0
Open ASCFile For Input As #InStream
Do While Not EOF(InStream)
Line Input #InStream, Record
GetNumRecs = GetNumRecs + 1
Loop
Close #InStream
End Function