"System resource exceeded" when updating recordset, and possibly at other points - vba

I'm experiencing an intermittent issue with an application that's Excel 2010 front-end, Access 2010 back end. It's in use by 5-10 users simultaneously. Recently, users have started intermittently receiving the following error:
Run-time error '3035': System resource exceeded.
Sometimes the Debug button is grayed out so I can't jump to the code that caused the error, but when it's available to click, it takes me to the following code:
'Open connection to back end DB
Set db = OpenDatabase(dbPath)
'Open a recordset of a table
Set RS = db.OpenRecordset(Tbl)
'loop through rows in a 2D array
For i = FR To LR
RS.AddNew
'loop through columns of the 2D array
For j = 1 to LC
'set values for various fields in the new record, using values from the array
Next
RS.Update
Next
Here, the RS.Update is marked as the line that's causing the error.
What's odd is that this problem comes and goes; users will repeatedly receive it when attempting to submit a certain data set, then, several hours later, when they try to submit the same data set again, the operation succeeds without the error. It's also perplexing that sometimes the Debug button is available and sometimes it isn't.
One issue might be the size of the Access back end; it's currently ~650 MB, and we didn't start getting these messages until it grew to around 600 MB.
Any ideas as to what could be causing this? Various Google hits indicate that this problem sometimes happens when a join query has too many fields, but this is just a recordset of a table, not a join query.

Ahh, methink that this is one of the strange errors that pop-up when you write a lot to a backend database that can't keep up with the management of the lock file.
The solution is to make sure you keep a connection open to the back-end database from each of your client and that you hold onto that connection until you close the client.
Just open a recordset to a table (say a dummy table with only one record), and keep that recordset open until you close the application.
Resource-wise, keeping this connection open will have no detrimental effect on performance or memory consumption, but it will ensure that the lock file is not continuously created/deleted every time a connection is open then closed.
Keeping that connection open will also substantially increase the performance of your data access.
Edit:
You should be more explicit when using recordsets and specify exactly the mode of operation you need:
Set RS = db.OpenRecordset(Tbl, dbOpenTable, dbFailOnError)
or, faster if you are only appending data:
Set RS = db.OpenRecordset(Tbl, dbOpenTable, dbAppendOnly + dbFailOnError)
Also make absolutely sure you close the recordset once you're finished with appending the data!:
Set RS = db.OpenRecordset(Tbl, dbOpenTable, dbAppendOnly + dbFailOnError)
With RS
'loop through rows in a 2D array
For i = FR To LR
.AddNew
'loop through columns of the 2D array
For j = 1 To LC
'set values for various fields in the new record,
'using values from the array
Next
.Update
Next
.Close
End With
Set RS = Nothing

This is caused by running out of available virtual memory (VM) aka swap disk. A 32 bit app cannot use more than 2gb and for some reason Access uses a lot of VM and when it needs more and cannot get any then you run out of system resources.
Solution is to make sure your VM is at least 4 times the RAM and to restart your PC at least daily, only this clears out the VM from garbage left lying around from other apps.
You will never have had this issue on a 32 bit OS, its only now with 64 bit OS that this happens.

Short Story
My 3035 error was caused by an add.new to a file without a primary key.
Composing a copy of the file with a primary key assigned and replacing the key-less file appears to have corrected the problem.
Long Story
I have been experiencing 3035 error on an Add.New to an existing backend table with >81,000 records. After searching the web for ideas and coming up dry I reflected on possible issues.
I compacted/repaired the backend files to no affect. Then decided to check the file design. It turns out there was no primary key assigned.
Assigning a primary key to the auto number ID field caused the same 3035 error! So I copied the data structure to a new file, assigned a primary key to the new file and then did a query append of the original file to the new file. Finally I renamed the files.
Using the new file appears to be working.

Related

Identifying exact location of data entry issues when bulk copying or importing from excel to access

One of the requirements of a project that I have is to allow users to import or copy and paste in bulk a few hundred rows from excel to access. However, there is a reasonable chance due to human error that there will be some data validation issues between the imported data and the table structure/referential integrity rules. I would like to be able to identify exactly the field/s and record/s where these issues are occuring so that I can point them out to the user for correction.
As such the standard error essages like 'you cannot add or change a record because a related record is required in...' or 'data type mismatch in criteria or expression' are not descriptive enough to the exact location of the problem so even if I catch them I can't really give a better descriptor anyway
I am debating importing to a completely free text temporary table, then looping an insert to move one row at a time from the temp table to the properly validated table and using dbfailonerror to catch issues on individual records that need correction (the user needs to correct them I can't do this through code)
My question is whether this is a reasonable approach, is there a better/easier way, or a way to get a more specific error from access rather than using a loop?
Thanks
There are 2 ways to do this. I'm not sure what method you are using to do the import but if is as simple as copying rows from the excel sheet to the table Access will generate a Paste_errors table that will show the rows it couldn't import. The wizard will do the same thing but I think its prone to crashing.
The way I typically do it is actually have the end user use an excel template with a VBA backend that does the uploading. You can check each value conditionally and give a better descriptive alert and/or shuttle any defective rows to a temporary table for you to review.
You can do this the opposite way and do the import through Access VBA but that would be more coding since you would have to create an Excel object in code, open the sheet, etc.
I setup a Quickbooks export of a accounts receivable table by creating a User DSN on the local machine pointing to the Access file, opening an ADO recordset and looping through the rows one column at a time applying logic to each row.
Quickbooks would ask for an existing file to dump the data into so I made that a template file on the network. It sounds like your users may have to enter directly into the spreadsheet so you would have to distribute the template but the results are the same.
Example of Looping through the sheet and validating rows
Create a DSN file to the database and store it on a shared drive, this way you don't have to hardcode the string and if you need to change something, you only have to change the dsn file instead of redistributing templates.
Public Sub Upload
'Declare the main recordset that the records will be uploaded to.
Dim rstUpload as New Adodb.Recordset
'Declare a utility recordset object that you can reuse to check referential tables
Dim rstCheck as New Adodb.recordset
'Declare a utility command object that you can reuse to check referential tables
Dim SQLCommand as New Adodb.Command
'Declare the connection object to the database
Dim dataConn as New Adodb.Connection
'A tracking flag if you find something in a row that won't upload
Dim CannotUpload as Boolean
'Open the connection to the access database
dataConn.Open "\\Server\Share\Mydatabase.dsn" 'Your dsn file'
Set SQLCommand.ActiveConnection = DataConn
rst.Open "yourTable", dataConn, adOpenDynamic, adLockBatchOptimistic
For i = 1 to 100 ' Rows
*You may want to do a pass through the rows so you can get an accurate count, usually just loop through the rows until a column that must have data is blank. If your users are skipping rows that is going to be a problem.
rstUpload.AddNew
'Set the tracking Flag to False indicating you can upload this row, this will be changed if any field cannot be validated
CannotUpload = False
'First Column/Field: 'Non critical field, any value will do
rstUpload("Field1").Value = Range(i,1).Value '
'Second Column/Field has a referential integrity constraints
'Run a query against the table that has the values you are trying to validate.
SQLCommand.CommandText = "Select IDField From YourTable where ID = " & Range(i,2).Value
Set rstCheck = SQLCommand.Execute
'Check if any records were returned, if none than the value you are checking is invalid
If rstCheck.RecordCount > 0 Then 'we matched the value in the cell
rstUpload ("Field2").Value = Range(i,2).Value
else
'Design a flagging method, maybe highlight the cell in question
CannotUpload = True
End if
....continue through your columns in the same fashion, reusing the SQLCommand Object and the Utility recordset when you need to lookup something in another table
'After last column
If CannotUpload = False Then
'Nothing triggered the flag so this row is good to go
rstUpload.Update
Else
'Cannot Upload is true and this row won't be uploaded
rstUpload.Cancel
End If
Next i
dataconn.Close
set dataConn = Nothing
set rstUpload = Nothing
set rstCheck = Nothing
set SQLCommand = Nothing

MS Access Admin State During Statement Execution

Background
I develop a number of tactical data capture tools all of which use the same approach of a split front and back end as below:
Excel VBA forms based front end (using ADO to connect to the back end)
Access 2007 (accdb) database as a back end
My preference would be to use SQL Server as a back end but this is not possible due to restrictions that I am not able to solve.
Each tool distributed has a varying number of users using the tools at the same time (anywhere between 10 - 300+). I understand that Access is not the ideal solution giving the potential number of concurrent users but once again, this is out of my control.
When the tools are in use, users sometimes receive the The database has been placed in a state by user 'Admin' on machine '***' that prevents it from being opened or locked. error.
Given the volume of transactions, the error occurs roughly 0.001% of the time.
I've read a number of articles on the topic, the majority of which end with an assumption that an object within the database is being modified or that a user is entering Design Mode which should be addressed by having a separated front and back end.
Question
A number of different types of queries occur including SELECT and INSERT INTO statements. The users do not directly access the database file and therefore no object as being modified and nothing is being put into Design Mode therefore why are users experiencing this error? Is it due to the shear number of users?
I use the same approach towards connecting to the database using the below method:
Public Function fGetOrderStatus() As Variant()
Dim oDB As ADODB.Connection
Dim oCM As ADODB.Command
Dim oRS As ADODB.Recordset
On Error GoTo Err:
Set oDB = New ADODB.Connection
oDB.Open gcConn
Set oCM = New ADODB.Command
With oCM
.ActiveConnection = oDB
.CommandText = "SELECT OrderStatusId, OrderStatus FROM ct_elh_OrderStatus WHERE Deleted Is Null"
.CommandType = adCmdText
Set oRS = .Execute
End With
If Not oRS.BOF And Not oRS.EOF Then
fGetOrderStatus = oRS.GetRows()
Else
Erase fGetOrderStatus
End If
oRS.Close
Set oRS = Nothing
oDB.Close
Set oDB = Nothing
Exit Function
Err:
MsgBox ("An unexpected error occurred. Please try again later."), vbCritical, "Error"
End Function
The following seems to be the cause of this error:
Jet locks a block of 256 bits in the MDB header. Included in this block of bits are bits that indicate a "passive shutdown", which causes the error message described in the "Symptoms" section to occur. You may have users that open and close the MDB file rapidly and the bits are not getting unlocked quickly enough. When a user tries to open the MDB file, if the program is not able to read the bits, Jet assumes that the user is in a "passive shutdown" or "admin mode", and therefore will not let the user open the MDB file.
It seems this error is due to the fact that you open and close your database to rapidly, especially if there are multiple users involved.
Why not open the connection when opening the user's front-end, and closing the connection when the user wishes to quit?
More information to be found here.

Running SAS Stored Process in Excel fails after two runs

I hope this is an appropriate place to ask this question.
I have recently built a data analysis tool in Excel that works by submitting inputs to a SAS Stored Process (as an 'input stream'), running the processes and displaying the results in Excel.
I also use some code to check for and remove all active stored processes from the workbook before running the process again.
This runs successfuly the first 2 times, but fails on the third attempt. It always fails on the third attempt and I can't figure out why.
Is there some kind of memory allocation for Excel VBA that's exhausted by this stage? Or some other buffer that's maxed out? I've stepped-in to every line of the VBA code and it appears to hang (on the third run) at the following line:
SAS.InsertStoredProcess processLoc, _
outputSheet.Range("A1"), , , inputStream
Code used to initiate SAS Add-in for Microsoft Office:
Dim SAS As SASExcelAddIn
Set SAS = Application.COMAddIns.Item("SAS.ExcelAddIn").Object
Code used to delete stored processes from target output sheet:
Dim Processes As SASStoredProcesses
Set Processes = SAS.GetStoredProcesses(outputSheet)
Dim i As Integer
For i = 1 To Processes.Count
' MsgBox Processes.Item(i).DisplayName
Processes.Item(i).Delete
Next i
Code used to insert and run stored process:
Dim inputStream As SASRanges
Set inputStream = New SASRanges
inputStream.Add "Prompts", inputSheet.Range("DrillDown_Input")
SAS.InsertStoredProcess processLoc, _
outputSheet.Range("A1"), , , inputStream
Cheers
On reflection, my theory here would be that you are hitting the limit of multibridge connections. Each multibridge connection represents a port, and the more ports you have the more parallel connections are enabled. By default there are three, perhaps you have two, or you are kicking off another STP at the same time?
This would explain the behaviour. I had a spreadsheet that called STPs and it would always fail on the fourth call because the first three were running. You can get around this by either a) increasing the number of multibridge connections or b) chaining your processes so they run sequentially.
I don't know if this'll be useful, but I had a similar problem running a DLL written in C++ through VBA. The problem occurred because the function in the DLL returned a double value, which I didn't need and so the code in VBA did
Call SomeProcessFromDLL()
But the process was returning a double floating point value which was 'filling up' some buffer memory in VBA and VBA has a limited buffer (I think it gave up at 8 tries). So the solution for me was
Dim TempToDiscard as Double
TempToDiscard = SomeProcessFromDLL()
Maybe looking at the documentation of the process being called would help here, especially if it's returning some value to be discarded anyway, like a
Return 0;
I never liked using the IOM in VBA, mainly due to issues with references and having to do client installs when rolling out applications. Last year I found a MUCH better way to connect Excel and SAS - using the Stored Process web application. Simply set up your server side SAS process with streaming output, and pass your inputs via an Excel Web query. No client installs, no worries about SAS version upgrades, hardly any code - am surprised it's not used more often!
See: http://rawsas.blogspot.co.uk/2016/11/sas-as-service-easy-way-to-get-sas-into.html

Access not exiting

Recently, my Access .mdb database started intermittently not allowing Access (both Access 2003 and 2007) to quit. If I quit (whether by pressing the X button or from the menu, then it closes the database and appears to exit Access, as well, but then it suddenly reappears (without any database open). The only way for me to exit at that point is from the task manager.
There are two significant changes that I did recently that might be related. 1) I started using the WinSCP .Net assembly to access an ftp server, which I had to install and register for COM from the instructions here. 2) I started using ODBC, first as a linked table, and then from VBA ADO code (see this). I doubt that this second change is causing this problem because I've had the problem both when I was using the linked tables and when with ADO.
This doesn't happen every time I open the database, and I haven't noticed a pattern. What could be causing this strange problem?
Edit - I found the root of the problem. By breaking my ftp download code at various points and seeing whether it will exit, I narrowed it down to the following:
Dim PDFFolders As Recordset
Set PDFFolders = CurrentDb.OpenRecordset("PDFFolders")
Dim syncOptions As TransferOptions
Set syncOptions = New TransferOptions
syncOptions.filemask = "*/*.pdf"
On Error Resume Next 'In case it doesn't exist
Do While Not PDFFolders.EOF
sess.SynchronizeDirectories SynchronizationMode_Local, info!RTFFolder, _
info!BasePDFFolder & "/" & PDFFolders!Name, False, , , _
syncOptions
PDFFolders.MoveNext
Loop
PDFFolders.Close
Set syncOptions = Nothing
Set PDFFolders = Nothing
On Error GoTo 0
If it runs the sess.SynchronizeDirectories statement, then access won't exit, otherwise it does. Looks to me like a bug win WinSCP.
I can do other things like downloading files, creating directories, etc. with no problem, but when it gets to this statement, it doesn't exit access afterwards.
Sticky instances of Access usually result from hanging object references.
If Access hangs the way you described, I would suspect a nasty circular reference.
To investigate on circular references, you basically have two options:
Inspect your code on circular dependencies - That might become tedious and not so easy. But if you know your code base deeply, you might have suspects where to look first.
Add logging to your code - In case you cannot spot the problem via inspection alone, you can consider adding consequent logging of object creation/deletion (through Class_Initialize/Class_Terminate). For larger code bases using classes heavily, this is a good investment to start with.
If your problem is with classes where you cannot control the code (as is your case), you might consider using that external classes only through wrapper classes where you can log creation/deletion. Of course in tricky cases, termination of the wrapper class does not mean termination of the wrapped class.
BTW, I strongly recommend to make sure to set every object reference explicitly to Nothing ASAP:
Set MyObj = GetMyObject()
' Proceed with coding here later
' First write the following line
Set MyObj = Nothing
Special thoughts have to be given in the case of local error handling to make sure to set the reference to Nothing in either case.
A good way to ensure this is using a With-block instead of an explicit variable (if the usage pattern allows to):
With GetMyObject()
' Use the object's members here
End With
With this pattern you save declaring the local variable and can be sure that the object reference does not survive the current method.
I still think it's a bug in WinSCP, but I found a workaround. I noticed that it only happened if I took the information from a table in the database, and not if I put in a hard-coded string. So I just added & vbNullString, which concatenates a blank string, which changes the data type from a Field to a String, and now it doesn't happen anymore.

Intermittent error when attempting to control another database

I have the following code:
Dim obj As New Access.Application
obj.OpenCurrentDatabase (CurrentProject.Path & "\Working.mdb")
obj.Run "Routine"
obj.CloseCurrentDatabase
Set obj = Nothing
The problem I'm experimenting is a pop-up that tells me Access can't set the focus on the other database. As you can see from the code, I want to run a Subroutine in another mdb. Any other way to achieve this will be appreciated.
I'm working with MS Access 2003.
This is an intermittent error. As this is production code that will be run only once a month, it's extremely difficult to reproduce, and I can't give you the exact text and number at this time. It is the second month this happened.
I suspect this may occur when someone is working with this or the other database.
The dataflow is to update all 'projects' once a month in one database and then make this information available in the other database.
Maybe, it's because of the first line in the 'Routines' code:
If vbNo = MsgBox("Do you want to update?", vbYesNo, "Update") Then
Exit Function
End If
I'll make another subroutine without the MsgBox.
I've been able to reproduce this behaviour. It happens when the focus has to shift to the called database, but the user sets the focus ([ALT]+[TAB]) on the first database. The 'solution' was to educate the user.
This is an intermittent error. As this is production code that will be run only once a month, it's extremely difficult to reproduce, and I can't give you the exact text and number at this time. It is the second month this happened.
I suspect this may occur when someone is working with this or the other database.
The dataflow is to update all 'projects' once a month in one database and then make this information available in the other database.
Maybe, it's because of the first line in the 'Routines' code:
If vbNo = MsgBox("Do you want to update?", vbYesNo, "Update") Then
Exit Function
End If
I'll make another subroutine without the MsgBox.
I've tried this in our development database and it works. This doesn't mean anything as the other code also workes fine in development.
I guess this error message is linked to the state of one of your databases. You are using here Jet connections and Access objects, and you might not be able, for multiple reasons (multi-user environment, unability to delete LDB Lock file, etc), to properly close your active database and open another one. So, according to me, the solution is to forget the Jet engine and to use another connexion to update the data in the "other" database.
When you say "The dataflow is to update all 'projects' once a month in one database and then make this information available in the other database", I assume that the role of your "Routine" is to update some data, either via SQL instructions or equivalent recordset updates.
Why don't you try to make the corresponding updates by opening a connexion to your other database and (1) send the corresponding SQL instructions or (2) opening recordset and making requested updates?
One idea would be for example:
Dim cn as ADODB.connexion,
qr as string,
rs as ADODB.recordset
'qr can be "Update Table_Blablabla Set ... Where ...
'rs can be "SELECT * From Table_Blablabla INNER JOIN Table_Blobloblo
set cn = New ADODB.connexion
cn.open
You can here send any SQL instruction (with command object and execute method)
or open and update any recordset linked to your other database, then
cn.close
This can also be done via an ODBC connexion (and DAO.recordsets), so you can choose your favorite objects.
If you would like another means of running the function, try the following:
Dim obj As New Access.Application
obj.OpenCurrentDatabase (CurrentProject.Path & "\Working.mdb")
obj.DoCmd.RunMacro "MyMacro"
obj.CloseCurrentDatabase
Set obj = Nothing
Where 'MyMacro' has an action of 'RunCode' with the Function name you would prefer to execute in Working.mdb
I've been able to reproduce the error in 'development'.
"This action cannot be completed because the other application is busy. Choose 'Switch To' to activate ...."
I really can't see the rest of the message, as it is blinking very fast. I guess this error is due to 'switching' between the two databases. I hope that, by educating the user, this will stop.
Philippe, your answer is, of course, correct. I'd have chosen that path if I hadn't developed the 'routine' beforehand.
"I've been able to reproduce this behaviour. It happens when the focus has to shift to the called database, but the user sets the focus ([ALT]+[TAB]) on the first database. The 'solution' was to educate the user." As it is impossible to prevent the user to switch application in Windows, I'd like to close the subject.