After a whole day tracking down a memory leak in my VB.NET project, I have traced the cause to a bug with FileMaker's ODBC driver !
To reproduce you'll need a database you can connect to (I have mine hosted on Server Advanced 11.0.3, but you can also host it locally), and the ODBC driver registered/installed on the PC (I tested versions 11.3 and 12.0, and the latest 12.2).
Start a new VB.NET WinForms project, add a button to the form and paste this code onto the button's click event:
Using cn_FM As New Odbc.OdbcConnection("DRIVER={FileMaker ODBC};SERVER=192.168.1.xxx;UID=admin;PWD=admin;DATABASE=test;")
cn_FM.Open()
End Using
All this code does is open a connection to a FileMaker database, however if you analyse the memory usage in Windows Task Manager you can easily see (by repeatedly clicking the button you just made) that cn_FM is not being disposed properly because the Handles keep increasing! I tried forcing Garbage Collection but this didn't do anything, so I assume its a problem with the driver itself.
Oh, and I tested connecting to a SQL database in the same way, and as you would expect, there was no handle leakage...
Can anyone confirm this is correct?
Edit: I tried various ways of opening and closing the connection, as well as actually querying the database for something in the using block. Also tried hosting the fp7 file locally, but still no go :(
FileMaker's ODBC drivers are horrible and they admit it. You'll also find that your CPU spikes to nearly 100% for every query you hit the FM server with. I've been griping at them about it for years.
Their "solution" was to introduce External SQL Sources, but that requires you to go the other direction. You can bind your VB database to FileMaker and then access the data just like actual FileMaker data. This will allow you to create scripts on the FM server to sync whatever tables you need to sync with your VB database.
It's not ideal, but that's going to be your best bet to get something together with good performance.
I got around this problem by making a persistent connection (declare and open it once and leave it open). But I need to check if its still open each time I want to use it, for example:
Public Sub CheckOpen(ByRef cn As Odbc.OdbcConnection)
If cn.State <> System.Data.ConnectionState.Open Then
cn.Close()
cn.Open()
End If
End Sub
If you have multiple FM database files then this may mean you need to have one connection for each file.
Side Note: FileMaker's xdbc_listener.exe process running on FMSA is also leaky. We have noticed a pattern that once it reaches just under 2GB memory usage it crashes. So keep in mind that the process may need constant restarting.
Related
I have an MS Access (2013) application with a split database. Everything seems to run smoothly except for occasionally I will get Error 3048: Cannot open any more databases.
The error occurs when the front end tries to run vba code which involves pulling data from the back end and will stall on any line with: Set DB = OpenDatabase() or DoCmd.RunSQL() commands.
The strange thing is that this error seems to be time based. I can access the back end hundreds of times without error if I do it quickly enough but after some time has passed (~1 hr) the error shows up. In fact, I can open the application and leave it running in the background (with no code running) then go back into it after an hour and I will get the error the first time the code tries to open the back end.
I've searched the length and breath of this site and google for solutions so I know this error has been addressed before. To save people reiterating the usual fixes I will list what I've tested for so far with no success:
Recordset limit: I'm not leaving any recordsets open, every time I open one I make sure to close it. The same for the databases. All my requests
to the back end are done via 3 or 4 vba functions and each of these has
a Rec.Close or DB.Close corresponding to every OpenRecordset()
and OpenDatabase() and I never have more than 2 recordsets open at
a time.
Control limit: I have 151 controls on the biggest form in the application so I should be below the limit (I believe this is 245 for a single form?)
Corrupt database: I've copied all my forms and code to a new Access database and run a Compact and Repair.
Machine Issue: I've tested the application on several machines and reproduced the same error.
Anyway with most of the above situations I would expect the application not to run at all, instead of running fine for a set amount of time and then crashing.
Some other points of note:
Citrix Users: The users are split between normal windows machines users who are experiencing this error and others who are using the application through a virtual desktop software (Citrix) who are having no issues. Unfortunately I don't know enough about this virtual desktop to really work out what that implies.
Background vs Foreground: Some users have claimed that the application only crashes if it has been running for a long time AND they switch over to another program and switch back. I've confirmed that simply switching between the application and other programs doesn't cause it to crash but haven't yet been able to leave it running in the foreground long enough to confirm if it crashes without switching between programs.
I've been struggling with this for days, anyone able to help me out?
I would like to switch between running on a database on the server (the real database) and a local copy that I keep for testing in localhost.
I tried to switch between db#1 (server) and db#2 (localhost) by changing the definition of the connection string in app.config (I also restarted VS to make sure nothing hides in memory). A strange thing happens: on the one hand if I query the db through datasource/preview data (in VS'10) indeed it brings the data from db#2. But when I run the application is still grabs the information from db#1, although as far as I know the connection string appears only in app.config, so I do not even understand where does it take the information from in order to connect to db#1.
While we're on the subject, what would be the more elegant way to switch between databases ? ideally from a pull-down menu I would chose the data source. But can I switch the connection string in run time ?
Visual studio maintains its own database connection info, presumably from its own app.config file. This is completely independent of your solutions app.config.
Switching databases as you suggest is relatively easy. Put 2 connection strings with different names in your app.config. Use a variable to hold the string you want and reference this wherever you are directly referencing the connection string now.
Just a little background: I am using Access 2010 to create forms and VBA code in an Access 2003 format database. For some reason, Access 2007 format databases always corrupt on me when I make changes and save them with a particular group of objects, but that's for another discussion.
When writing VBA code in this Access 2003 database, any time my code breaks (via breakpoint or an unhandled error) and I make a correction, Access tells me that it can't save back to the database because another user has it open. However, I am the only user working on the database; this is a local copy of the database and it's sitting on my desktop.
The LDB file can't be deleted because Access is using it. When I first load the database, I see my machine name and "Admin" when opening the LDB in a text or hex editor. After a break, I see that plus a duplicate entry, but this time around "admin" has a lower-case "A."
Closing the database and reopening it fixes the problem but makes it needlessly cumbersome to debug my code. Anyone else encounter this issue and/or have a fix for it?
It might be helpful to know what your code is doing when this happens. Certainly that's not normal behavior. For instance, are you opening another database with New Access.Application? Are you using ADO or DAO to access records in the database with a connection string?
There are no external connections to the database at all.
It may not matter if there are external connections to the database if you are using a connection string to connect to the open database; not sure but that may be seen as an external connection... you may want to use CurrentDB for DAO, or CurrentProject.Connection as your ActiveConnection for any ADO queries.
I am assuming that this problem persists through reboots; but for the sake of argument, try closing out Access and going to the task manager to make sure you have no other instances of MSAccess.exe running. You might even try closing all Office products and/or making sure that Access is the only Office product running. I have seen some weird conflicts between Microsoft Communicator and Outlook; so it's not entirely out of the question for Access to have issues with another MS product.
You may also want to check the size of the database to make sure it's not exceeded 2GB. That causes the infamous "Invalid parameter" error; perhaps it might be causing this as well.
With no other details about how your program works, we may only be able to offer generic advice like this.
I have discovered a way to cause the problem discussed above (and thereby to correct it). Turns out if you create a database object and set it to the current database, you get this problem.
That is,
dim cdb as database
set cdb = currentdb
From this point on, you're cooked.
Instead, figure a way around this by possibly using currentdb directly or not using it at all.
This worked for me.
In your VBA Try checking that all your open Connections to the database are closed. Until the connection is open the LDB fill will be there.
Same symptom of not being able to save form or code mods after application had started. I found a workaround today! In the startup of my first form of the app, I had issued a "DAO.DBEngine.SetOption dbMaxLocksPerFile, 20000". Commenting this statement removed the problem. I did no further testing, but FYI, the DBEngine call was before any reference or attempt to use CurrentDB(). Also the current default on my Access 16 install is 9,500.
I thought I might answer here, since I stumbled upon this question while having a similar issue. Essentially, it boiled down to this: I could either edit forms, VBA, etc. or edit information in the local database (which I'm using as a cache) with currentDB. I also have a backend database, but the locking was clearly on the frontend database.
The solution ended up being weird, but stupidly simple. When the frontend starts up, I have it immediately create a connection to the backend using OpenRecordset (and similarly to you, that backend was still on my own computer for testing purposes). I tried temporarily disabling that code, and suddenly it wasn't an issue anymore. And it turns out, once I call currentDB, I can then call OpenRecordset to open the connection to the backend, and suddenly it isn't a problem anymore.
Tl;Dr: if you're calling OpenRecordset somewhere in your code to connect to a backend, be sure to call something like set db = currentDB beforehand, then everything works. (That is, probably until I publish this answer and Access then decides it doesn't want to anymore).
Why this fixed it is beyond me, someone with more knowledge can maybe answer that.
The solution:
options > current database > click enable -track name auto correct info
My Application (Vb.net, Access 2003/2007) is to scan Access Database files for activex controls and to generate report accordingly.
Problem:
Getting an error like:
"Insufficient memory to continue the execution of the program."
The above error occurs while scanning for older version of Access files like prior to office 2000.
And the line of code where I get this is as follows:
Dim oForm As Access.Form
Dim oAccess as Access.Application
oForm = oAccess.Forms(objForms.Name)
But it opens the file and form as well.
Need Help:
Whether it is possible to read the file (Access Forms and Reports) or not?
Please provide me reference or any solution.
You appear to doing COM automation of Access to open the forms and then cycle through their controls looking for certain properties.
Another solution would also involve automating Access, but it wouldn't require actually opening the form, and that's the undocumented Application.SaveAsText command. You'd do something like this:
Application.Saveastext acForm, "dlgWebBrowser", _
"C:\Output\dlgWebBrowser.txt"
You would then have to figure out how ActiveX controls are described in that file. If that file looks like the code for a VB form, that's because that's precisely what it is.
The example above had an IE web browser control on it, and after a dump of OLE data, it had this in it:
OLEClass ="Microsoft Web Browser"
Class ="Shell.Explorer.2"
GUID = Begin
0x54c1ea41936d2046b9dc5b29905976e3
End
I would expect that all ActiveX controls will have an OLEClass, but I non-native avoid ActiveX controls on principle because of the problems they can cause if not properly installed when you try to run the app.
In fact, that could be the source of the problem -- if you open the Access form on a machine that doesn't have the relevant ActiveX control registered, it's going to fail, and the form won't open.
My bet is that Application.SaveAsText is going to sidestep that problem entirely, since the form doesn't have to be opened.
I've seen behaviour very similar to this before. Access 97 files will sometimes report an 'out of memory' error if you try to open them on a computer with more than (I think) 1Gb of RAM. The error doesn't always manifest itself immediately - sometimes the project can appear to run normally but crash when you try to open a particularly large object.
In the case where we did run into this the users were running an old Access 97 database on new XP machines they'd been upgraded to with modern amounts of RAM. Tech support for the company tried everything they could think of - including complete office reinstalls, applying all patches etc. but eventually had to resort to removing RAM from the computers - whereupon the errors went away and everything was rock solid again. I am uncertain as to the exact cause, but it will be connected with memory management in the Access 97 file format (I believe the error is on MSDN somewhere but I wasn't directly involved with Tech support hunting the solution down - I'd just written the application many years before)
I'd suggest you're only way out is to use a special, low memory, PC to run the application.
Background:
I'm working with a program coded in C++ which uses ODBC on SQL Native Client to establish connections to interact with a SQL Server 2000 database.
Problem:
My connections are abstracted into an object which opens a connection when the object is instantiated and closes the connection when the object is destroyed. I can see that the objects are being destroyed: their destructor are firing and inside of these destructors, SQLDisconnect( ConnHandle ) is being called, followed by SQLFreeHandle( SQL_HANDLE_DBC, ConnHandle ); However, watching the connection count using sp_Who2 or the Performance Monitor in SQL shows the connection count increasing without relent, despite these connections being destroyed.
This hasn't proven problematic until executing a chain of functions that runs long enough to create several thousand of these objects and as such, several thousands of connections.
Question:
Has anyone seen anything like this before? What might be causing this? My initial google searches haven't proven very fruitful!
EDIT:
I have verified that SQLDisconnect is returning without error.
Connection pooling is off. In fact, when I attempt to enabling it using SQLSetEnvAttr, my application crashes when the 2nd call to SQLDriverConnect is made.
Check that you are not using connection pooling. If it is turned on, it will cache opened connections for some (configurable) time.
If you are not using connection pooling, then you must check return value of the SQLDisconnect(). You may have some transaction executing or rollbacking that wont let SQL Disconnect() release your connection.
You have more details on how to check for SQLDisconnect errors at MSDN.
I believe I have seen the same issue in an application that uses MFC and ODBC, rather than the SQL native client API directly. Occaisonally my application hangs on shutdown, the stack trace is:
sqlncli!CCriticalSectionNT::Enter
sqlncli!SQLFreeStmt
sqlncli!SQLFreeConnect
sqlncli!SQLFreeHandle
odbc32!UnloadDriver
odbc32!FreeDbc
odbc32!DestroyIDbc
odbc32!FreeIdbc
odbc32!SQLFreeConnect
mfc42!CDatabase::Close
mfc42!CDatabase::Free
mfc42!CDatabase::~CDatabase
Try as I might, I cannot see anything that might cause such a hang. I'd be grateful if anyone can suggest a solution. It seems others have seen similar issues online, but to date I haven't found any solution.
sqlncli!CCriticalSectionNT::Enter
sqlncli!SQLFreeStmt
sqlncli!SQLFreeConnect
sqlncli!SQLFreeHandle
odbc32!UnloadDriver
odbc32!FreeDbc
odbc32!DestroyIDbc
odbc32!FreeIdbc
odbc32!SQLFreeConnect
mfc42!CDatabase::Close
mfc42!CDatabase::Free
mfc42!CDatabase::~CDatabase
From your stacktrace not having a bottom, can we assume that the CDatabase is a global variable?
Possibly in a dll?
We found your exact symptoms if attempting disconnect from SQL Server from within the destructor of a global variable.
Using the MDAC ODBC drivers works successully.
Moving the code out of the destructor works sucessfully.
It seems something to do with sql native client not liking being called from inside a DllMain.