I am working on migrating a MS Access Database over to a newer SQL platform.
But, with all of the users who are currently using it, we're migrating slowly/carefully.
The first step is that we are re-writing the VBA code into C#, which is then deployed in a .dll along with the database.
Now, the VBA code calls into the C# to do the business logic, then the VBA continues to do the displays/UI, while Access still hosts the database.
The problem comes in that I have a report that is being run after the business logic from the C# in one place, and apparently MS Access has a cache, which clears every 5 seconds. So, the transaction that occurs in the C# code writes to the database, but the VBA code is still using the cache. This is causing errors, as the records added to the database (which the VBA report is trying to report on) don't exist in the cache yet...
I'm guessing that the C# .dll must be getting treated as a "second connection" to the MS Access database, which is what seems to typically cause this error in my searches (thinks that one process is writing, and the other is reading).
Since the cache is cleared out every 5 seconds, we can just put the process to sleep, and wake it up after 5 seconds, and then run the report, but that's pretty terrible for an end user.
And, making things difficult, the cache seems like it only gets used in the deployed version (so, when running from source / in debug mode, the error never happens).
Doing some searches, there seems to be plenty of people who have said "just refresh the cache." But, the question is: within VBA, how do you refresh the cache?
Any advice would be welcome.
Thanks
I've been fighting the same issue for years as I write a lot of tools around an old Powerbuilder application that has an Access MDB back end.
The cache does exist and it is VERY real. When data is inserted on a different connection than it is queried on, the cache can be directly observed and measured. It was also documented by Microsoft before they blackholed a bunch of their old articles...
Microsoft Jet has a read-cache that is updated every PageTimeout milliseconds (default is 5000ms = 5 seconds). It also has a lazy-write mechanism that operates on a separate thread to main processing and thus writes changes to disk asynchronously. These two mechanisms help boost performance, but in certain situations that require high concurrency, they may create problems.
I've found a couple workarounds that are not the best, but somewhat make due until I find something better or can re-write the app with a better back end database.
The seemingly best answer I've found (that may actually work for you since you say you need VBA) is to use JRO.RefreshCache. I've been trying to figure out how to implement this using C# or VB.net without any luck. Below is a link to a code example where you execute the RefreshCache method on your 2nd connection that needs to pull the data. I have not tested this myself.
https://documentation.help/MSJRO/jrmthrefreshcachex.htm
A workaround I've found that will deliver the query results within 500ms to 1000ms of insert time (instead of anywhere between 500 and 5000 ms - or more):
Use System.Data.ODBC instead of OleDB, with connection string: Driver={Microsoft Access Driver (*.mdb, *.accdb)};Dbq=;
If someone knows how to use the JRO.RefreshCache method with OLEDB and C# or VB.net, I'd be forever grateful. I believe the issue is it's looking for an ADO connection to be passed in, not an OLEDB connection.
I not aware of ANY suggesting that some 5 second cache exits? Where did this idea come from????
Furthermore, if you have 5 users, then you not going to be able to update their cache, are you?
In other words, the issue of some cache for one user still not going to solve or work with mutli-users anyway, is it?
The simple matter is if you load up a form with 100 reocrds, and then other users are ALSO working on that 100 rows, then all users will not see other changes until such time you tell access to re-load the form.
You can do this with a me.Refresh in the form, and then it will show changes made by other users (or even your c# code!!!).
However, that not really the soluion here.
How does near EVERY system deal with this issue?
Answer:
You don't, you "design" the software to take the user work flow into account.
So, in place of loading up a form with 100 rows of data? (which you should not, unless SUPER DUPER reason exists for doing that).
The you provide a UI in which the user FIRST searches for whatever it is they want to work on.
In other words, say you just booked a user on a tour. Now, they call the office back, and want to change some details of that tour. But, a different tour staff might pick up the phone. So, now a 2nd user opens the tour?
So, you solve that issue by NOT loading all the tours into that form in the first place.
you provide a search screen, so they can search for the user, find the user, maybe type in a invoice number or whatever.
You display the results in a pick list, and then launch the form to the ONE record (and perhaps detail records from child tables).
So there no concpet of a cache in Access anymore then there is in c#.
However, if you load up a datatable in c#, and then display that data?
Well, what about the other users on that system. They will not see changes to that data ANY MORE then the current access form.
So, if you want to update some data in c#? Then fine, but you need/want to do two things:
First, before you call any c# code that may update the current form reocrd? You need to FORCE a data save of that current record BEFORE you call any code, be it VBA code, or c# code that going to update that current reocrd the user is working on.
You can in Access save the current reocrd in MANY different ways, but the typical approach is:
' single record save - current record
if me.dirty then me.dirty = false
' VBA or c# code goes here.
' optional refresh the current form to reflect changes
me.Refresh
So, in most cases, it is the "design" of your software that will solve this issue.
For example, in the tour example, or in fact ANY system, the user can't work, can't update, and can't do their job UNLESS they first find/search and have a means to bring up that form + record data in the first place.
So, ANY typical good design will:
Ask the user for that name, invoce number or whatever.
Display the results of the search, and THEN allow the user to pick the record/data to work on. When they are done, they close that form and are RIGHT BACK to the search form to do battle with the next customer or task or phone call or whatever.
So, a search form might look like this:
In above, I typed in smi, and then displayed a pick list.
The user can further type in say part of the first name, and thus now get this:
So, maybe they type in a invoice number, customer number, booking number or whatever.
So, you display the results, and then they can select the row or "thing" to work on.
thus, we click on the row (or above glasses button), and then jump to the ONE record.
so, the user does whatever they have to do with the customer. Now, when done, they close the ONE thing, the ONE main reocrd.
This not only saves the data (so others in the office can now use that booking data), but it also means the data is saved. and they are NOW right back at the search screen, ready to do battle with the next customer.
So, not only does this mean we have a VERY bandwith friednly design (we only pull the one main reocrd into that form), but it also is better for work flow.
The Access form's cache thus becomes a non issue, since we only dealing with the one record.
And as I pointed out, if the system is multi-user, then you NOT going to be able to udpate and deal with multiple users cached data anyway, are you?
Think of ANY system you EVER used from a software point of view.
When you use google, does it download the WHOLE internet, and then you use ctrl-f to search megs and megs of data in the browser?
Nope!
you search first, get a list of that search, and THEN pick one!!
And when that list is display, maybe others on the internet are udpateing, and add new data - but if that was cached in your browser, then it would not work!!!
And same goes for a desktop accounting system. You don't load up all accounts, and THEN have the user go ctrl-f to search all the data. You search for the customer, invoice number and PICK ONE to work on.
And it does not make sense to load up a form with 1000 customers, and then go ctrl-f to find that customer. Same goes for a instant banking machine. It does not download ALL customers and THEN let you search. It asks you FIRST to get what you need. So, be it browser based, desktop based, or JUST ABOUT ANY software you use?
You quite much elminate the cache issue, since not pre-loading boatloads of data, but asking and letting the user search for the data they need.
So, in regards to the Access form data and cache?
If you are on a form, and call VBA code, or c# code or whatever?
If that code update the current form, you have NO MORE OR LESS of a issue when calling VBA code, or c# code!!!! If that code updates the current form, and the reocrd is dirty (has pending edits), then you get that message about the current form's reocrd having been udpated by another user!!!
So, your cache issue does NOT IN ANY WAY exist MORE or LESS as a issue in typical Access software.
As a genreal rule, if you are on a form with pending edits, and say want to pop up some form to edit releated data?
You have to ensure that pending edits are SAVED before you launch an form that can edit the same data, or run code that can/may edit that data.
As a result, ZERO cache issues should exist, and they no more or no less exist when calling sql or VBA update code in a form then calling some c# code from that form.
So, write the pending update for that form.
Then run your VBA, SQL, or c# code.
And then do a me.Refresh to display any changes made by those external routines.
there is no documetjion, or ANY article I can find that suggests some kind of 5 seocnd cache or update - it is a urban myth, and your software challenge here in regards to use c# or VBA, or even SQL server stored procedures?
They are all the same issue, and I dare say that often access is used as a front end to SQL server, and ALL OF the SAME issues exist when using SQL server with ms-access.
I have a front-end form that fetches data from a back-end table. The front-end has multiple users and they use the back-end table simultaneously. The user uses a form to generate reports. Let's say if a user click the "view" button, the background process will insert data to a table and it will show the result.
However, my problem is if 2 different users from different locations click the "view" button from their end machine at the same time, the process on the back-end table runs twice and it duplicates the data.
I know we can delete duplicate data from the table itself but what I'm trying to do is if there's a way to stop one process if there's already a same process running.
I have found a scenario similar to my problem but it is only caused by clicking the button twice.
Here's the link for the similar problem.
I hope someone could help me out. It's my firs time to use VBA and I have no previous knowledge/experience with it.
I was not able to prevent a process for running twice but we were able to find a workaround for my concern by assigning a compound index to the table. It no longer accepts duplicates regardless of the process running multiple times simultaneously.
I have list of records in TIBDataSet (Embarcadero Delphi) and I need to locate and modify one record in this list. There is chance that underlying database record has been changed by other queries and operations since TIBDataSet had been opened. Therefor I would like to call RefreshSQL for this one record (to get the latest data) before making any changes and before making post. Is it possible to do so and how?
I am not concerned about state of other records and I am sure that the record under consideration will always be updated and those updates will be commited before I need to changes this record from TIBDataSet.
As far as I understand then RefreshSQL is used for automatic retrieve of changes after TIBDataSet has posted upates to database. But I need manual (explicit) retrieval of the latest state before doing updates.
Try adding a TButton to your form and add the following code to its OnClick handler:
procedure TForm1.btnRefreshClick(Sender: TObject);
begin
IBQuery1.Refresh; // or whatever your IBX dataset is called
end;
and set a breakpoint on it.
Then run your app and another one (e.g. 2nd instance of it) and change a row in the second app, and commit it back to the db.
Navigate to the changed row in your app and click btnRefresh and use the debugger to trace execution.
You'll find that TDataSet.Refresh calls its InternalRefresh which in turn calls TIBCustomDataSet.InternalRefresh. That calls inherited InternalRefresh, which does nothing, followed by TIBCustomDataSet.InternalRefreshRow. If you trace into that, you'll find that it contructs a temporary IB query to retrieve the current row from the server, which should give you what you want before making changes yourself.
So that should do what you want. The problem is, it can be thoroughly confusing trying to monitor the data in two applications because they may be in different transaction states. So you are rather dependent on other users' apps "playing the transactional game" with you, so everyone sees a consistent view of the data.
I'm doing a db in access 2002 and I've had some problems with the locking shared mode. I have an app that has a lot of programs and almost 10 users logged on. And one of things that the app do is open a report, that user choose the program and the number (usually has 4 numbers for each program), and before the report open, I open it in hidden mode and I edit the report for current program/number, with this rotine I just have one report that may turn in hundreds... saving memory and optimizing it. Then I save it and open it again in view mode.
But the problem is when has other person using the app, so the access can't edit and save report... just in exclusive mode!
Has a cmd in vba to allow momentarily changes in shared mode? I don't know, like freeze all user, save and then unfreeze them?
Or any other suggestion?
About create a Front-End/Back-End I think that is impracticable, cause is a beta version and I have to update it often and I already tried to do this also, but it's became too slow... I splitted then in a database(just tables) into network and front-end with all querys, forms, reports and linked tables in local PC, but it really became tooo slow. If someone can help me let it faster would solve my problems too
I splitted the db and I'm trying optimize it. I read a lot about it on the web and I changed all Access setting that I saw that need to be changed and now I get a faster program. But slower than with a single app.
But now there are just fell Forms that making my app slow.
For example I have some Forms that always when I close it I spends a long time waiting it to close ;(
So I realised that this Forms are always saving before close. And always that a form need to save(with linked tables taht are in the network) waste a lot of time, so I need to avoid this.
But I didn't get it so far...
I realise that this forms are saving because in form_open I hidde some columns(that are different for each program) and edit it caption. And then if I need to close form, it saves and waste this such time!
How could I hidde/edit this columns withou need to save form? Or how could I close form without save structure changes?
I know how to do that with just a button, but these forms are datasheets and I can close it only in "X" Form button. And unfortunattely Access dont have BeforeClose event, and in a OnClose event it save before go to this sub!
See if you can make use of a WhereCondition with the DoCmd.OpenReport Method to avoid the need to modify your report's design at run time.
The WhereCondition is applied to your report's existing record source query as if it where written into that query's WHERE clause.
So if the record source for YourReport is ...
SELECT program_id, some_number, another_field
FROM YourTable;
... then this ...
DoCmd.OpenReport "YourReport", _
WhereCondition:="program_id = 7 AND some_number = 22"
would give you the same set of rows as would revising the record source to this ...
SELECT program_id, some_number, another_field
FROM YourTable
WHERE
program_id = 7
AND some_number = 22;
This advantage of this approach, if you can make it work for your situation, is that you would no longer need exclusive access to the db since you're not actually changing the report's design.
The related issue about beta status making it impractical to split the application is something you should re-consider carefully. Splitting ensures you can easily preserve the data in the BE when you roll out changes to the FE application. Even if you've come up with another method to avoid losing data when you change versions, that method can not be simpler than segregating the data into a BE file.
And when you split the application, each user should get their own copy of the FE file which is stored locally on their machine's hard drive; those FE files will contain links to the tables in the BE file which is stored on a file share.
Keeping the users' FE applications updated as you release new versions is a problem which has been solved. For example, see Tony Toews' Auto FE Updater. And you can find other approaches by searching the web.
If your concern is performance with a split application, check Tony's Microsoft Access Performance FAQ.
Sorry I don't know if this is something simple, or even where the problem fits in the greater scheme of programming.
So in my unsophisticated ways, my programs have always been of the scheme: 1. start program, 2. wait while program runs, 3. program is done and gone.
What I am doing now is creating a table from a long list of transactions (10,000s of). The table has several combo boxes for the user to select filters. Right now, every time the user changes a filter, the entire log is re-processed, which takes half a minute or a minute.
What I would rather do is have the trade log held in memory, or somehow latently but more immediately available. But not have the program "spinning" in the background. So the user could go about using Excel unaware that the program is ready in the background in case they want to update the table later, or not.
Does that make sense? If it can't be done in VBA, I'd still be curious how it would be done in another environment, say C#, if it could be. Thanks.
If the frequency of updates to the options trade is low enough you could separate reading and processing the option trades from the filtering process:
Step 1 - Refresh - read the logs and process them, storing the results in global containers (arrays, collections, dictionaries, objects ...)
Step 2 - User requests - show form - user chooses filters - show/store results extracted from the global containers.
There are several options
Firstly, is the code correctly structured? For example, do you really need to re-process everything or can a re-write be more efficient?
If you cannot avoid resource intensive code, notify the user with a progress bar or message. Also consider the use of DoEvents which frees up the operating system so that Excel can process other events.
DoEvents is slow and dirty. Even better look at this link DoEvents is slow!!! Here are faster methods
Rewrite your code to work asynchronously. Create a class, a handler and deal with each transaction asynchronously.
You could write some VBScript/Javascript and push the task out to run independently of Excel/VBA. Eg there's an example Here
Don't use VBA :)
Edit: How are you filtering? If you're iterating through thousands of items in an array testing for criteria it can be very slow. Excel's Advanced Filter is very quick and could process hundreds of thousands of rows with multiple criteria quickly.
When a macro in Excel VBA runs, the user cannot use Excel anymore, running the VBA "stucks" the whole program.
Here are a few tips to find a workaround for your problem :
Keep the vba running : load the data a first time when launching the combobox and then display results to the user every time he asks for but keep a combobox loaded so that vba keeps its context and memory
Load the data in Excel Worksheet, even hidden and then use it when the user asks for some data
Give us more info on what you are doing, from where you are loading the data, how you can cache it, what is your current code, what you tried... so that we can help you more
Regards,
Max