I am working on migrating a MS Access Database over to a newer SQL platform.
But, with all of the users who are currently using it, we're migrating slowly/carefully.
The first step is that we are re-writing the VBA code into C#, which is then deployed in a .dll along with the database.
Now, the VBA code calls into the C# to do the business logic, then the VBA continues to do the displays/UI, while Access still hosts the database.
The problem comes in that I have a report that is being run after the business logic from the C# in one place, and apparently MS Access has a cache, which clears every 5 seconds. So, the transaction that occurs in the C# code writes to the database, but the VBA code is still using the cache. This is causing errors, as the records added to the database (which the VBA report is trying to report on) don't exist in the cache yet...
I'm guessing that the C# .dll must be getting treated as a "second connection" to the MS Access database, which is what seems to typically cause this error in my searches (thinks that one process is writing, and the other is reading).
Since the cache is cleared out every 5 seconds, we can just put the process to sleep, and wake it up after 5 seconds, and then run the report, but that's pretty terrible for an end user.
And, making things difficult, the cache seems like it only gets used in the deployed version (so, when running from source / in debug mode, the error never happens).
Doing some searches, there seems to be plenty of people who have said "just refresh the cache." But, the question is: within VBA, how do you refresh the cache?
Any advice would be welcome.
Thanks
I've been fighting the same issue for years as I write a lot of tools around an old Powerbuilder application that has an Access MDB back end.
The cache does exist and it is VERY real. When data is inserted on a different connection than it is queried on, the cache can be directly observed and measured. It was also documented by Microsoft before they blackholed a bunch of their old articles...
Microsoft Jet has a read-cache that is updated every PageTimeout milliseconds (default is 5000ms = 5 seconds). It also has a lazy-write mechanism that operates on a separate thread to main processing and thus writes changes to disk asynchronously. These two mechanisms help boost performance, but in certain situations that require high concurrency, they may create problems.
I've found a couple workarounds that are not the best, but somewhat make due until I find something better or can re-write the app with a better back end database.
The seemingly best answer I've found (that may actually work for you since you say you need VBA) is to use JRO.RefreshCache. I've been trying to figure out how to implement this using C# or VB.net without any luck. Below is a link to a code example where you execute the RefreshCache method on your 2nd connection that needs to pull the data. I have not tested this myself.
https://documentation.help/MSJRO/jrmthrefreshcachex.htm
A workaround I've found that will deliver the query results within 500ms to 1000ms of insert time (instead of anywhere between 500 and 5000 ms - or more):
Use System.Data.ODBC instead of OleDB, with connection string: Driver={Microsoft Access Driver (*.mdb, *.accdb)};Dbq=;
If someone knows how to use the JRO.RefreshCache method with OLEDB and C# or VB.net, I'd be forever grateful. I believe the issue is it's looking for an ADO connection to be passed in, not an OLEDB connection.
I not aware of ANY suggesting that some 5 second cache exits? Where did this idea come from????
Furthermore, if you have 5 users, then you not going to be able to update their cache, are you?
In other words, the issue of some cache for one user still not going to solve or work with mutli-users anyway, is it?
The simple matter is if you load up a form with 100 reocrds, and then other users are ALSO working on that 100 rows, then all users will not see other changes until such time you tell access to re-load the form.
You can do this with a me.Refresh in the form, and then it will show changes made by other users (or even your c# code!!!).
However, that not really the soluion here.
How does near EVERY system deal with this issue?
Answer:
You don't, you "design" the software to take the user work flow into account.
So, in place of loading up a form with 100 rows of data? (which you should not, unless SUPER DUPER reason exists for doing that).
The you provide a UI in which the user FIRST searches for whatever it is they want to work on.
In other words, say you just booked a user on a tour. Now, they call the office back, and want to change some details of that tour. But, a different tour staff might pick up the phone. So, now a 2nd user opens the tour?
So, you solve that issue by NOT loading all the tours into that form in the first place.
you provide a search screen, so they can search for the user, find the user, maybe type in a invoice number or whatever.
You display the results in a pick list, and then launch the form to the ONE record (and perhaps detail records from child tables).
So there no concpet of a cache in Access anymore then there is in c#.
However, if you load up a datatable in c#, and then display that data?
Well, what about the other users on that system. They will not see changes to that data ANY MORE then the current access form.
So, if you want to update some data in c#? Then fine, but you need/want to do two things:
First, before you call any c# code that may update the current form reocrd? You need to FORCE a data save of that current record BEFORE you call any code, be it VBA code, or c# code that going to update that current reocrd the user is working on.
You can in Access save the current reocrd in MANY different ways, but the typical approach is:
' single record save - current record
if me.dirty then me.dirty = false
' VBA or c# code goes here.
' optional refresh the current form to reflect changes
me.Refresh
So, in most cases, it is the "design" of your software that will solve this issue.
For example, in the tour example, or in fact ANY system, the user can't work, can't update, and can't do their job UNLESS they first find/search and have a means to bring up that form + record data in the first place.
So, ANY typical good design will:
Ask the user for that name, invoce number or whatever.
Display the results of the search, and THEN allow the user to pick the record/data to work on. When they are done, they close that form and are RIGHT BACK to the search form to do battle with the next customer or task or phone call or whatever.
So, a search form might look like this:
In above, I typed in smi, and then displayed a pick list.
The user can further type in say part of the first name, and thus now get this:
So, maybe they type in a invoice number, customer number, booking number or whatever.
So, you display the results, and then they can select the row or "thing" to work on.
thus, we click on the row (or above glasses button), and then jump to the ONE record.
so, the user does whatever they have to do with the customer. Now, when done, they close the ONE thing, the ONE main reocrd.
This not only saves the data (so others in the office can now use that booking data), but it also means the data is saved. and they are NOW right back at the search screen, ready to do battle with the next customer.
So, not only does this mean we have a VERY bandwith friednly design (we only pull the one main reocrd into that form), but it also is better for work flow.
The Access form's cache thus becomes a non issue, since we only dealing with the one record.
And as I pointed out, if the system is multi-user, then you NOT going to be able to udpate and deal with multiple users cached data anyway, are you?
Think of ANY system you EVER used from a software point of view.
When you use google, does it download the WHOLE internet, and then you use ctrl-f to search megs and megs of data in the browser?
Nope!
you search first, get a list of that search, and THEN pick one!!
And when that list is display, maybe others on the internet are udpateing, and add new data - but if that was cached in your browser, then it would not work!!!
And same goes for a desktop accounting system. You don't load up all accounts, and THEN have the user go ctrl-f to search all the data. You search for the customer, invoice number and PICK ONE to work on.
And it does not make sense to load up a form with 1000 customers, and then go ctrl-f to find that customer. Same goes for a instant banking machine. It does not download ALL customers and THEN let you search. It asks you FIRST to get what you need. So, be it browser based, desktop based, or JUST ABOUT ANY software you use?
You quite much elminate the cache issue, since not pre-loading boatloads of data, but asking and letting the user search for the data they need.
So, in regards to the Access form data and cache?
If you are on a form, and call VBA code, or c# code or whatever?
If that code update the current form, you have NO MORE OR LESS of a issue when calling VBA code, or c# code!!!! If that code updates the current form, and the reocrd is dirty (has pending edits), then you get that message about the current form's reocrd having been udpated by another user!!!
So, your cache issue does NOT IN ANY WAY exist MORE or LESS as a issue in typical Access software.
As a genreal rule, if you are on a form with pending edits, and say want to pop up some form to edit releated data?
You have to ensure that pending edits are SAVED before you launch an form that can edit the same data, or run code that can/may edit that data.
As a result, ZERO cache issues should exist, and they no more or no less exist when calling sql or VBA update code in a form then calling some c# code from that form.
So, write the pending update for that form.
Then run your VBA, SQL, or c# code.
And then do a me.Refresh to display any changes made by those external routines.
there is no documetjion, or ANY article I can find that suggests some kind of 5 seocnd cache or update - it is a urban myth, and your software challenge here in regards to use c# or VBA, or even SQL server stored procedures?
They are all the same issue, and I dare say that often access is used as a front end to SQL server, and ALL OF the SAME issues exist when using SQL server with ms-access.
At the moment i use delphi xe3 and an online database,
I use
db.active:=False;
DB.sql.text:='slect*from database';
db.active:=True;
to show and filter records, but when i get to about 1000 records in the database, it becomes very slow, is there a way to make the program usable while the db is being loaded,
kind of like facebook loads while you type, and shows records when they're found.
thank you
Th problem is in * in your select, and ofc in amount of data. You must draw data as it goes from server. I don't know what components do you use so any information about it will be helpfull... In DevExpress's grid component there is property called "GridMode" for this.
I'm building a Sharepoint 2010 export tool for back up reasons (a bit like the filemanager from Metavis).
When downloading a file to local disk I need to back up the metadata associated with the document. Which I will store in a csv-file. My first approach was to iterate all listItem.fieldvalues, but that doesn't really work because some fieldvalues are complex types, which would needlessly complicate the backup file. Some values even have line endings, for example "MetaInfo". Furthermore not all values are needed to restore the content when that might be necessary.
So my idea is to only get the values from the Fieldvalues collection which are needed to do a functional restore, supplemented with all the user added metadata.
To do this I want to check all fieldvalues against an exclusion list to see if it is present. If it is present don't back up. If it is it is either user generated metadata or a value I need like for instance "author", "created".
So my question is, does anyone know of a list of all fieldvalues keys?
Or is there a better approach to my problem?
Thanks
Update: Well, as I was iterating through the FieldValues collection any way. It was easy to do a dump of all the values to a CSV. Running it once was enough to get all the values. Now all I need to write is an xml file for configuration. This leaves the question: is there a better way of doing this?
Filter the list fields by writing following code
using System;
using Microsoft.SharePoint.Client;
clientContext.Load(
listItems,
items => items
.Include(
item => item["Title"],
item => item["Category"],
item => item["Estimate"]));
Source: http://msdn.microsoft.com/en-us/library/ee857094.aspx#SP2010ClientOM_Creating_Windows_Console_Application
You can create an view with all fields, get the view using sharepoint object model and and get its column name from collection and filter them as per your requirement.
I have finished the application. As I wrote in my update I have made a list of all fieldValues by exporting them to a CSV file. After that I made a configuration-file with a boolean 'Backup'. This makes it possible to control which values are to be used when a backup is made.
I retrospect I think a configuration file was not needed. The values used when backing up are so much part of the whole workings of the program that a configuration file gives an administrator or casual future developer the impression that a simple reconfiguring will fulfill there needs.
I can now see that if the program needs to change due to new requirements the code has to be changed anyway. So even though setting a value to 'True' will change the output. Some other code has to be written as well. If I were to write it again I would probably use constants. This makes it all less dynamic, but still fulfill the needs of the program.
(BTW a list of all the names off the standard fieldValues would have been nice to start with. I would publish it here, but I don't have access to the file anymore, because I switched jobs recently.)
The templates for these HTML emails are all the same, but there are just different variables for say, first name, last name and such.
Would it just make sense to store the most minimal of data that I need, and load the template in and replace the variables everytime?
Another option would be to actually create the HTML file and store a reference to it, which probably would be the easiest to do except it might be a pain managing the files, and it adds complexity in regards to migrating, file permissions, et cetera.
Looking for opinions from people who've done this before...
GOAL/PURPOSE/USE:
I have a booking engine. When users make a booking, they are sent a confirmation email, generated from the sessionized booking data.
This email provides a "Cannot view this email? See it here" link which provides a web view of the email, in addition to a plaintext view.
I need to display the same email that was sent out, in addition to the plaintext view.
The template is subject to change, but I think because of that very fact I should have a table of templates and map the data to a template.
That's what I would do, because the template layout may change over the time, but the person information should remain the same. So, it makes sense to just store the person information in the database and leave the template out from the database.
In fact, it would be even better if you use template engine such as Velocity (in Java) to construct your HTML emails... very easy, by the way.
On the one hand cpu is more expensive then memory, so mostly it is better to save more data to reduce cpu power used by computation.
But in your case, I would save the minimal data, the emails or what you are tying to save, because it allows you to easily remodel your templates, and to reuse the data at multiple places of your application.
You persist redundant data (especially because of the template) which is in no way normalized. I would not suggest to do that. But mentioned in the comment it is important what you want to do with that data.
If you only save the data you need you could for example exchange that template easy and use another one.
Yea, your right on track. I did a similar thing. All dynamic/runtime variables were starting from ##symbol.
So in database you would have one Template table. One table would be for dynamic/runtime variables. One table for Mapping between Template and dynamic/runtime variables.
tblTemplate - TemplateID, TemplateValue
tblRuntimeVariables - RuntimeVariableID, VariableString, VariableSQL
tblMapping - TemplateID, RuntimeVariableID, RuntimeVariableValue
Advantage of using an extra mapping table is that on adding new dynamic variables to existing change would mean making no change to existing database. Only more rows would be added to tblMapping.
In my case I was also having one extra column for storing SQL Statements in tblRuntimeVariables in case the value for a runtime variable is fetched from database.
I would like to be able to loop through all of the defined parameters on my reports and build a display string of the parameter name and value. I'd then display the results on the report so the user knows which parameters were used for that specific execution. The only problem is that I cannot loop through the Parameters collection. There doesn't seem to be an indexer on the Parameters collection, nor does it seem to implement IEnumerable. Has anyone been able to accomplish this? I'm using SSRS 2005 and it must be implemented within the Report Code (i.e., no external assembly). Thanks!
Unfortunately, it looks like there's no simple way to do this.
See http://www.jameskovacs.com/blog/DiggingDeepIntoReportingServices.aspx for more info. If you look at the comments of that post, there are some ways to get around this, but they're not very elegant. The simplest solution will require you to have a list of the report parameters somewhere in your Report Code, which obviously violates the DRY principle, but if you want the simplest solution, you might just have to live with that.
You might want to rethink your constraint of no external assembly, as it looks to me that it would be much easier to do this with an external assembly. Or if your report isn't going to change much, you can create the list of parameter names and values manually.
If I'm understanding your question, just do what I do:
Drop a textbox on the report, then while you are setting up the report, insert the following:
="Parameter1: " + Parameters!Parameter.Label + ", Parameter2: " + Parameters!Parameter2.Label...
Granted, it's not the prettiest thing, but it does work pretty well in our app.
And I'm using Labels instead of Values since we have datetime values, and the user only cares about either the short date or the month and year (depending on circumstance), and I've already done that formatting work in setting up the parameters.
I can think of at least two ways to do this. The first might work, the second will definitely work.
Use the web service. I'm pretty sure I saw API for getting a collection of parameters. Even if there's no direct access you can always create a standard collection and copy the ReportParameter objects from one to the other in a foreach loop - and then access Count, with individual parameter properties available by dereferencing the ReportParameter instances.
Reports are RDL. RDL is XML. Create an XmlDocument and load the RDL file, then use the DOM to do, well, anything you like up to and including setting default values or even rewriting connection strings.
If your app won't have file-system access to the RDL files you can get them via the web service.