Import partial table from remote DB into a local microsoft access DB - sql

Problem:
I'm trying to import a portion of a table from a remote informix db into an access 2016 database. The table to be partially imported contains account information and is > 2 GB (which as I understand it is the max size of a access db).
I only need information for the last year which is much less than 2 GB but the information must be updated at the end of each day.
Two solutions I have seen online are:
(A) Link the tables:
I'm reluctant to do this for two reasons:
1) I don't want additional load placed on the server everytime the enduser of the access db runs a report.
2) I don't want any changes carried out in the access DB to carry over to the live informix DB (this is absolutely cruical)
(B) Copy over the entire table via the access GUI
I don't think this is a viable solution for the following reasons:
1) The table is over 2GB
2) Reports must be ran on the table everyday with daily updated information. This would require pulling the entire informix table into access everyday.
Work So far:
I have succefully connected to the informix db from access via creating a dsn for the informix db.
I have also sucessfully imported a smaller table from the informix server via VBA with the following code:
DoCmd.TransferDatabase acImport, "ODBC Database", "ODBC;DSN=My_DSN_Name;UID=odbc;PWD=My_PWD;LANGUAGE=us_english;" & "DATABASE = My_DB_Name", acTable, "My_Destination_Tbl_Name", "TestTableImport", False, True
My ideal solution would see the enduser of the access db press a button that updates the local access copy of the db with all changes made to the informix db since the last update.
I have limited experience with access and VBA so would really appreciate any pointers on how I should proceed. Perhaps I have overlooked something simple?
Thanks for reading and any advice you can provide,
John

Ummmm.... assuming the data in the table is static (no updates to column data, just new rows) you could do something like importing whatever subset of data you want in Access and then every time you click a button run a macro to check what is the latest row in the access table.
Use that info to select only new rows from the ODBC source and insert them in the access table.
I tried this with Access 2013:
Sub test()
Dim cnDB As New ADODB.Connection
Dim rsRecords As New ADODB.Recordset
' find last row
Set db = CurrentDb
strSQL = "SELECT TOP 1 empno As lastemp FROM informix_employee ORDER BY empno DESC"
Set rs = db.OpenRecordset(strSQL, dbOpenSnapshot)
lastemp = rs("lastemp")
rs.Close
' get new rows from odbc source
DoCmd.SetWarnings False
cnDB.Open "DSN=ids1210;uid=informix;pwd=password"
rsRecords.Open "Select * from employee where empno>" & lastemp, cnDB
Do While Not rsRecords.EOF
strNaam = rsRecords.Fields(0).Value
rsRecords.MoveNext
' update access table
DoCmd.RunSQL ("insert into informix_employee (empno) values ('" + strNaam + "')")
Loop
rsRecords.Close
Set rsRecords = Nothing
cnDB.Close
Set cnDB = Nothing
db.Close
End Sub
It's a basic example, but should give you an idea of how to do it.
The code above inserts only one column per row (empno), you will need go through your table and do the insert with all the columns (basically create the insert ............. statement for your table)
'informix_employee' is the Access table.
'employee' is the Informix table.
Also, you need to reference ADO, e.g. 'Microsoft ActiveX Data Objects 6.1' to get it working, otherwise it will fail as the ADODB object wont be there.
If the Informix table does get some column updates (not just new rows) it would be quite difficult, as nothing (IDS/ODBC/Access) will be able to tell you which rows were updated without getting the data and comparing it with the previous set.

Related

Create and run append queries for multiple linked tables

I am trying to write a VBA code in Microsoft Access that will create and run 60 queries to select new records from 60 linked tables and insert them into 60 tables of the same format. Some background may help here:
I have a large database (lets call the original database "A") that will eventually have over 60 tables, 60 forms & 60 reports. When one of our workers goes out into the field and doesn't have an internet connection, they are going to create new records on a copy of the database stored on their desktop (lets call the duplicate database "B"). Once they have an internet connection, I want them to be able to press a button on either database (I have been trying to code the macro on database A because I thought that would be easiest) and have the new records they created on database B inserted into database A.
I have found code online that seems to be just what I need but when I try to run the macro it gives Error 3022, which says:
The changes you requested to the table were not successful because they would create duplicate values in the index, primary key, or relationship
I have tried running this macro with both databases on my desktop with only 1 linked table with a primary key that is an autonumber, I tried running it with a random autonumber, I tried not having any primary key or index or autonumber and even no records at all. I even tried running it without any linked tables. All ways of trying give me the same Error 3022. I really don't want to create 60 queries one by one so any help would be greatly appreciated. Thank you wizards in advance :)
Here is the code I have tried:
Public Sub ImportTableData(ByVal pstrTable As String, ByVal pstrDb As String)
Dim strSql As String
strSql = "INSERT INTO " & pstrTable & vbNewLine & _
"SELECT *" & vbNewLine & _
"FROM " & pstrTable & " IN '" & pstrDb & "';"
CurrentDb.Execute strSql, dbFailOnError
End Sub
Public Sub ImportAllTables()
Const cstrDb As String = "C:\MyPath\DatabaseB.accdb"
Dim tdf As TableDef
Dim strMsg As String
On Error GoTo ErrorHandler
For Each tdf In CurrentDb.TableDefs
'ignore system and temporary tables '
If Not (tdf.Name Like "MSys*" Or tdf.Name Like "~*") Then
Call ImportTableData(tdf.Name, cstrDb)
End If
Next tdf
ExitHere:
On Error GoTo 0
Set tdf = Nothing
Exit Sub
ErrorHandler:
Select Case Err.Number
Case 3078
strMsg = "Input table " & tdf.Name & " not found."
MsgBox strMsg
Resume Next
Case Else
strMsg = "Error " & Err.Number & " (" & Err.Description & ") in procedure ImportAllTables"
MsgBox strMsg
GoTo ExitHere
End Select
End Sub
When I remove the piece of code: 'dbFailOnError', I get different errors. First thing that pops up when I run the macro is: ''Input table 'LocalTableName' not found'', which is the table I am trying to add the records to. Once I click 'Ok' on that pop up box, that is when Error 3134 pops up which says 'Syntax error in INSERT INTO statement'. I am assuming Error 3134 only pops up because it cannot find the local input table (the fist pop up box).
Also, I tried changing the line of code that says: 'Const cstrDb As String = ''C:\MyPath\DatabaseB.accdb''' to instead point to database A (which is the one I am coding the macro on) like this: 'Const cstrDb As String = ''C:\MyPath\DatabaseA.accdb'''. This doesn't give me the first pop up that says ''Input table 'LocalTableName' not found'' but it still gives Error 3134.
I have no idea what I am doing wrong and have spent over 20 hours on this problem trying dozens of different things. Any help would be greatly appreciated :)
Well, the main issue is how you going to ensure that the PK (primary keys) and FK (foreign keys) remain the same when they go out to the field and start entering data?
There is a good chance that PK/FK values will now be duplicated, or be the same for on one of the field users.
If a user out in the field adds a record, and someone at main work location adds record, they now are to very likly have the same PK value.
I suppose this might work if you use random autonumbers (never even knew that was a option after all these years!!!) - but I can't say even that going to be 100% reliable.
And when you import that copy from the out in the field user, then either:
You always ensure you accept and take the same PK value, or you let access generate a new PK - but if you do that, then the child records FK value would have to be updated then.
You not really try to do a simple import, but are attempting to do a database synchronization- a stunning and VERY advanced concept. And a very challenging problem.
Access (mdb format) did at one time support what is called database replication. This feature would be ideal for your setup.
However, but for newer accDB formats, it not supported anymore. (and quite sure by around access 2010, replication support was dropped anyway).
So, you could try random for the autonumber. I mean, you simple cannot have the PK's being duplicated on each computer - plain and simple.
The other possbile?
You add to each and every table that has a PK, and add a new column called PKF (f for in the field).
And for every table that has a FK, you add a new column called FKF (again, add F to this).
So, in the field, your PK/FK used is NOT the same as the main master database at work.
So, I wrote a Android sync routine based on above. The applcation was Access, but I moved the data to sql server (but same idea). I moved to sql server since my android phone could use its local database (sqlLite) and hit sql server. (but, it would be difficult to get android to hit some server and read + use a accDB file - but was easy to have Android phone hit the sql server database directly).
Gee, maybe they could use Android phones!!! but, this would assume you up to speed writing android software, 100% conformable with SQL server, and also access. I was lucky, had all 3 skill sets, so that is the road and hammer I choose.
And speaking of above?
Maybe your lucky, and you have sql server running at work (not express edition, but full edition). I suggest this considering, since the free edition of SQL express can be what we call a replication subscriber to a main sql database. This allows you to sync your local database with the main mothership database.
So, adopting free SQL server express on each field laptop could be a possible solution. Then when they get to a working network, they sync the database using replication.
But, you could try and roll your own sync system.
I did that for an android applcation I wrote, and for a desktop Access application I had. (but, to make all the moving parts easy, I did adopt sql server for database - continued to use Access as the application/UI part).
Now, using "random" for the PK looks to be a possible solution. I just don't know how random, and if this choice can reliable avoid PK collisions for autonumbers.
Random seems like the best road - but ONLY if that choice would prevent duplicate PK id's being used out in the field for new records.
Edit: Random - not even close - it not random enough
So, a bit of research - no, random PK will not work, you still often wind up with collisions - so that idea is off the table.
I figured it out finally. Basically I changed the line of code that executes the SQL to debug.print. This showed me what was going on. The problem is I had linked the tables from the other database when they didn't need to be linked. The names of the local tables would be 'Table A' & the linked tables were 'Table A1'. So there would be a query generated for 'Table A' and another query generated for 'Table A1'. Since there are no tables by the name 'Table A1' in Database B, the query wouldn't work. Plus the fact that, in the line of code executing the SQL, there was an option that says 'dbFailOnError' so since half of the queries weren't working, this option rolls back any updates made by the queries that did work.
I removed all of the linked tables and the macro runs perfectly, unless there are records that are the same on both databases. If I remove 'dbFailOnError' from the code, then the macro runs well no matter what.
So the macro is doing what I want it to but I would like to keep the 'dbFailOnError' part of the code so I will have to do 2 things. First, I have to solve the problem with the primary keys. The answer on this thread by Albert describes this problem well. Second, I have to adjust the SQL to be able to only select records that don't already exist in Database A. I am assuming I can do this by adding a WHERE to the end of the SQL.
I will make an update once I fix these problems, or if I just run an SQL server instead.
Thank you everyone for your help :)

Getting the number of affected records by an action query in VBA

I'm wondering if it is possible to pick a variable which is being used by MS Access when inserting a specific amount of new data rows to a table which happens during a VBA runtime.
Screenshot of the MS Access Notification (in German):
MS Access notifies me here in that case that in the middle of a running VBA script 2327 new data rows are being added to the table. I'm not a programmer but I feel that this variable must be stored somewhere (at least temporary).
Does anyone has an idea how to pick/access this number from the window when it appears and then store it into an VBA Integer for further use in the future?
Update:
Basically, I have a main database which has to be updated every day by an import file.
In the first step there is a check for already existing datasets which will therefore only updated by an UPDATE-Query.
UPDATE ReiseMaster
INNER JOIN Update_Import
ON(ReiseMaster.Col1 = Update_Import.Col1)
SET ReiseMaster.Col2 = Update_Import.Col2,
ReiseMaster.Col3 = Update_Import.Col3;
Then, in the second step new datasets which are not existing in the main database will be inserted into it will all the information they contain. This is the SQL-Query which is responsible for appending new data rows during the VBA procedure:
INSERT INTO ReiseMaster ([Col1],[Col2],[Col3])
SELECT [Col1],[Col2],[Col3] FROM Update_Import
WHERE NOT EXISTS(SELECT 1 FROM ReiseMaster
WHERE Update_Import.[Col1] = ReiseMaster.[Col1]);
I am struggling with identifying the amount of new datasets after the procedure, which is in fact already somehow determined by MS Access itself (see Pop-Up). So my idea was to just use the number which is already determined by MS Access.
All SQL-Queries are stored in a string and run by the "DoCmd.RunSQL" command.
Using DAO, it's really easy to get the number of records affected after executing a query.
You can just use the Database.RecordsAffected property:
Dim db As DAO.Database
Set db = CurrentDb 'Required, don't use CurrentDb.Execute else this won't work
Dim strSQL as String
strSQL = "UPDATE ReiseMaster INNER JOIN Update_Import" 'and the rest of your query
db.Execute strSQL, dbFailOnError 'Execute the query using Database.Execute
Dim recordsAffected As Long
recordsAffected = db.RecordsAffected 'Get the number of records affected
However, this won't allow you to see it before committing the query. To see it and display a prompt, you can use transactions (short sample here), and rollback the transaction if the user doesn't choose to commit.

How to create a live connection between MS Access and SQL Server?

The data I am using is stored in a SQL database and an Access database. Regular data entry needs to be made in the Access database, however, I would like to write my queries in SQL Management Studio. I have connected the two databases using an ODBC connection and I can now view the tables in Management studio but the connection is not live, so any updates in Access are not reflected in the tables in Management Studio.
You can create a linked table from Access to SQL Server (INSERT INTO). You can use SQL or VBA to export all data from Access to SQL Server.
Here is one idea.
Sub modExportToPG()
Dim rs As DAO.Recordset
'We only want to export physical user defined visible and hidden tables
Set rs = CurrentDb.OpenRecordset("SELECT Name " & _
" FROM MSysObjects " & _
" WHERE Type=1 AND Flags < 9 ORDER BY Name;")
Do Until rs.EOF
'export the tables but export as lower case table names
DoCmd.TransferDatabase acExport, "ODBC Database" _
, "ODBC;DRIVER={PostgreSQL Unicode};DATABASE=mydb;SERVER=myserver;PORT=5432;UID=myuser;PWD=mypwd" _
, acTable, rs("Name"), LCase(rs("Name"))
rs.MoveNext
Loop
rs.Close
End Sub
Also, see this for more ideas.
https://www.mssqltips.com/sqlservertip/1480/configure-microsoft-access-linked-tables-with-a-sql-server-database/
Came upon this s little late but I am in the midst of working through a project where we update SQL side tables at the same time that we update local and shared (server side) Access tables. It is a fairly extensive application, but the SQL updates are relatively easy and there are options. The application integrates with a Point of Sale back office server and we are sensitive to the data that gets updated instantly, so those items get tucked into batches that require approval before they get "posted" to SQL. Those items that are not sensitive we post immediately using the afterupdate property for each field on the main form using the following code (sample only):
Private Sub Field.AfterUpdate
Dim Var1 as string
Dim Var2 as string
Dim Var3 as string
'Capture Before and after values for the field (as well as using the_
newvalue to post to SQL, we store these separately as audit records_
so we capture both old and new).
Var1 = Me.Field.OldValue
Var2 = Me.Field.NewValue
Var3 = Me.PID.Text
'Execute the SQL update
DoCmd.DbExecute "Update dbo.SQLTABLE SET SQLField = '" & Var2 & "'_
WHERE SQLTABLE.PID = '" & Var3 & "'
End Sub
Note: PID is a record id attached to every record in the access form that matches a corresponding record in a SQL Table (primary key).
It is a lot of work if you have a number of fields to post but it's as good as it gets using linked tables in my experience. This is a simple example and ours are far more extensive, but it does the trick.

use ms access 2010 form to drill down into a table on a server

So I have a server and an access database. I would like to look at specific information from a table on the server, but this table has billions of records (too big for access).
Would it be possible to create a form where a user clicks on a specific name on that form and it runs a pass through query to the server to pull just those specific names? So if I click on the name "Tom" it runs a pass through query to the server and returns all records in the "name" column equal to "Tom".
What kind of server? MSSQL? MySQL? Oracle?
With same VBA code it is not that difficult to get results from a MSSQL server (don't know about other DBs). Add a button to your form and add a onClick-event.
Private Sub btnTEST_Click()
Dim cnn As New ADODB.Connection
Call cnn.Open("YOUR_CONNECTION_STRING")
Dim rs As ADODB.Recordset
Set rs = cnn.Execute("SELECT Field1 FROM BigTable WHERE Field2 = Forms!YourForm!lstboxField")
While Not rs.EOF
Debug.Print rs("Field1")
rs.MoveNext
Wend
rs.Close
End Sub

Using ADO/DAO Connection to Download data from SQL Server

I am trying to figure out how to download using an ADO/DAO connection in Access VBA to get the contents of a table from SQL server. I am trying to avoid using a linked table because the DB requires a password and I keep running into issues with getting it to not ask for the login info. Are there any ideas or references for me to start with on this matter?
It appears either way you'll need to provide SQL credentials.
There's more involved without linking a table, basically you'd want a recordset for the source and the "target" table to iterate over.
targetrs = CurrentDb.OpenRecordset("Target", dbOpenTable)
Dim Con As New ADODB.Connection
Dim sqlStr As String
Con.Open _
"Provider = sqloledb;" & _
"Data Source=SqlServer;" & _
"Initial Catalog=MyDB;" & _
"User ID=sa;" & _
"Password=p#ssW0rd;"
Dim rsSource As New ADODB.Recordset
rsSource.Open "select * from SOURCE", Con
do until rsSource.eof
targetrs.addnew
for each field in rsSource
targetrs.fields(field.Name) = rsSource.fields(field.Name)
next
targetrs.update
rssource.movenext
loop
Since you still have to have the credentials, you could dynamically link the table instead:
docmd.TransferDatabase acLink,"ODBC Database",
"ODBC;Driver={SQL Server};Server=MySQLServer;Database=MYSQLDB;
Uid=USER;Pwd=PASSWORD",acTable,"SQLtable","MyAccessTable"
Use of a linked table does not require you store or have the user password in that linked table.
If you execute a SINGLE logon at application startup then all linked tables will work.
Linked tables work WITHOUT a prompt for user or password.
Linked tables work WITHOUT you having to store the user ID or password in the link.
Access will cache the user name + password if you logon as per the instructions here:
http://blogs.office.com/b/microsoft-access/archive/2011/04/08/power-tip-improve-the-security-of-database-connections.aspx
So to download a table to a local, then you ONLY need this code:
For a new local table (create table query):
CurrentDb.Execute "SELECT * INTO LocalTableCreate FROM ServerTable"
Append to existing table:
CurrentDb.Execute "INSERT INTO LocalTable SELECT * FROM ServerTable"
And if some really strange reason and desire exists create and promote world poverty and do things the hard way like a turtle with time to waste and not use a linked table?
Well you could create a linked table via the “transfer database” command. It is only one extra line of code in front of the above code and then AGAIN the above two examples would work fine.
However I see little if any advantage to creating + deleting a linked table.
I suppose for reasons of performance or perhaps for security or the legitimate reason of you not knowing the table ahead of time? Then I would suggest you use a saved a pass-though query as performance will be even faster.
So you can use this code:
Dim qdfPass As DAO.QueryDef
Set qdfPass = CurrentDb.QueryDefs("MyPass")
qdfPass.SQL = "select * from dbo.MyTable;"
CurrentDb.Execute "INSERT INTO LocalTable SELECT * FROM MyPass”
Note that the sql used in above qerydef MUST be native T-SQL and can be a view or even a store procedure like:
qdfPass.SQL = "exec sp_myCoolStoreProc;"
And the stored procedure can even be passed a parameter like this:
qdfPass.SQL = "exec sp_myCoolStoreProc " & strMyParam
and then :
CurrentDb.Execute "INSERT INTO LocalTable SELECT * FROM MyPass”
So we can even use a select into/append from a store procedure by doing the above and the table/sql server side is dynamic or can even be a stored procedure. Again VERY little code.
I would suggest you avoid the idea proposed here to write recordset looping code unless one really has the desire to write looping code when none is required. And things like PK would have to be dealt with separate in code if you use such loops since the local pk column may need to be skipped (you simply leave that column out of the select SQL).
Note again that the connection string saved for the pass-though query does NOT require the user ID and password by using the above link showing how to “logon” to SQL Server. And if the table is known, then again a saved table link or pass-though query will suffice here.