Uploading Volatile Table using VBA - sql

I have a local table in an MS Access 2010 database. I would like to use ADODB to load the local table into a Teradata environment within my spool space using CREATE VOLATILE TABLE command
This should be achievable i think using arrays but i was wondering is there a better way (speedy too if possible) to batch load the records into the teradata environment for example
CREATE VOLATILE TABLE EXAMPLE AS (SELECT * FROM LOCAL TABLE)
WITH DATA PRIMARY INDEX(Set Index Keys)
ON COMMIT PRESERVE ROWS;
Thanks for your help
Just a quick update on this [24/01/2013]
I have managed to get the data from the access database, Load it into an Array, Create a Volatile table in my teradata warehouse using spool space and then attempt to load the array into the table
I am getting the error: TEMP_TABLE already exists. when trying to append the records to the table
I am almost there so any help would be greatly appreciated. Thank you again for your help
Public Function Open_Connection()
On Error GoTo ErrorHandler
' http://voices.yahoo.com/teradata-ms-excel-vba-2687156.html
' The connection is used to connect with the DBMS,
' The Recordset to surf the result of some SELECT queries
' The Command to send the sql requests to Teradata.
Dim cn As ADODB.Connection
Set cn = New ADODB.Connection
Dim rs As ADODB.Recordset
Set rs = New ADODB.Recordset
Dim cmdSQLData As ADODB.Command
Set cmdSQLData = New ADODB.Command
Dim myArray() As Variant
' Open Connection With Teradata
cn.Open "Data Source=Database; " & _
"Database=Database; " & _
"Persist Security Info=True; " & _
"User ID=Me; " & _
"Password=FakePassword; " & _
"Session Mode=ANSI;"
' Which database it has to send the query
Set cmdSQLData.ActiveConnection = cn
' Query to create the violotile table in Teradata'
Query = "CREATE VOLATILE TABLE TEMP_TABLE ( " & _
"field1 VARCHAR (39)," & _
"field2 VARCHAR (44)," & _
"field3 DECIMAL (18, 3)," & _
"field4 CHAR (3))" & _
"ON COMMIT PRESERVE ROWS;"
' Query is assessed as Text
cmdSQLData.CommandText = Query
' Specifies which kind of command VBA has to execute
cmdSQLData.CommandType = adCmdText
' Remove Query Timeouts
cmdSQLData.CommandTimeout = 60
'VBA just run the query and send back the result
Set rs = cmdSQLData.Execute()
' Retrieve the sharepoint records into an Array
myArray = Retrieve_Sharepoint_Records
intNumberRows = UBound(myArray, 2) + 1 ' number of records/rows in the array
rowcounter = 0
' Append the Rows to the temp table
For rowcounter = 0 To intNumberRows - 1
' Debug.Print myArray(0, rowcounter) & " | " & myArray(1, rowcounter) & " | " & myArray(2, rowcounter) & " | " & myArray(3, rowcounter)
AppendQuery = "INSERT INTO TEMP_TABLE VALUES ('" & myArray(0, rowcounter) & "','" & myArray(1, rowcounter) & "'," & myArray(2, rowcounter) & ",'" & myArray(3, rowcounter) & "');"
cmdSQLData.CommandText = Query
cmdSQLData.CommandType = adCmdText
cmdSQLData.CommandTimeout = 60
Set rs = cmdSQLData.Execute()
Next
' Clean up
cn.Close
Set cn = Nothing
Set rs = Nothing
Set cmdSQLData = Nothing
ErrorHandler:
If (Len(Err.Description) > 0) Then
MsgBox (Err.Description)
End If
End Function

You may have better success using a Global Temporary Table as the object definition is stored within the DBC Data Dictionary on Teradata. The population of the table is session specific using the TEMPORARY space assigned to the user or the profile of the user.

Just to answer my own question, I should change the second code line of below
cmdSQLData.CommandText = Query
To
cmdSQLData.CommandText = AppendQuery
It then works perfectly albeit slowly
Again thank you all for your help

Related

Excel import to SQL via VBA (Run-Time Error)

I am trying to import via a VBA button a ton of Excel data (around 30k + daily) into an existing table in SQL server. My question is how can I do this as simple as possible, code speaking?
The headers both in my Excel file and SQL table are 1:1 the same so I just want to import everything into the SQL table
This is what I started to write but when I try to make the code work I get a "Run-time error '-2147217865 (80040e37): Invalid object name "dbo.Rawdata".
Private Sub cmdImport_Click()
Dim r As Range
Dim c As Range
Set r = Sheet1.Range("A6:DA269239")
Dim con As ADODB.Connection
Set con = New ADODB.Connection
con.ConnectionString = _
"Provider=MSOLEDBSQL;" & _
"Server =localhost\name" & _
"Database =name;" & _
"Trusted_Connection=yes;"
con.Open
Dim iRowNo As Integer
Dim strn_reference As String
Dim batchInsert As String
Dim batchSize As Integer
batchSize = 1000
iRowNo = 0
For Each cl In r
iRowNo = iRowNo + 1
batchInsert = batchInsert + (IIf(iRowNo > 1, ",", "")) + "('" & Replace(cl.Value2, "'", "''") & "')"
If (iRowNo = batchSize) Then
con.Execute "insert into dbo.Rawdata (trn_reference) Values " & batchInsert
iRowNo = 0
batchInsert = ""
End If
Next
If Len(batchInsert) > 0 Then con.Execute "insert into dbo.Rawdata (trn_reference) Values " & batchInsert
MsgBox "Reference Numbers imported"
con.Close
Set con = Nothing
End Sub
Thank you everyone for the help!
I guess you should refer to the table name from SQL server not dbo.Rawdata but directly:
Rawdata|"Insert into Rawdata(column_name) VALUES ('" & vba_variable & "')"
This should be the SQL statement from VBA.
This work for me very well.

VBA Update sql from Excel to MS SQL Server 2008 using ListColumns

I am working on an Excel 2010 Workboox where a macro pulls in data from a database table. The users can then update the values of Column06) to a specific value if needed. Once complete, they can run a macro to run a SQL update so Column06 in the database is updated where COLUMN01 and COLUMN02 are in the database table. I know the ADO connection is working as I tried with a very generic sql which worked fine. I know that the table could be of varying lengths, so I knew I probably needed to loop through the rows, and that's where I'm stuck.
I tried setting up a loop similar to another solution I found online, and started getting Run-Time Error 91 "Object variable or with block variable not set". I think is due to the ListObject.ListColumns I'm using in the new Update Statement. I've tried using other examples to declare these, but it usually ends up in other errors. I must be missing something, or doing something wrong. Any help would be greatly appreciated.
Sub Updatetbl_data()
'
' Updatetbl_data Macro
' test
Sheets("Sheet2").Select
Dim cnn As ADODB.Connection
Dim uSQL As String
Set cnn = New Connection
cnnstr = "Provider=SQLOLEDB; " & _
"Data Source=MySource; " & _
"Initial Catalog=MyDB;" & _
"User ID=ID;" & _
"Password=Pass;" & _
"Trusted_Connection=No"
cnn.Open cnnstr
' New Update Statement idea based on possible solution found online
Dim row As Range
For Each row In [tbl_data].Rows
uSQL = "UPDATE tbl_data SET Column06 = '" & (row.Columns (row.ListObject.ListColumns("Column06").Index).Value) & _
"' WHERE Column01 = '" & (row.Columns(row.ListObject.ListColumns ("Column01").Index).Value) & _
"' AND Column02 = '" & (row.Columns(row.ListObject.ListColumns("Column02").Index).Value) & "' "
'Debug.Print (uSQL)
cnn.Execute uSQL
Next
cnn.Close
Set cnn = Nothing
Exit Sub
'
End Sub
Perhaps row.Columns is not designed for what you want to achieve. You can give this link to another article on stackoverflow a look for some more information. Next, I made some changes to your code which might do the trick.
' ... ... ...
Dim row As Range
'For Each row In [tbl_data].Rows ==>> has to be replaced by
'For Each row In [tbl_data] ==>> which returns all cells, perhaps better the following
Const ColNbr_Column01 As Long = 1
Const ColNbr_Column02 As Long = 2
Const ColNbr_Column06 As Long = 6
' now, select only the first column of the range [tbl_data]
For Each row In Range( _
[tbl_data].Cells(1, 1).Address, _
[tbl_data].Cells([tbl_data].Rows.Count, 1).Address)
' now, use offset to reach to the columns in the row
uSQL = "UPDATE tbl_data SET Column06 = '" & row.Offset(0, ColNbr_Column06).Value & _
"' WHERE Column01 = '" & row.Offset(0, ColNbr_Column01).Value & _
"' AND Column02 = '" & row.Offset(0, ColNbr_Column02).Value & "' "
'Debug.Print (uSQL)
' ... ... ...
This is the basic concept.
Sub InsertInto()
'Declare some variables
Dim cnn As adodb.Connection
Dim cmd As adodb.Command
Dim strSQL As String
'Create a new Connection object
Set cnn = New adodb.Connection
'Set the connection string
cnn.ConnectionString = "Provider=SQLOLEDB.1;Integrated Security=SSPI;Persist Security Info=True;Initial Catalog=Database;Data Source=Server_Name"
'Create a new Command object
Set cmd = New adodb.Command
'Open the Connection to the database
cnn.Open
'Associate the command with the connection
cmd.ActiveConnection = cnn
'Tell the Command we are giving it a bit of SQL to run, not a stored procedure
cmd.CommandType = adCmdText
'Create the SQL
strSQL = "UPDATE TBL SET JOIN_DT = '2017-10-08' WHERE EMPID = 1"
'Pass the SQL to the Command object
cmd.CommandText = strSQL
'Execute the bit of SQL to update the database
cmd.Execute
'Close the connection again
cnn.Close
'Remove the objects
Set cmd = Nothing
Set cnn = Nothing
End Sub

Extracting data from over a million records

I have an Excel file in which I have set up a connection with an Access database. In the Excel file I have a list of names in column A, and I want to search these names in the Access database and return back two fields from that database. I need to do this for around 200-300 names.
Here is my code:
N = Cells(Rows.Count, "A").End(xlUp).Row
Application.DisplayAlerts = False
strDB = ThisWorkbook.Path & "file.accdb"
Set objConnection = New ADODB.Connection
objConnection.Open "Provider=Microsoft.ACE.OLEDB.12.0; Data Source=" & strDB
For i = 2 To N
Dim rstTable As ADODB.Recordset
Set rstTable = New ADODB.Recordset
lookup = Range("A" & i).Value
strSQL = "SELECT NAME1,NAME2 FROM DATA WHERE [Field2]= """ & lookup & """;"
'Store query output
rstTable.Open Source:=strSQL, ActiveConnection:=objConnection
'Paste results to Transactions sheet
Worksheets("Sheet1").Range("B" & i).CopyFromRecordset rstTable
'Close the record set & connection
rstTable.Close
objConnection.Close
Next i
This works (kindof) but it takes an extremely long time and randomly crashes. Any ideas how to improve this?
Making sure there is a key on the lookup field will help. I would suggest making a copy of the workbook and test external data from Access or MS Query to see if that gives a performance gain over VBA.
When using MS Query or data from Access, you can modify the command text in the connection properties and use ? in the where clause to specify the parameter in the worksheet (so you don't lose that functionality).
I modified your SQL statement. Replace the Where [Field2] = "xxx" by Where [Field2] IN ("xxx", "yyy", "zzz").
N = Cells(Rows.Count, "A").End(xlUp).Row
Application.DisplayAlerts = False
strDB = ThisWorkbook.Path & "file.accdb"
Set objConnection = New ADODB.Connection
objConnection.Open "Provider=Microsoft.ACE.OLEDB.12.0; Data Source=" & strDB
For i = 2 To N
lookup = lookup & "'" & Range("A" & i).Value & "', "
Next i
lookup = left(lookup, len(lookup) - 2)
Dim rstTable As ADODB.Recordset
Set rstTable = New ADODB.Recordset
strSQL = "SELECT NAME1,NAME2 FROM DATA WHERE [Field2] IN (" & lookup & ");"
'Store query output
rstTable.Open Source:=strSQL, ActiveConnection:=objConnection
'Paste results to Transactions sheet
Worksheets("Sheet1").Range("B" & i).CopyFromRecordset rstTable
'Close the record set & connection
rstTable.Close
objConnection.Close
You close the connection after the first iteration, so your next iteration -- which does not have code to open the connection -- would fail. So you should move the objConnection.Close out of the loop.
But, even then, to execute the same kind of query over and over again, just with a different argument, can be done in one go, using the IN (...) syntax:
' Declare all your variables
Dim N As Long
Dim strDB As String
Dim objConnection As ADODB.Connection
Dim rstTable As ADODB.Recordset
Dim strSQL As String
N = Cells(Rows.Count, "A").End(xlUp).Row
Application.DisplayAlerts = False
strDB = ThisWorkbook.Path & "file.accdb"
Set objConnection = New ADODB.Connection
objConnection.Open "Provider=Microsoft.ACE.OLEDB.12.0; Data Source=" & strDB
' collect the values in comma-separated string
lookup = ""
For i = 2 To N
lookup = lookup & ",""" & Range("A" & i).Value & """"
Next i
' Chop off the first comma
lookup = Mid(lookup, 2)
' Perform a single query, but also select the Field2 value
Set rstTable = New ADODB.Recordset
strSQL = "SELECT Field2, NAME1,NAME2 FROM DATA WHERE [Field2] IN (" & lookup & ");"
' query output
rstTable.Open Source:=strSQL, ActiveConnection:=objConnection
' Retrieve values
While Not rstTable.EOF
lookup = rstTable.Fields(0).Value
' Locate in which row to put the result
For i = 2 To N
If lookup = Range("A" & i).Value Then
Range("B" & i).Value = rstTable.Fields(1).Value
Range("C" & i).Value = rstTable.Fields(2).Value
End If
Next i
rstTable.MoveNext
Loop
' Close the record set & connection
rstTable.Close
objConnection.Close
You can do what you described, but I think it's far more efficient to do this in Access itself. Just create a table with your names and do an Inner Join to the table you want to find 2 fields. Should take less than a minute, and probably less than 30 seconds.

Access VBA query to SQL Server

Hello experts I'm having trouble in my update query from SQL Server. Running first a select query then pass the result to currentdb.execute (to update the table of the access file currently using), using Access vba I'm not doing it right. I really hope you could help me. Maybe you guys know much better way to run my procedure:
connect to sql server 2008, run select query.
pass the result of select query to an access database execute command (or if you have a better idea) to update a table in the current access file that is using.
The error I'm getting to the code is Type mismatch and highlighting .OpenSchema.
These is part of the code that I made wrong (and I really have no idea how to do this).
dbObj.Execute ("UPDATE ACCESS.tbl_Name RIGHT JOIN " & _
conn.OpenSchema(adSchemaTables, Array(Empty, Empty, Empty, "SQLSVR.tbl_Name")) & _
" ON ACCESS.tbl_Name.FieldName_access = " & rst!FieldName_sqlsvr & _
" SET ACCESS.tbl_Name.FieldName_access = " & rst!FieldName_sqlsvr)
These is the whole code.
Option Compare Database
Sub LocalSQLServerConn_Test()
Dim dbOjb As DAO.Database
Dim strDBName As String
Dim strUserName As String
Dim strPassword As String
Set dbObj = CurrentDb()
Set conn = New adodb.Connection
Set rst = New adodb.Recordset
strDBName = "DataSet"
strConnectString = "Provider = SQLOLEDB.1; Integrated Security = SSPI; " & _
"Initial Catalog = " & strDBName & "; Persist Security Info = True; " & _
"Workstation ID = ABCDE12345;"
conn.ConnectionString = strConnectString
conn.Open
strSQL = "SELECT DISTINCT SQLSVR.tbl_Name.FieldName_sqlsvr FROM SQLSVR.tbl_Name"
rst.Open Source:=strSQL, ActiveConnection:=conn, _
CursorType:=adOpenDynamic, LockType:=adLockOptimistic
If rst.RecordCount = 0 Then
MsgBox "No records returned"
Else
rst.MoveFirst
Do While Not rst.EOF
dbObj.Execute ("UPDATE ACCESS.tbl_Name RIGHT JOIN " & _
conn.OpenSchema(adSchemaTables, Array(Empty, Empty, Empty, "SQLSVR.tbl_Name")) & _
" ON ACCESS.tbl_Name.FieldName_access = " & rst!FieldName_sqlsvr & _
" SET ACCESS.tbl_Name.FieldName_access = " & rst!FieldName_sqlsvr)
rst.MoveNext
Loop
End If
conn.Close
rst.Close
Set dbObj = Nothing
End Sub
You should add a linked table (or a pass-through query) to get the data from SQL Server, create an Update Query in your MDB, using a JOIN to update all rows at once (your can use the query designer for this part) and then execute that query using CurrentDb.Execute.

MS Access linked to SQL server views

we have an issue with an access database we are upgrading to use SQL Server as its data store.
This particular database links to 2 sql databases, so I thought to simplify things, we have a view in the main database that linked to each table in the secondary database. That way access would only need to talk directly with one SQL database.
When we linked access to the database views we choose which fields were the primary keys so the views were not readonly. We have standard code that refreshes all links when a database opens to pickup any changes and the linked views become readonly because the primary key information is lost.
Is there a way of refreshing the links to views while retaining the primary key information?
John
I have included my entire ODBC Reconnect function below. This function is predicated with the idea that I have a table called rtblODBC which stores all of the information I need to do the reconnecting. If you implement this function, you will NOT need to worry about connecting to multiple SQL databases, as that is handled smoothly with each table to be reconnected having its own connection string.
When you get towards the end you will see that I use DAO to recreate the primary keys with db.Execute "CREATE INDEX " & sPrimaryKeyName & " ON " & sLocalTableName & "(" & sPrimaryKeyField & ")WITH PRIMARY;"
If you have any questions, please ask.
Public Function fnReconnectODBC( _
Optional bForceReconnect As Boolean _
) As Boolean
' Comments :
' Parameters: bForceReconnect -
' Returns : Boolean -
' Modified :
' --------------------------------------------------'
On Error GoTo Err_fnReconnectODBC
Dim db As DAO.Database
Dim rs As DAO.Recordset
Dim tdf As DAO.TableDef
Dim sPrimaryKeyName As String
Dim sPrimaryKeyField As String
Dim sLocalTableName As String
Dim strConnect As String
Dim varRet As Variant
Dim con As ADODB.Connection
Dim rst As ADODB.Recordset
Dim sSQL As String
If IsMissing(bForceReconnect) Then
bForceReconnect = False
End If
sSQL = "SELECT rtblODBC.LocalTableName, MSysObjects.Name, MSysObjects.ForeignName, rtblODBC.SourceTableName, MSysObjects.Connect, rtblODBC.ConnectString " _
& "FROM MSysObjects RIGHT JOIN rtblODBC ON MSysObjects.Name = rtblODBC.LocalTableName " _
& "WHERE (((rtblODBC.ConnectString)<>'ODBC;' & [Connect]));"
Set con = Access.CurrentProject.Connection
Set rst = New ADODB.Recordset
rst.Open sSQL, con, adOpenDynamic, adLockOptimistic
'Test the recordset to see if any tables in rtblODBC (needed tables) are missing from the MSysObjects (actual tables)
If rst.BOF And rst.EOF And bForceReconnect = False Then
'No missing tables identified
fnReconnectODBC = True
Else
'Table returned information, we don't have a perfect match, time to relink
Set db = CurrentDb
Set rs = db.OpenRecordset("rtblODBC", dbOpenSnapshot)
'For each table definition in the database collection of tables
For Each tdf In db.TableDefs
'Set strConnect variable to table connection string
strConnect = tdf.Connect
If Len(strConnect) > 0 And Left(tdf.Name, 1) <> "~" Then
If Left(strConnect, 4) = "ODBC" Then
'If there is a connection string, and it's not a temp table, and it IS an odbc table
'Delete the table
DoCmd.DeleteObject acTable, tdf.Name
End If
End If
Next
'Relink tables from rtblODBC
With rs
.MoveFirst
Do While Not .EOF
Set tdf = db.CreateTableDef(!localtablename, dbAttachSavePWD, !SourceTableName, !ConnectString)
varRet = SysCmd(acSysCmdSetStatus, "Relinking '" & !SourceTableName & "'")
db.TableDefs.Append tdf
db.TableDefs.Refresh
If Len(!PrimaryKeyName & "") > 0 And Len(!PrimaryKeyField & "") > 0 Then
sPrimaryKeyName = !PrimaryKeyName
sPrimaryKeyField = !PrimaryKeyField
sLocalTableName = !localtablename
db.Execute "CREATE INDEX " & sPrimaryKeyName & " ON " & sLocalTableName & "(" & sPrimaryKeyField & ")WITH PRIMARY;"
End If
db.TableDefs.Refresh
.MoveNext
Loop
End With
subTurnOffSubDataSheets
fnReconnectODBC = True
End If
rst.Close
Set rst = Nothing
con.Close
Set con = Nothing
Exit_fnReconnectODBC:
Set tdf = Nothing
Set rs = Nothing
Set db = Nothing
varRet = SysCmd(acSysCmdClearStatus)
Exit Function
Err_fnReconnectODBC:
fnReconnectODBC = False
sPrompt = "Press OK to continue."
vbMsg = MsgBox(sPrompt, vbOKOnly, "Error Reconnecting")
If vbMsg = vbOK Then
Resume Exit_fnReconnectODBC
End If
End Function
A good deal of DSN less code that re-links access tables to SQL server often deletes the links first, then recreates the link. The code then sets up the connection string. Thus, it is the deleting that causes you to lose what the primary key was/is.
I actually recommend that you modify your re-link code as to not delete the table links.
Try something like:
For Each tdfCurrent In dbCurrent.TableDefs
If Len(tdfCurrent.Connect) > 0 Then
If Left$(tdfCurrent.Connect, 5) = "ODBC;" Then
strCon = "ODBC;DRIVER={sql server};" & _
"SERVER=" & ServerName & ";" & _
"DATABASE=" & DatabaseName & ";" & _
"UID=" & UserID & ";" & _
"PWD=" & USERpw & ";" & _
"APP=Microsoft Office 2003;" & _
"WSID=" & WSID & ";"
End If
End If
tdfCurrent.Connect = strCon
tdfCurrent.RefreshLink
End If
Next tdfCurrent
This works a litte better for me (note the moved end if's):
Dim dbCurrent As Database
Set dbCurrent = CurrentDb()
StatusList.SetFocus
StatusList.AddItem ("starting... ")
I = DoEvents()
Dim tdfCurrent As DAO.TableDef
For Each tdfCurrent In dbCurrent.TableDefs
If Len(tdfCurrent.Connect) > 0 Then
If Left$(tdfCurrent.Connect, 5) = "ODBC;" Then
strCon = "ODBC;DRIVER={sql server};" & _
"SERVER=" & ServerName & ";" & _
"DATABASE=" & DatabaseName & ";" & _
"UID=" & UserID & ";" & _
"PWD=" & USERpw & ";" & _
"APP=Microsoft Office 2003;" & _
"WSID=" & WSID & ";"
StatusList.AddItem ("fixing " & tdfCurrent.Name)
tdfCurrent.Connect = strCon
tdfCurrent.RefreshLink
End If
End If
I = DoEvents()
Next tdfCurrent
StatusList.AddItem ("----Done.")
The ODBC check is correct, even though the "ODBC;" part doesn't show in the MSysObjects view.