VBA New Database Connection - sql

How to change the code below to prevent what you see in the screenshot.
I am running a macro with the following code
Dim conn As ADODB.Connection
Dim rec1 As ADODB.Recordset
Dim thisSql As String
Set conn = New ADODB.Connection
Dim sConn As String
sConn = "Provider=SQLOLEDB;Trusted_Connection=Yes;Server=xyz;Database=xyz;UID=xyz;PWD=xyz"
conn.Open sConn
' this is creating multiple connections.
Set rec1 = New ADODB.Recordset
rec1.Open thisSql, conn
which runs a SQL Server query (which is around 20 lines long and contains 4 joins). Everything is fine except for the fact that after a couple times of running it my DB admin says that my query is loading up the DB too much.
Now, my query could be causing the problem, or it could be that Excel is starting to run multiple connections at once. Some evidence for this is the screenshot below and the fact that the load on the database appears to increase with time.
How do I establish a DB connection without constantly creating new connections?
Has anyone had similar problems working with Excel DB macros?
UPDATE
While the answers below were very useful (especially for someone starting out in VBA), it seems that the main reason my query was taking up load was a combination of multiple connections and having overlooked a line in my code:
With Sheets("FVols").QueryTables.Add(Connection:=rec1, Destination:=Sheets("FVols").Range("A1"))
.name = "data"
.FieldNames = True
.Refresh BackgroundQuery:=True <<<<<<<<<<<<<<<<<<<<<<<-----
End With

You only need to open the connection once. That literally means you can execute multiple queries on that one active connection. You must close the connection and free the reference (specially with ADODB) to avoid running into collisions and other connection related problems.
If you know the queries you are going to be executing you can create an array (or collection) and add queries to the queue.
While you already have an open connection to work with you can keep executing queries.
Scan through code there is not much difference between yours and mine so you should be able to see what is going on and where. Please, ask questions in the comments if anything is unclear
Sub DbConnection()
Dim cn As ADODB.Connection
Set cn = New ADODB.Connection
Dim rs As ADODB.Recordset
Dim strConn As String
strConn = "Driver={SQL Server};Server=; Database=; UID=; PWD="
cn.Open strConn
Dim queryArr, i
queryArr = Array("SELECT * FROM [MyTable]", "SELECT * FROM [MyOtherTable]")
For i = LBound(queryArr) To UBound(queryArr)
ExecuteQuery queryArr(i), cn, rs
Next i
cn.Close
Set cn = Nothing
End Sub
Private Sub ExecuteQuery(query As Variant, ByRef cn As ADODB.Connection, ByRef rs As ADODB.Recordset)
Set rs = New ADODB.Recordset
With rs
.ActiveConnection = cn
.Open CStr(query)
Sheets(1).Range("A1").CopyFromRecordset rs
.Close
End With
Set rs = Nothing
End Sub
Now, you only need to execute the DBConnection() once and all the queries you listed in the array will be executed.
Alternatively, if your queries are created at run-time you can pass it to the DbConnection() as a parameter.
Sub DbConnection(queryQueue As Collection)
Dim cn As ADODB.Connection
Set cn = New ADODB.Connection
Dim rs As ADODB.Recordset
Dim strConn As String
strConn = "Driver={SQL Server};Server=HELIUM\PRI; Database=sourcedata; UID=tabula; PWD=Tabula123!"
cn.Open strConn
For i = 1 To queryQueue.Count
ExecuteQuery queryQueue.Item(i), cn, rs
Next i
cn.Close
Set cn = Nothing
End Sub
Private Sub ExecuteQuery(query As Variant, ByRef cn As ADODB.Connection, ByRef rs As ADODB.Recordset)
Set rs = New ADODB.Recordset
With rs
.ActiveConnection = cn
.Open CStr(query)
Sheets(1).Range("A1").CopyFromRecordset rs
.Close
End With
Set rs = Nothing
End Sub
Update:
You can declare your connection as a Global Variable. Now you can run the DBConnection() as many times as you like and you will not be creating a new connection each time. Instead you will be using the global connection object.
Option Explicit
Public cn As ADODB.Connection
Sub DbConnection()
Set cn = New ADODB.Connection
Dim rs As ADODB.Recordset
Dim strConn As String
strConn = "Driver={SQL Server};Server=; Database=; UID=; PWD="
cn.Open strConn
Set rs = New ADODB.Recordset
With rs
.ActiveConnection = cn
.Open "SELECT * FROM [MyTable]"
Sheets(1).Range("A1").CopyFromRecordset rs
.Close
End With
Set rs = Nothing
cn.Close
Set cn = Nothing
End Sub

Are you releasing the connection variable when you've finished with it? i.e.
Set rec1 = Nothing
The connection won't close fully if not.

Related

If i create a disconnected ADO recordset from scratch in VBA how do i set the base table information for UpdateBatch?

I have been using disconnected recordsets for a few weeks now, typically retrieving data from SQL Server, disconnecting the rs and filtering/formatting in VBA. Now i'm trying to do the reverse and create a new ADO recordset from scratch, and then connect it to my database and use UpdateBatch to insert the recordset into the database without using a loop. I have a fully populated recordset at this point, hooked it back up to my connection string, and try UpdateBatch. Understandably, it has no information at this point about what table I'm trying to update (only Data Source and Initial Catalog via the connection string). Is there a recordset property that I use to provide the table in question? Additionally, the table I'm trying to import into has a GUID field (first field) that I have left blank on purpose in my disconnected recordset assuming that upon import, SQL Server would assign this GUID/primary key automatically.
The specific error I'm getting after "rs.UpdateBatch" is
Run-time error '-2147467259 (80004005)'"
Insufficient base table information for updating or refreshing.
I know I could use a loop and a SQL command "INSERT INTO ...". I'd like to use a recordset object though since those provide much more functionality as a container for data. One thing I haven't tried is to first retrieve a recordset from the table in question, then clear it and re-populate it with the new data so that the recordset itself retains all of the original database and table properties. If that's the only/best approach I can try that route too. I just wanted to see if it was possible to create an ADO recordset, populate it, and then insert it into a matching table of my choice.
dim rs as ADODB.Recordset
set rs = New ADODB.Recordset
With rs.Fields
.append "alias", adVarChar, 255
.append "textA", adVarChar, 255
.append ......
End With
rs.Open
rs.AddNew Array(0, 1, 2, ..., n), Array(val0, val1, val2, ..., valn)
rs.Update
call importRS(rs)
rs.close
set rs = nothing
After rs.update above some recordsets may need to go to a database, other recordset objects are just used to expedite filtering and sorting so I just use them as a convenient container and they'd never go to importRS()
However, IF I need to send the disconnected recordset to a database, i'd like to just pass the recordset object to another function that serves the purpose of opening the connection, sending the update, and closing the connection. The code below would serve that purpose which is why i'd like to wait to establish a connection until this point, right at the end after my rs is populated.
sub importRS(byref rs as ADODB.Recordset)
dim cn as ADODB.Connection
set cn = New ADODB.Connection
cn.ConnectionString = strConnection 'my connection string variable'
cn.Open
rs.ActiveConnection = cn
rs.UpdateBatch '-------error message appears on this line
cn.close
set cn = nothing
You can get the data, (wherever it may be) into an array and add to the recordset using a loop. Then then when the loop is finished, you do rs.updatebatch as follows:
Private Sub SaveToSQLSever()
Dim lngLastRow As Long
Dim arrySheet As Variant
Dim rs As ADODB.Recordset
Dim cn As ADODB.Connection
Dim strCn As String
Set cn = New ADODB.Connection
Set rs = New ADODB.Recordset
strCn= "Provider=VersionOfSQL;User ID=*********;Password=*********;"
& _ "Data Source=ServerName;Initial Catalog=DataBaseName"
cn.Open strCn
On Error Goto exiting
'*********************************************************
'If the data is coming from a sheet
'Set to your Range
With Sheets("SheetName")
lngLastRow = .Range("A2").CurrentRegion.Rows _
(.Range("A2").CurrentRegion.Rows.Count).Row
arrySheet = .Range("A1:G" & lngLastRow).Value2
End With
'Else populate the array and pass it to this Sub
'*************************************************************
'Note the property parameters
'.Source = Table That you want to populate
With rs
.ActiveConnection = cn
.Source = "Select * from TableName"
.CursorType = adOpenDynamic
.CursorLocation = adUseClient
.LockType = adLockBatchOptimistic
.Open
End With
For i = LBound(arrySheet,1) To UBound(arrySheet,1)
rs.AddNew
For j = LBound(arrySheet,2) To UBound(arrySheet,2)
rs.Fields(j).Value = arrySheet(i,j)
Next j
rs.MoveNext
Next i
rs.UpdateBatch 'Updates the table with additions from the array
i = 0
'******************************************************************
'Note that you can also refer to the Field Names Explicitly Like So:
For i = LBound(arryData,1) To UBound(arryData,1)
With rs
.AddNew
.Fields("FieldName1").Value = arryData(i,1)
.Fields("FieldName2").Value = arryData(i,2)
.Fields("FieldName3").Value = arryData(i,3)
.Fields("FieldName4").Value = arryData(i,4)
.Fields("FieldName5").Value = arryData(i,5)
.Fields("FieldName6").Value = arryData(i,6)
.Fields("FieldName7").Value = arryData(i,7)
End With
Next i
rs.UpdateBatch
'******************************************************************
MsgBox "The data has successfully been saved to the SQL Server", _
vbInformation + vbOKOnly,"Alert: Upload Successful"
exiting:
If cn.State > 0 Then cn.Close
If rs.State > 0 Then rs.Close
Set cn = Nothing
Set rs = Nothing
End Sub
Edit: As per OP's request to pass an existing recordset to a SQL table, below should do so:
Private Sub SendRcrdsetToSQL(ByRef rsIn As ADODB.Recordset)
Dim arrySheet As Variant
Dim rsSQL As ADODB.Recordset
Dim cn As ADODB.Connection
Dim strCn As String
Set cn = New ADODB.Connection
strCn= "Provider=VersionOfSQL;User ID=*********;Password=*********;"
& _ "Data Source=ServerName;Initial Catalog=DataBaseName"
cn.Open strCn
On Error Goto exiting
Set rsSQL = New ADODB.Recordset
With rsSQL
.ActiveConnection = cn
.Source = "Select * from TableName"
.CursorType = adOpenDynamic
.CursorLocation = adUseClient
.LockType = adLockBatchOptimistic
.Open
End With
'disconnect the recordset and close the connection
Set rsSQL.ActiveConnection = Nothing
cn.Close
Set cn = Nothing
rsIn.MoveFirst
rsSQL.MoveLast
'Add the records from the passed recordset to the SQL recordset
Do While Not rsIn.EOF
With rsSQL
.AddNew
.Fields("FieldName1").Value = rsIn.Fields("FieldName1").Value
.Fields("FieldName2").Value = rsIn.Fields("FieldName2").Value
.Fields("FieldName3").Value = rsIn.Fields("FieldName3").Value
.Fields("FieldName4").Value = rsIn.Fields("FieldName4").Value
.Fields("FieldName5").Value = rsIn.Fields("FieldName5").Value
.Fields("FieldName6").Value = rsIn.Fields("FieldName6").Value
.Fields("FieldName7").Value = rsIn.Fields("FieldName7").Value
End With
rsIn.MoveNext
Loop
rsSQL.UpdateBatch
MsgBox "The data has successfully been saved to the SQL Server", _
vbInformation + vbOKOnly,"Alert: Upload Successful"
exiting:
If cn.State > 0 Then cn.Close
If rsIn.State > 0 Then rsIn.Close
If rsSQL.State > 0 Then rsSQL.Close
Set cn = Nothing
Set rsIn = Nothing
Set rsSQL = Nothing
End Sub
The only way I was able to get this to work was by running a query to build the structure of my Recordset. So your code becomes something like this:
Private Sub Command1_Click()
Dim cn As ADODB.Connection
Set cn = New ADODB.Connection
cn.ConnectionString = "<your connection string>"
cn.CursorLocation = adUseClient
cn.Open
Dim rs As ADODB.Recordset
Set rs = New ADODB.Recordset
Set rs.ActiveConnection = cn
rs.Open "select * from states where 1<>1", , adOpenStatic, adLockBatchOptimistic
rs.AddNew Array("Abbrev", "Name", "Region", "SchoolDataDirect"), Array("TN", "TestName", "MyRegion", 1)
Set rs.ActiveConnection = Nothing
cn.Close
ImportRS rs
End Sub
Private Sub ImportRS(ByRef rs As ADODB.Recordset)
Dim cn As ADODB.Connection
Set cn = New ADODB.Connection
cn.ConnectionString = "<your connection string>"
cn.CursorLocation = adUseClient
cn.Open
Set rs.ActiveConnection = cn
rs.UpdateBatch
Set rs.ActiveConnection = Nothing
cn.Close
End Sub

Creating persistent ADODB connection on workbook/worksheet opening

Right now I have an Excel worksheet full of hundreds (if not thousands) of functions, each one individually opening an ADODB connection to my SQL Server, executing a command, then closing the connection - this has led to some lengthy waiting periods while refreshing data.
Public Function Alere(StoredProcedure As String, paramItem As Range)
Dim conn As ADODB.Connection
Dim rs As ADODB.Recordset
connString = [connection string]
Set conn = New ADODB.Connection
Set rs = New ADODB.Recordset
conn.Open sConnString
Set rs = conn.Execute("[query]")
If Not rs.EOF Then
Alere = rs(0)
Else
Alere = "-"
End If
End Function
Would it be possible to open my connection on worksheet_activate(), and have it persist so that I can just reference it at any point? I have attempted putting the following at the top of the module:
Public conn As ADODB.Connection
Public rs As ADODB.Recordset
Set conn = New ADODB.Connection
Set rs = New ADODB.Recordset
conn.Open ("DSN=" & Worksheets("Variables").[H2] & ";Trusted_Connection=Yes;WSID=" & Environ("COMPUTERNAME") & ";DATABASE=" &
Worksheets("Variables").[H3])
...but I get issues regarding "Set" not being allowed outside of a function/sub.
So is there any way to have the connection persist and I simply continue to execute commands through it?

ADO Connection and Recordset with HTTP data source

I have the below Macro reading from a Database table stored as a txt file on the local C drive and returning an SQL query.
Public Function getData(fileName As String) As ADODB.Recordset
Dim cN As ADODB.Connection
Dim RS As ADODB.Recordset
Set cN = New ADODB.Connection
Set RS = New ADODB.Recordset
cN.Open ("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\Cloud\;Persist
Security Info=False;Extended Properties=""text; HDR=Yes; FMT=Delimited;
IMEX=1;""")
RS.ActiveConnection = cN
RS.Source = "select top 10 * from " & fileName
Set getData = RS
End Function
I can then call the function and return data using the below, so far so good...
Sub Cloud()
Dim a As ADODB.Recordset
Set a = getData("file.txt")
a.Open
MsgBox (a.GetString())
a.Close
End Sub
BUT now I would like to move 'file.txt' from C:\Cloud\ to a HTTP location, ie http://it.wont.work/
How would I amend the above for this to work? I've searched and tested but nothing seems to work... I either get internet login failed or ISAM not found.
Many thanks

Connection String for MS Access Database incorrect

I am trying to extrapolate data from an MS Access 2007/2010 Database.
I have the following code in VBA but the connection string is incorrect. I have added the relevant REFERENCES libraries
Private Sub btnGetMsAccessData_Click()
Dim sConn As String
Dim oConn As ADODB.Connection
Dim oRs As ADODB.Recordset
Dim sSQL As String
sConn = "Provider=Microsoft.ACE.OLEDB.12.0;User ID=Admin;Data Source=\\MyNetworkPath\BP-MasterDashboard Source\BP_Planning_by_PT_dept_be.accdb;Mode=Read"
Set oConn = New ADODB.Connection ' Open a connection.
oConn.Open
sSQL = "SELECT * FROM Tbl_Start_Leaver" ' Make a query over the connection.
Set oRs = New ADODB.Recordset
oRs.Open sSQL, , adOpenStatic, adLockBatchOptimistic, adCmdText
MsgBox oRs.RecordCount
oConn.Close ' Close the connection.
Set oConn = Nothing
End Sub
It fails saying Unknown Application error on the oConn.Open line.
I have tried to link a Workbook to one of the tables and this works fine.
I then looked at the "Connection" and copied it into my code but still no joy.
Keeps saying :
Automation Error
Unexpected Error
Any ideas would be appreciated.
Thanks in advance.
While the connection string was incorrect, there were other issues as well. Such as, not assigning the connection String to the ADODB Connection object as well as others. Here is the updated code that I hope will get you operational
Private Sub btnGetMsAccessData_Click()
'Ensure you add a reference to Microsoft ADO Objects
Dim oConn As New ADODB.Connection
Dim oRs As New ADODB.Recordset
Dim sSQL As String: sSQL = "SELECT * FROM Tbl_Start_Leaver"
'Corrected Connection String from Thomas Inzina
Dim sConn As String: sConn = "Provider=Microsoft.ACE.OLEDB.12.0;UID=Admin;Data Source=" & _
"\\MyNetworkPath\BP-MasterDashboard Source\BP_Planning_by_PT_dept_be.accdb;Mode=Read"
With oConn
.ConnectionString = sConn ' You need to assign the connection string to the ADODB.Connection Object
.Open
End With
'Make sure the connection isn't open before opening the recordset
'You also need to specify which connection you want to use as the second parameter (this was missed)
If oRs.State <> adStateOpen Then oRs.Open sSQL, oConn, adOpenStatic, adLockBatchOptimistic, adCmdText
'Close Connection and RS
If oConn.State = adStateOpen Then oConn.Close
If oRs.State = adStateOpen Then oRs.Close
'Clean Up
Set oRs = Nothing
Set oConn = Nothing
End Sub

Cannot open ADO Recordset more than once

I have a function that is passed a SQL string, queries a SQL server, and return the results via a an array. This function works perfectly the first time, but I try to call it again it get:
Operation is not allowed when the object is closed
Oddly enough this error occures right after the Open statement. I've Googled about everything I can think of, with no good results.
Public Function SQL_Query(SQLCommand As String) As Variant
Dim cn As New ADODB.Connection
Dim SQLrs As New ADODB.Recordset
cn.ConnectionString = "Provider=SQLNCLI11;Server=10.XXX.XXXX.XXX;DataBase=Database1;Trusted_Connection=yes;"
cn.Open
SQLrs.CursorLocation = adUseClient
Call SQLrs.Open(SQLCommand, cn, adOpenStatic, adLockBatchOptimistic)
SQLrs.MoveFirst
SQL_Query = RecordSet2Array(SQLrs.GetRows)
SQLrs.Close
Set SQLrs = Nothing
cn.Close
Set cn = Nothing
End Function
If I'm reading right - you're opening the SQL connection again, but not the recordset (SQLrs). You have to create a new instance each time on open and close both when done, or you tend to get these issues on re-running.
I've not experienced it personally, but I've been told that Dim x as NEW y can lead to issues.
Try this change to see if it helps:
Public Function SQL_Query(SQLCommand As String) As Variant
'changes here vvvvvvvv
Dim cn As ADODB.Connection
Dim SQLrs As ADODB.Recordset
Set cn = New ADODB.Connection
Set SQLrs = New ADODB.Recordset
'to here ^^^^^^^^^^^^^^
cn.ConnectionString = "Provider=SQLNCLI11;Server=10.XXX.XXXX.XXX;DataBase=Database1;Trusted_Connection=yes;"
cn.Open
SQLrs.CursorLocation = adUseClient
Call SQLrs.Open(SQLCommand, cn, adOpenStatic, adLockBatchOptimistic)
SQLrs.MoveFirst
SQL_Query = RecordSet2Array(SQLrs.GetRows)
SQLrs.Close
Set SQLrs = Nothing
cn.Close
Set cn = Nothing
End Function
So, I've finally tracked this to SQL query I was using in my second call to the function. It creates a temp table and returns records from that temp table. This is apparently and known issue. However it's easily corrected by SET NOCOUNT ON. So I've updated my code to the following and everything is working fine now.
Public Function SQL_Query(SQLCommand As String) As Variant
Dim cn As New ADODB.Connection
Dim SQLrs As New ADODB.Recordset
cn.ConnectionString = "Provider=SQLNCLI11;Server=10.XXX.XXXX.XXX;DataBase=Database1;Trusted_Connection=yes;"
cn.Open
cn.Execute "SET NOCOUNT ON"
SQLrs.CursorLocation = adUseClient
Call SQLrs.Open(SQLCommand, cn, adOpenStatic, adLockBatchOptimistic)
SQLrs.MoveFirst
SQL_Query = RecordSet2Array(SQLrs.GetRows)
SQLrs.Close
Set SQLrs = Nothing
cn.Close
Set cn = Nothing
End Function