I have a procedure in VBA that I've been using for some time now to pull the data set from SQL DB.
It always took less then 1 min to populate the template but since few days it's been taking more than 10 mins. Nothing changed in the database, nothing changed in my VBA code either:
It gets stuck on:
Sheets("NEW").Range("A3").CopyFromRecordset rst
I even upgraded from Office 2013 to 2016 but the issue still persists. Any guesses what could be the reason?
My code
Dim cnn As New ADODB.Connection
Dim rst As New ADODB.Recordset
Dim ConnectionString As String
Dim StrQuery As String
Dim Combine As String
Dim ow As Long
Dim ok As Integer
Dim i As Long
ConnectionString = "Provider=SQLOLEDB.1;Password=xxx;Persist Security Info=True;User ID=xxxx;Data Source=xxxx;Use Procedure for Prepare=1;Auto Translate=True;Packet Size=4096;Use Encryption for Data=False;Tag with column collation when possible=False;Initial Catalog=Share_Assignment_Automation"
cnn.Open ConnectionString
cnn.CommandTimeout = 900
StrQuery = "SELECT [Organization],null as [ERP Segment 1],[Effective date of assignment to Team],[Expiration date of assignment to Team],null as [Last Modified By],null as [Last Modified Date],[Territory Type Name],[Forecastable Flag],[Inside Sales Flag] FROM [vw_SHARETeam] with (nolock) "
rst.Open StrQuery, cnn
Sheets("TEAM").Range("A3").CopyFromRecordset rst
rst.Close
I had same issue. My query was 7 seconds but the paste was 60 seconds. (100,000 rows).
The issue was that I was pasting into an excel table which was auto-expanding its range.
I changed my code to disable auto-expand, then paste data, then manually expand the table range. This now takes 15 seconds total.
Related
I am using ADODB recordset to import some data from Postgres to Excel using VBA. I have one column where the number of characters exceed 255. Unfortunately, when pulling the data from Postgres, recordset gets populated only upto 255 characters for that column. Rest of the data is cut off. Is there anyway to overcome this limit? I have verified that there is no issue with the character limit for that column within Excel.
EDIT:
I cant share the code so here is an example of what my VBA code looks like:
Dim objConnection As Object
Dim objRecordset As Object
Dim sqlstring As String
ConStr= "Driver={PostgreSQL UNICODE};Server=IP
address;Port=5432;Database=myDataBase;Uid=myUsername;Pwd=myPassword;"
sqlstring = 'SELECT columnA, columnB, columnC from tableA'
Set objConnection = CreateObject("ADODB.connection")
Set objRecordset = CreateObject("ADODB.Recordset")
objRecordset.CursorLocation = 3
objConnection.Open ConStr
objRecordset.Open sqlstring, objConnection
For i = 0 To 2
If objRecordset.Fields(i).Value <> " " Or
objRecordset.Fields(i).Value <> Null Or objRecordset.Fields(i).Value
<> "" Then
ActiveCell.Value = objRecordset.Fields(i).Value
ActiveCell.Offset(0, 1).Activate
End If
Next i
In the above example, ColumnB is 1500 characters n length. But in Excel I see only 255 characters. The Recordset itself picks up only 255 characters. I am trying to overcome this limit.
I was able to resolve the issue by editing the connection string. Followed the page: odbc.postgresql.org/docs/config.html and added "B0=5000" in the connection string.
I'm trying to run a list of SQL queries where a condition exists for "code" and the values sit in a range of cells on another sheet (from cells A2 to A385).
I have the code below, however, I get an invalid object name for SQLQueries!$A2:A385
So, I understand the syntax is not correct but I'm struggling to find the correct one regardless of reading numerous articles.
Sub RunSQLQueries()
'Select SQLQueries sheet
Sheets("SQLQueries").Activate
'Initializes variables
Dim cnn As New ADODB.Connection
Dim rst As New ADODB.Recordset
Dim ConnectionString As String
Dim StrQuery As String
'Setup the connection string for accessing MS SQL database
ConnectionString = "Provider=SQLOLEDB; Data Source=HOSTNAME; Initial Catalog=DBNAME; UID=domain\user; Integrated Security=SSPI"
'Opens connection to the database
cnn.Open ConnectionString
'Timeout
cnn.CommandTimeout = 900
'Build SQK queries
StrQuery = "SELECT * FROM table WHERE code IN (SELECT * FROM [SQLQueries!$A2:A385])"
'Performs the queries
rst.Open StrQuery, cnn
'Select Results sheet
Sheets("Results").Activate
'Dumps all the results from the StrQuery into cell A2 of the active sheet
Range("A2").CopyFromRecordset rst
End Sub
The result I'm expecting is for a SQL query to be run using each condition from the range of values with the results being populated on the "Results" sheet from cells A2 down
The query string is literally sent to the database server, and since your sql attempts to refer to an excel list that the server cannot access it returns an error. The server is looking for a table named [SQLQueries!$A2:A385]
To stick with your current plan, you will need to pass the the IN () clause literally or by vba variable that is formatted as such:
IN ( 'item1', 'item2' ...)
Note:You can remove single quotes if the items are numeric
I advise rethinking the plan by either
1) if it is possible to adjust things in the database side: Can you create a new reference table to join to the actual table or create a view that only returns desired rows? Then you would need a job that tweaks the filtering view/ table before running the query. The idea being you would not need to adjust the query each time bc a constant sql string would return the rows you need
Or
2) if the source table has say 100k rows or less, and data is not too wide, just select all the rows into excel in a new sheet, then filter that sheet (add a new column in excel that returns true using vlookup against your reference sheet) or use vlookup on your reference sheet to pull the desired columns
Here's a suggestion:
StrQuery = "SELECT * FROM table WHERE code IN (" & _
InList(Sheets("SQLQueries").Range("A2:A385"),True) & ")"
Function to create a SQL "in" list given a range:
Function InList(rng As Range, quoted As Boolean)
Dim qt, a, r As Long, c As Long, rv As String, v, sep As String
a = rng.Value
qt = IIf(quoted, "'", "")
For r = 1 To UBound(a, 1)
For c = 1 To UBound(a, 2)
v = Trim(a(r, c))
If Len(v) > 0 Then
rv = rv & sep & qt & v & qt
sep = ","
End If
Next c
Next r
InList = rv
End Function
Notes:
Pass False as the second argument if you have numeric values
I wouldn't use this for very large lists
Be very certain you're not at risk from possible SQL injection issues: parameterized queries are always preferable but do not work with "in" lists
I am writing a SQL query that checks excel values against a database. While running my Excel macro, I create worksheet 2 (ws2) and need to run a query which checks if each of the values in column F = table.number.
I know I can use Cells to get a single value and wrap it in a for loop but that takes up too much processing and requires too many SQL extracts. The column in ws2 is called "REFERENCE" and has all the data below it. Ideally, I would like to write the SQL query like:
select * from table where ws2.REFERENCE = table.number
Is there a way to do this?
I suppose you can do the comparison in either Excel or in SQL Server, right. I think one viable solution is to import data from SQ Server, including field names, and do compare this to what you already have in Excel.
Sub Conn2SQL()
Dim cnn1 As New ADODB.Connection
Dim mrs As New ADODB.Recordset
Dim iCols As Integer
Set cnn1 = New ADODB.Connection
cnn1.ConnectionString = "driver={SQL Server};server=MyDBServer;uid=MyuserID;pwd=mypassword;database=MyDB"
cnn1.ConnectionTimeout = 30
cnn1.Open
SQry = "use MyDB select * from TableName"
mrs.Open SQry, cnn1
For iCols = 0 To mrs.Fields.Count - 1
Worksheets("Sheet2").Cells(1, iCols + 1).Value = mrs.Fields(iCols).Name
Next
Sheet2.Range("A2").CopyFromRecordset mrs
mrs.Close
cnn1.Close
End Sub
I connect to an oracle database with an ADODB connection in Excel. When i run the query in SQL developer, it run about 8 secs (this is for the full result of 45 000 records). When i run the same quakes very in Excel vba it takes very long, about 20 minutes. Why would it take so long in Excel?
Here is an example of my code,
Set rs = New ADODB.Recordset
Set con = New ADODB.Connection
con.CursorLocation = adUseClient
con.Open "Provider=OraOLEDB.Oracle;Data Source=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)" & _
"(HOST=JASONPC)(PORT=1521)))(CONNECT_DATA=(SID=jasonpc)));User Id=jason;Password=jason;DeferUpdChk=true"
rs.CursorType = ADODB.adOpenStatic
rs.LockType = adLockOptimistic
rs.Open sql, con
Set rs.ActiveConnection = Nothing
con.Close
Set con = Nothing
If you execute for full 45K records it will slow your execution or hangs Excel. So run it from 5K query
First 10k then next 10k and so on Using row num in the query...
So you need to modify your based on that Query
I have an MS Access (.accdb) table with data like the following:
Location Number
-------- ------
ABC 1
DEF 1
DEF 2
GHI 1
ABC 2
ABC 3
Every time I append data to the table I would like the number to be unique to the location.
I am accessing this table through MS Excel VBA - I would like to create a new record (I specify the location in the code) and have a unique sequential number created.
Is there a way to setup the table so this happens autmatically when a record is added?
Should I write a query of some description and to determine the next number per location, and then specify both the Location & Number when I create the record?
I am writing to the table as below:
Set rst = New ADODB.Recordset
rst.CursorLocation = adUseServer
rst.Open Source:="Articles", _
ActiveConnection:=cnn, _
CursorType:=adOpenDynamic, _
LockType:=adLockOptimistic, _
Options:=adCmdTable
rst.AddNew
rst("Location") = fLabel.Location 'fLabel is an object contained within a collection called manifest
rst("Number") = 'Determine Unique number per location
rst.Update
Any help would be appreciated.
Edit - Added the VBA code I am struggling with as question was put on-hold
I suspect that you are looking for something like this:
Dim con As ADODB.Connection, cmd As ADODB.Command, rst As ADODB.Recordset
Dim newNum As Variant
Const fLabel_Location = "O'Hare" ' test data
Set con = New ADODB.Connection
con.Open "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=C:\Users\Public\Database1.accdb;"
Set cmd = New ADODB.Command
cmd.ActiveConnection = con
cmd.CommandText = "SELECT MAX(Number) AS maxNum FROM Articles WHERE Location = ?"
cmd.CreateParameter "?", adVarWChar, adParamInput, 255
cmd.Parameters(0).Value = fLabel_Location
Set rst = cmd.Execute
newNum = IIf(IsNull(rst("maxNum").Value), 0, rst("maxNum").Value) + 1
rst.Close
rst.Open "Articles", con, adOpenDynamic, adLockOptimistic, adCmdTable
rst.AddNew
rst("Location").Value = fLabel_Location
rst("Number").Value = newNum
rst.Update
rst.Close
Set rst = Nothing
Set cmd = Nothing
con.Close
Set con = Nothing
Note, however, that this code is not multiuser-safe. If there is the possibility of more than one user running this code at the same time then you could wind up with duplicate [Number] values.
(To make the code multiuser-safe you would need to create a unique index on ([Location], [Number]) and add some error trapping in case the rst.Update fails.)
Edit
For Access 2010 and later consider using an event-driven Data Macro and shown in my other answer to this question.
You need to add a new column to your table of data type AutoNumber.
office.microsoft.com: Fields that generate numbers automatically in Access
You should probably also set this column as your primary key.
For Access 2010 and newer, this is a better way to do it. It uses the table's Before Change Data Macro to derive the next sequential number and put it in the [Number] field of the new record:
The advantages of this approach are:
The sequence number will be applied whenever a new record is added, regardless of how it is added.
The Excel VBA code does not have to worry about creating the sequence number; it "just happens".
Since this code resides at the table level it should be safe for a multi-user environment.
For more information on Data Macros, see
Create a data macro