I am using ADODB recordset to import some data from Postgres to Excel using VBA. I have one column where the number of characters exceed 255. Unfortunately, when pulling the data from Postgres, recordset gets populated only upto 255 characters for that column. Rest of the data is cut off. Is there anyway to overcome this limit? I have verified that there is no issue with the character limit for that column within Excel.
EDIT:
I cant share the code so here is an example of what my VBA code looks like:
Dim objConnection As Object
Dim objRecordset As Object
Dim sqlstring As String
ConStr= "Driver={PostgreSQL UNICODE};Server=IP
address;Port=5432;Database=myDataBase;Uid=myUsername;Pwd=myPassword;"
sqlstring = 'SELECT columnA, columnB, columnC from tableA'
Set objConnection = CreateObject("ADODB.connection")
Set objRecordset = CreateObject("ADODB.Recordset")
objRecordset.CursorLocation = 3
objConnection.Open ConStr
objRecordset.Open sqlstring, objConnection
For i = 0 To 2
If objRecordset.Fields(i).Value <> " " Or
objRecordset.Fields(i).Value <> Null Or objRecordset.Fields(i).Value
<> "" Then
ActiveCell.Value = objRecordset.Fields(i).Value
ActiveCell.Offset(0, 1).Activate
End If
Next i
In the above example, ColumnB is 1500 characters n length. But in Excel I see only 255 characters. The Recordset itself picks up only 255 characters. I am trying to overcome this limit.
I was able to resolve the issue by editing the connection string. Followed the page: odbc.postgresql.org/docs/config.html and added "B0=5000" in the connection string.
Related
I'm trying to run a list of SQL queries where a condition exists for "code" and the values sit in a range of cells on another sheet (from cells A2 to A385).
I have the code below, however, I get an invalid object name for SQLQueries!$A2:A385
So, I understand the syntax is not correct but I'm struggling to find the correct one regardless of reading numerous articles.
Sub RunSQLQueries()
'Select SQLQueries sheet
Sheets("SQLQueries").Activate
'Initializes variables
Dim cnn As New ADODB.Connection
Dim rst As New ADODB.Recordset
Dim ConnectionString As String
Dim StrQuery As String
'Setup the connection string for accessing MS SQL database
ConnectionString = "Provider=SQLOLEDB; Data Source=HOSTNAME; Initial Catalog=DBNAME; UID=domain\user; Integrated Security=SSPI"
'Opens connection to the database
cnn.Open ConnectionString
'Timeout
cnn.CommandTimeout = 900
'Build SQK queries
StrQuery = "SELECT * FROM table WHERE code IN (SELECT * FROM [SQLQueries!$A2:A385])"
'Performs the queries
rst.Open StrQuery, cnn
'Select Results sheet
Sheets("Results").Activate
'Dumps all the results from the StrQuery into cell A2 of the active sheet
Range("A2").CopyFromRecordset rst
End Sub
The result I'm expecting is for a SQL query to be run using each condition from the range of values with the results being populated on the "Results" sheet from cells A2 down
The query string is literally sent to the database server, and since your sql attempts to refer to an excel list that the server cannot access it returns an error. The server is looking for a table named [SQLQueries!$A2:A385]
To stick with your current plan, you will need to pass the the IN () clause literally or by vba variable that is formatted as such:
IN ( 'item1', 'item2' ...)
Note:You can remove single quotes if the items are numeric
I advise rethinking the plan by either
1) if it is possible to adjust things in the database side: Can you create a new reference table to join to the actual table or create a view that only returns desired rows? Then you would need a job that tweaks the filtering view/ table before running the query. The idea being you would not need to adjust the query each time bc a constant sql string would return the rows you need
Or
2) if the source table has say 100k rows or less, and data is not too wide, just select all the rows into excel in a new sheet, then filter that sheet (add a new column in excel that returns true using vlookup against your reference sheet) or use vlookup on your reference sheet to pull the desired columns
Here's a suggestion:
StrQuery = "SELECT * FROM table WHERE code IN (" & _
InList(Sheets("SQLQueries").Range("A2:A385"),True) & ")"
Function to create a SQL "in" list given a range:
Function InList(rng As Range, quoted As Boolean)
Dim qt, a, r As Long, c As Long, rv As String, v, sep As String
a = rng.Value
qt = IIf(quoted, "'", "")
For r = 1 To UBound(a, 1)
For c = 1 To UBound(a, 2)
v = Trim(a(r, c))
If Len(v) > 0 Then
rv = rv & sep & qt & v & qt
sep = ","
End If
Next c
Next r
InList = rv
End Function
Notes:
Pass False as the second argument if you have numeric values
I wouldn't use this for very large lists
Be very certain you're not at risk from possible SQL injection issues: parameterized queries are always preferable but do not work with "in" lists
I am writing a SQL query that checks excel values against a database. While running my Excel macro, I create worksheet 2 (ws2) and need to run a query which checks if each of the values in column F = table.number.
I know I can use Cells to get a single value and wrap it in a for loop but that takes up too much processing and requires too many SQL extracts. The column in ws2 is called "REFERENCE" and has all the data below it. Ideally, I would like to write the SQL query like:
select * from table where ws2.REFERENCE = table.number
Is there a way to do this?
I suppose you can do the comparison in either Excel or in SQL Server, right. I think one viable solution is to import data from SQ Server, including field names, and do compare this to what you already have in Excel.
Sub Conn2SQL()
Dim cnn1 As New ADODB.Connection
Dim mrs As New ADODB.Recordset
Dim iCols As Integer
Set cnn1 = New ADODB.Connection
cnn1.ConnectionString = "driver={SQL Server};server=MyDBServer;uid=MyuserID;pwd=mypassword;database=MyDB"
cnn1.ConnectionTimeout = 30
cnn1.Open
SQry = "use MyDB select * from TableName"
mrs.Open SQry, cnn1
For iCols = 0 To mrs.Fields.Count - 1
Worksheets("Sheet2").Cells(1, iCols + 1).Value = mrs.Fields(iCols).Name
Next
Sheet2.Range("A2").CopyFromRecordset mrs
mrs.Close
cnn1.Close
End Sub
I have a procedure in VBA that I've been using for some time now to pull the data set from SQL DB.
It always took less then 1 min to populate the template but since few days it's been taking more than 10 mins. Nothing changed in the database, nothing changed in my VBA code either:
It gets stuck on:
Sheets("NEW").Range("A3").CopyFromRecordset rst
I even upgraded from Office 2013 to 2016 but the issue still persists. Any guesses what could be the reason?
My code
Dim cnn As New ADODB.Connection
Dim rst As New ADODB.Recordset
Dim ConnectionString As String
Dim StrQuery As String
Dim Combine As String
Dim ow As Long
Dim ok As Integer
Dim i As Long
ConnectionString = "Provider=SQLOLEDB.1;Password=xxx;Persist Security Info=True;User ID=xxxx;Data Source=xxxx;Use Procedure for Prepare=1;Auto Translate=True;Packet Size=4096;Use Encryption for Data=False;Tag with column collation when possible=False;Initial Catalog=Share_Assignment_Automation"
cnn.Open ConnectionString
cnn.CommandTimeout = 900
StrQuery = "SELECT [Organization],null as [ERP Segment 1],[Effective date of assignment to Team],[Expiration date of assignment to Team],null as [Last Modified By],null as [Last Modified Date],[Territory Type Name],[Forecastable Flag],[Inside Sales Flag] FROM [vw_SHARETeam] with (nolock) "
rst.Open StrQuery, cnn
Sheets("TEAM").Range("A3").CopyFromRecordset rst
rst.Close
I had same issue. My query was 7 seconds but the paste was 60 seconds. (100,000 rows).
The issue was that I was pasting into an excel table which was auto-expanding its range.
I changed my code to disable auto-expand, then paste data, then manually expand the table range. This now takes 15 seconds total.
I've queried an MS-Access database from within VBA, and returned the result set to an array, as follows:
Sub ChartData()
Dim cn As Object
Dim rs As Object
Dim strSql As String
Dim strConnection As String
Set cn = CreateObject("ADODB.Connection")
' Hard code database location and name
strConnection = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=\\server1\myDB.mdb"
' Construct SQL query
strSql = "TRANSFORM Count(Names) AS CountOfNames SELECT Ticker FROM Pricing GROUP BY Ticker PIVOT Source; "
' execute the query, and return the results to the rs object
cn.Open strConnection
Set rs = cn.Execute(strSql)
' Copy the result set to an array
Dim myArr() As Variant
myArr = rs.GetRows
' Close the connection
rs.Close
Set rs = Nothing
cn.Close
Set cn = Nothing
...
End Sub
Next, I'd like to insert one column into myArr, which has dynamic dimensions. I've attempted to use ReDim Preserve for this purpose, but learned that ReDim Preserve will only allow changing the first dimension. For example, the following code results in the Run-time error, Subscript out of range:
Sub ChartData()
...
Dim newRowCount As Integer
Dim newColCount As Integer
newRowCount = UBound(myArr, 2) + 1
newColCount = UBound(myArr, 1) + 2
ReDim Preserve myArr(newColCount, newRowCount) ' Run-time error here
End Sub
Is there an elegant way to work-around this ReDim Preserve limitation to insert a column without wiping the data?
Thanks!
Consider adjusting SQL query at the source as suggested by #TimWilliams in the comments which avoids memory overhead and use of manipulating data objects.
Crosstab queries use an aggregate groupby within its structure which you can easily add scalar values including empty, numeric, or string values (which will be the same across all rows):
TRANSFORM Count(Names) AS CountOfNames
SELECT Ticker, NULL As EmptyColumn, 1 As AllOnesColumn, 'SQL in Excel' As AllStringColumn
FROM Pricing
GROUP BY Ticker
PIVOT Source;
Alternatively, save the crosstab query as a stored query object in Access database and run a typical select statement in Excel ADO connection referencing the stored query. Again, you can add scalar columns as needed:
SELECT storedquery.*, NULL As EmptyColumn, 1 As AllOnesColumn,
'SQL in Excel' As AllStringColumn
FROM storedquery
I have some code to read records from a database. when i read , the recordset variable modifies the table value to its own format.
For Eg:
In Database
Time value is 12345 (Not date & time) but when record set reads it, it
comes as For Eg: 23-06-2012
10:15:23
I just found that the recordset itself stores values in its own format after doing.
Set rs = CmdSqlData.Execute()
So is there any way to define a recordset as String?
Here is the code.
Dim rs As ADODB.RecordSet
Set rs = CmdSqlData.Execute()
Do While (rs.EOF = FALSE And rs.BOF = FALSE)
p = rs.GetRows(1)
cell(1,1) = p(0,0)
Loop
Can anyone please let me know how to read the data as String (as it is in database) so that no change in format will occur.
Note: I can not convert Excel cell format due to other requirements but I want to read everthing as String From Table
If you write
CStr(p(0,0))
To a cell, Excel will convert to the appropriate type for the content, so if p(0,0) is a number, the cell will be numeric.
However, if you write
ActiveSheet.Cells(1, 1) = "'" & p(0, 0)
Cell A1 will contain '2 to view, but can be manipulated as a string. This is left over from the early days of Excel, where to enter a string you had to prefix it with a single quote.
A1
2
=A1=2 FALSE
=A1="2" TRUE