I have a view in a PostgreSQL database. Executing the view in pgAdmin is very fast (10,000 records). But executing "Select * From myView;" from VBA is VERY slow. I connect to the database using ADO and ODBC. Can anyone give me a hint on how to speed up things?
A small example:
Sub TEST()
Dim CN As New ADODB.Connection
Dim RS As New ADODB.Recordset
Dim Data As Variant
Dim SQL As String
Dim ConStr As String
ConStr = "Server=11.22.33.44;" & _
"DSN=PostgreSQL35W 32bit;" & _
"UID=xxx;" & _
"PWD=xxx;" & _
"Database=xxx;" _ &
"Port=5432;" & _
"CommandTimeout=12"
SQL = "Select * From myView;"
Debug.Print Now()
CN.ConnectionString = ConStr
CN.Open
Debug.Print Now()
RS.ActiveConnection = CN
RS.Source = SQL
RS.CursorType = adOpenStatic
RS.LockType = adLockReadOnly
Debug.Print Now()
RS.Open
Debug.Print Now()
Data = RS.GetRows
Debug.Print Now()
RS.Close
CN.Close
Set RS = Nothing
Set CN = Nothing
End Sub
This gives output:
10/08/2016 16:14:26
10/08/2016 16:14:26
10/08/2016 16:14:26
10/08/2016 16:14:38
10/08/2016 16:18:50
That is "RS.Open" takes 00:00:12, and "Data = RS.GetRows" 00:04:12.
In pgAdmin it takes less than a second to show all 10,000 records.
I found out to use OLE DB. And it is FAST!
Downloaded PgOleDb: https://www.connectionstrings.com/pgoledb/info-and-download
Copied the two DLLs to C:\Windows\SysWOW64.
Ran "Regsvr32 PGOLEDB.DLL".
Connection string: https://www.connectionstrings.com/pgoledb
Provider=PostgreSQL OLE DB Provider;Data Source=myServerAddress;
location=myDataBase;User ID=myUsername;password=myPassword;
The command "timeout=1000;" does not function.
Related
I am trying to do a Select statement from an SQL server using an ADODB connection, within Microsoft Access VBA. The select statement is very complex with calculations and would work better to keep that to try and turn in to a connection with the DB.
I can append the data row by row, but am wondering how I could append as simple as possible as a block of data to save time/vba effort. (Lots of data!)
In order to simplify the example, I have put a basic SQL statement, but if you could review this and let me know if there is a way to append as a recordset block or what would be a better option... The code below works for pasting as a block into excel, and I would think I only need to change that commented out line's method to append in Access...
Option Private Module
Private Const DataServer = "GLSSQLMADPS2"
Private Const sqlconstring = "Driver={SQL Server};Server=" & DataServer & "; Database=PERF_MGMT_BWRSRV_PROD; Trusted_Connection=yes; Connect Timeout = 600"
Sub M2DSQLPull()
Dim cn As ADODB.Connection
Dim rs As ADODB.Recordset
Dim sql As String
Dim sql As String
sql = "SELECT " & _
"Employee_Token_NR, " & _
"Null as 'Sum_Month', " & _
"1000 AS 'Data_ID', " & _
"DATEADD(m, DATEDIFF(m, 0, MIN(CALL_TS)), 0) AS 'Metric_1' " & _
"FROM PMBSED_PhoneEvaluations " & _
"GROUP BY Employee_Token_NR "
Set cn = New ADODB.Connection
cn.CommandTimeout = 120
cn.Open sqlconstring
Set rs = New ADODB.Recordset
rs.Open SqlString(i), cn, adOpenStatic, adLockReadOnly
'How I would paste in an excel spreadsheet, how do I block paste in an access database?
'ThisWorkbook.Worksheets("RawData").Cells(RowNumber, 1).CopyFromRecordset rs
RowNumber = RowNumber + rs.RecordCount
rs.Close
cn.Close
End Sub
Thank you for any assistance!
-Robert
I am trying to connect to connect to Netezza using VBA. I have enabled the following:
Microsoft Excel 15.0 Object Library
Microsoft Office 15.0 Object Library
Microsoft ActiveX Data Objects 6.1 Library
Visual Basic for Applications
Here is my code:
Sub NZConn()
Dim cmd As New ADODB.Command
Dim rs As New ADODB.Recordset
Dim x As Variant
Set cmd = New ADODB.Command
Set RS = New ADODB.Recordset
cmd.ActiveConnection = "Driver={Netezza " & _
"ODBC};servername=servername;port=####;database=database;" & _
"username=username;password=password;"
cmd.ActiveConnection.CursorLocation = adUseClient
cmd.CommandTimeout = 120
cmd.CommandType = adCmdText
x = "Write Query here"
cmd.CommandText = x
Set rs = cmd.Execute
Sheet1.Range("A1").CopyFromRecordset rs
cmd.ActiveConnection.Close
End Sub
I can get the code to run without throwing back an error, but there is nothing that is pasted from the record set, which leads me to believe that is may have something to do with the structure of the connection string.
I have the server, user id, password, database, port, and driver.
Would I need to establish / open an ActiveConnection first?
I was able to figure out the issue on my own. I found that there is a command line builder in the 'Tools' tab in Aginity, which helped specify the exact connection string I needed to connect to Netezza. Once I had this connection string, I was getting an 'architecture mismatch' error. After downloading the 32-bit ODBC drivers for Netezza, the methodology worked perfectly. Here is the updated code below:
Dim cn As ADODB.Connection
Dim rs As ADODB.Recordset
Dim iCols As Integer
Dim DB As String, User As String, PW As String, ConnectionString As String
Dim Server As String, Query As String
Dim SQLTable As Worksheet
Set cn = New ADODB.Connection
Set rs = New ADODB.Recordset
Set SQLTable = Sheet1
Server = SQLTable.Cells(2,3).Value
User = SQLTable.Cells(2,4).Value
PW = SQLTable.Cells(2,5).Value
DB = SQLTable.Cells(2,6).Value
Query = SQLTable.Cells(2,7).Value
ConnectionString = "Driver={NetezzaSQL};" & _
"server=" & Server & ";" & _
"UserName=" & User & ";" & _
"Password=" & PW & ";" & _
"Database=" & DB & ";" & _
"Query Timeout=120"
cn.Open (ConnectionString)
rs.Open (Query), cn
For iCols = 0 To RS.Fields.count - 1
Worksheets("Sheet2").Cells(1, iCols + 1).Value = rs.Fields(iCols).Name
Next
Worksheets("Sheet2").Cells(2, "A").CopyFromRecordset rs
rs.Close
cn.Close
NB:
"IBM NETEZZA ODBC DRIVER – 32 BIT" is what I downloaded
"ODBC-DRIVER-FOR-NETEZZA-7-X86" is what showed up in my software center to install
"Name: NetezzaSQL ; Version: 7.00.04.41188 ; Company: www.ibm.com ; File: NSQLODBC.DLL" is what is shown now in my 32-bit 'ODBC Data Source Administrator' window
I think your connection string is ok, and yes you should need to open a connection first.
Like this:
AccessConnect = "Driver={Netezza " & _
"ODBC};servername=servername;port=####;database=database;" & _
"username=username;password=password;"
Dim Conn1 As New adodb.Connection
Conn1.ConnectionString = AccessConnect
Conn1.Open
then it would be
Set RS = Conn1.Execute(x) 'where x is your query
I have an Access DataBase and a form in Excel VBA. All the data I input into the DB is input through the VBA form.
This DB contains all the benefits cards we already received this year in the company. But the same employee can ask for the card twice or more, so we'll have more than one record on the DB for him.
What I need is when the number of records is greater than one, the SQL statement result should appear in a Excel report.
I use the SELECT (*) COUNT statement to know when there is more than one record that is compatible with the search criterion. But I can't make the result appear in an Excel file.
Here is my code:
Public Function Relatorio()
Dim sql As String
Dim cn As ADODB.Connection
Dim rs As ADODB.Recordset
Dim rel As String
Set cn = New ADODB.Connection
cn.ConnectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" & enderecoDB & ";Jet OLEDB:Database"
cn.Open
Set rs = New ADODB.Recordset
sql = "INSERT INTO OPENROWSET('Microsoft.ACE.OLEDB.12.0', 'Excel 12.0;Database=" & enderecoDB & ";', 'SELECT * FROM [Planilha1$]') SELECT * FROM controle WHERE BP = " & controlectform.nmbpbox.Value & ";"
rs.Open sql, cn
End Function
When I run this code it gives me a message saying something like:
Can't locate the OPENROWSET Table exit
I'm not able to install new programs, so I need to do this using only Excel VBA and the Access DB.
How can I make this work?
I don't believe Access supports the OPENROWSET, dynamic table you're working with there. I have a lot of old projects that do this though, so here's my method
Public Function Relatorio()
Dim sql As String
Dim cn As ADODB.Connection
Dim rs As ADODB.Recordset
Dim rel As String
Set cn = New ADODB.Connection
cn.ConnectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" & enderecoDB & ";Jet OLEDB:Database"
cn.Open
Set rs = New ADODB.Recordset
dim path_To_XLSX
dim name_of_sheet
path_To_XLSX = "c:\temp\output.xlsx"
name_of_sheet = "Planilha1"
sql = sql = "SELECT * INTO [Excel 12.0;Database=" & path_To_XLSX & "]." & name_of_sheet & " FROM controle WHERE BP = '" & controlectform.nmbpbox.Value & "';"
rs.Open sql, cn
'If this application is in an unsecure environment, use the following code instead! This is to prevent a SQL injection, security concern here.
'As it is an Access Database, this is likely overkill for this project
'Create Command Object.
Set Cmd1 = New ADODB.Command
Cmd1.ActiveConnection = cn
cmd1.CommandText = "SELECT * FROM controle INTO [Excel 12.0;Database=" & path_To_XLSX & "]." & name_of_sheet & " WHERE BP = ?"
' Create Parameter Object.
Set Param1 = Cmd1.CreateParameter(, adInteger, adParamInput, 5) 'use adVarchar for strings(versus adInteger), https://www.w3schools.com/asp/met_comm_createparameter.asp
Param1.Value = controlectform.nmbpbox.Value
Cmd1.Parameters.Append Param1
Set Param1 = Nothing
Set Rs = Cmd1.Execute()
End Function
I had this challenge so many years ago that I cant remember but this link ring the bell. check if it help.
https://stackoverflow.com/a/28889774/382588
try { connw.Open(); OleDbCommand command; command = new OleDbCommand( "Update Deliveries " + "SET Deliveries.EmployeeID = ?, Deliveries.FIN = ?, Deliveries.TodaysOrders = ? , connw); command.Parameters.Add(new OleDbParameter("#EMPID", Convert.ToDecimal(empsplitIt[1]))); command.Parameters.Add(new OleDbParameter("#FIN", truckSplit[1].ToString())); command.Parameters.Add(new OleDbParameter("#TodaysOrder", "R")); catchReturnedRows = command.ExecuteNonQuery();//Commit connw.Close(); } catch (OleDbException exception) { MessageBox.Show(exception.Message, "OleDb Exception"); }
you can use this, to print the actual SQL.
Private Sub Command2_Click()
Dim db As Database
Dim qr As QueryDef
Set db = CurrentDb
For Each qr In db.QueryDefs
TextOut (qr.Name)
TextOut (qr.SQL)
TextOut (String(100, "-"))
Next
End Sub
Public Sub TextOut(OutputString As String)
Dim fh As Long
fh = FreeFile
Open "C:\Users\rs17746\Desktop\Text_Files\sample.txt" For Append As fh
Print #fh, OutputString
Close fh
End Sub
Here is one more version for you. This will export the results of each query, each to a separate text file.
Private Sub Command0_Click()
Dim qdf As QueryDef
Dim strFileName As String
For Each qdf In CurrentDb.QueryDefs
If Left(qdf.Name, 1) <> "~" Then
'you need to figure out TransferText command. Maybe
'you won't be lazy and expect people to read it to
'you and tutor you on how it works.
strFileName = qdf.Name
'Docmd.TransferText ....
DoCmd.TransferText transferType:=acExportDelim, TableName:=strFileName, FileName:="C:\test\" & strFileName & ".txt", hasfieldnames:=True
End If
Next qdf
MsgBox "Done"
End Sub
I have an sql database and I am able to connect with excel spreadsheet. But when I update the table from excel directly it's not updating the database and once I click refresh all the entered data is no longer in the excel table
Is it possible to update sql database from excel without using any queries?
There are many ways to do this. I'd recommend something like this, to push data from Excel to SQL Server.
Sub ButtonClick()
'TRUSTED CONNECTION
On Error GoTo errH
Dim con As New ADODB.Connection
Dim rs As New ADODB.Recordset
Dim strPath As String
Dim intImportRow As Integer
Dim strFirstName, strLastName As String
Dim server, username, password, table, database As String
With Sheets("Sheet1")
server = .TextBox1.Text
table = .TextBox4.Text
database = .TextBox5.Text
If con.State <> 1 Then
con.Open "Provider=SQLOLEDB;Data Source=" & server & ";Initial Catalog=" & database & ";Integrated Security=SSPI;"
'con.Open
End If
'this is the TRUSTED connection string
Set rs.ActiveConnection = con
'delete all records first if checkbox checked
If .CheckBox1 Then
con.Execute "delete from tbl_demo"
End If
'set first row with records to import
'you could also just loop thru a range if you want.
intImportRow = 10
Do Until .Cells(intImportRow, 1) = ""
strFirstName = .Cells(intImportRow, 1)
strLastName = .Cells(intImportRow, 2)
'insert row into database
con.Execute "insert into tbl_demo (firstname, lastname) values ('" & strFirstName & "', '" & strLastName & "')"
intImportRow = intImportRow + 1
Loop
MsgBox "Done importing", vbInformation
con.Close
Set con = Nothing
End With
Exit Sub
errH:
MsgBox Err.Description
End Sub
You can also try this, which uses a Where Clause.
Sub InsertInto()
'Declare some variables
Dim cnn As adodb.Connection
Dim cmd As adodb.Command
Dim strSQL As String
'Create a new Connection object
Set cnn = New adodb.Connection
'Set the connection string
cnn.ConnectionString = "Provider=SQLOLEDB.1;Integrated Security=SSPI;Persist Security Info=True;Initial Catalog=Northwind;Data Source=Excel-PC\SQLEXPRESS"
'cnn.ConnectionString = "DRIVER=SQL Server;SERVER=Excel-PC\SQLEXPRESS;DATABASE=Northwind;Trusted_Connection=Yes"
'Create a new Command object
Set cmd = New adodb.Command
'Open the Connection to the database
cnn.Open
'Associate the command with the connection
cmd.ActiveConnection = cnn
'Tell the Command we are giving it a bit of SQL to run, not a stored procedure
cmd.CommandType = adCmdText
'Create the SQL
strSQL = "UPDATE TBL SET JOIN_DT = '2013-01-22' WHERE EMPID = 2"
'Pass the SQL to the Command object
cmd.CommandText = strSQL
'Execute the bit of SQL to update the database
cmd.Execute
'Close the connection again
cnn.Close
'Remove the objects
Set cmd = Nothing
Set cnn = Nothing
End Sub
Yes, you can directly via VBA or with other tools.
via VBA (via qry)
via SSIS (https://www.simple-talk.com/sql/ssis/moving-data-from-excel-to-sql-server-10-steps-to-follow/)
via managament studio (https://www.mssqltips.com/sqlservertutorial/203/simple-way-to-import-data-into-sql-server/)
via MS ACCESS (with ODBC connection to server)
...
I am working on an Access application which needs to establish a connection to an External database (Netezza database). I am currently using VBA code with ADODB objects to connect to the server.
Whenever I execute the code, 'Timeout expired' error occurs.
I have tried to reset the Timecounter to 180 seconds. Still the error is not resolved.
This is the code I've been using:
Private Sub CONNECT_Click()
Dim openSQL As ADODB.Connection
Set openSQL = New ADODB.Connection
openSQL.ConnectionTimeout = 180
openSQL.Open "odbc;servername=xxx;dsn=xxx;database=xxx;uid=xxx;pwd=xxx;port=xxx"
openSQL.Close
End Sub
I've also tried the below code:
Private Sub Modify_Click()
Dim objConnection As ADODB.Connection
Dim objRecordSet As ADODB.Recordset
Dim strConnectionString As String
Set objConnection = New ADODB.Connection
Set objRecordSet = New ADODB.Recordset
'Define the Odbc connection string.
strConnectionString = "odbc;servername=xxx;dsn=xxx;database=xxx;uid=xxx;pwd=xxx;port=xx"
'Instantiate the Connection object and open a database connection.
'var cnn
objConnection.Open strConnectionString
'objConnection.Open "dsn=xxx;User ID=xxx;Password=xxx"
Dim strSQL As String
'Define SQL SELECT statement.
strSQL = "INSERT INTO Table_1 (col1,col2,col3, col4) VALUES ('" & Form1.col1 & "', '" & Form1.col2 & "', '" & Form1.col3 & "', '" & Form1.col4 & "');"
'Use the Execute method to issue a SQL query to database.
cnn.Execute strSQL
End Sub
Thanks in advance!