I am trying to connect R to a postgreSQL database. He is what I have been trying in R:
require("RPostgreSQL")
pw<- {
"password"
}
# loads the PostgreSQL driver
drv <- dbDriver("PostgreSQL")
# creates a connection to the postgres database
# note that "con" will be used later in each connection to the database
con <- dbConnect(drv, dbname = "DBname",
host = "localhost", port = 5432,
user = "user", password = pw)
rm(pw) # removes the password
# check for the test_table
dbExistsTable(con, "test_table")
# FALSE >>> Should be true
I cannot figure out why it is not properly connecting to my database. I know that the database is on my computer as I can connect to it in the terminal and with pgAdmin4. Any help is greatly appreciated.
Thanks
I have had better success with the RPostgres package in combination with DBI and I know that RPostgreSQL just released a new version in May after no changes for a while. RPostgres is pretty active
## install.packages("devtools")
#devtools::install_github("RcppCore/Rcpp")
#devtools::install_github("rstats-db/DBI")
#devtools::install_github("rstats-db/RPostgres")
library(RPostgres)
library(DBI)
pw<- {
"password"
}
con <- dbConnect(RPostgres::Postgres()
, host='localhost'
, port='5432'
, dbname='DBname'
, user='user'
, password=pw)
rm(pw) # removes the password
dbExistsTable(con, "test_table")
install.packages("RPostgreSQL")
require("RPostgreSQL")
# this completes installing packages
# now start creating connection
con <- dbConnect(dbDriver("PostgreSQL"),
dbname = "dbname",
host = "localhost",
port = 5432,
user = "db_user",
password = "db_password")
# this completes creating connection
# get all the tables from connection
dbListTables(con)
One issue could be table permission
GRANT ALL PRIVILEGES ON your_table TO user
Replace your_table and user with your own credentials
You can get table from \dt and user from \du
Related
I can connect to a Hive (or LLAP) database using pyhive and I can query the database fixing the server host. Here is a code example:
from pyhive import hive
host_name = "vrt1553.xxx.net"
port = 10000
connection = hive.Connection(
host=host_name,
port=port,
username=user,
kerberos_service_name='hive',
auth='KERBEROS',
)
cursor = connection.cursor()
cursor.execute('show databases')
print(cursor.fetchall())
How could I connect using Zookeeper to get a server name?
You must install the Kazoo package to query Zookeeper and find the host and port of your Hive servers:
import random
from kazoo.client import KazooClient
zk = KazooClient(hosts='vrt1554.xxx.net:2181,vrt1552.xxx.net:2181,vrt1558.xxx.net:2181', read_only=True)
zk.start()
servers = [hiveserver2.split(';')[0].split('=')[1].split(':')
for hiveserver2
in zk.get_children(path='hiveserver2')]
hive_host, hive_port = random.choice(servers)
zk.stop()
print(hive_host, hive_port)
Then just pass hive_host and hive_port to your Connection constructor:
connection = hive.Connection(
host=hive_host,
port=hive_port,
username=user,
kerberos_service_name="hive",
auth="KERBEROS",
)
And query as a standard python sql cursor. Here is using pandas:
df = pd.read_sql(sql_query, connection)
I am trying to create an external table using SQL server 2019 to sybase.
I am already able to create a linked server to sybase using the same driver and login information.
I am able to exec this code with no error:
CREATE EXTERNAL DATA SOURCE external_data_source_name
WITH (
LOCATION = 'odbc://jjjjj.nnnn.iiii.com:xxxxx',
CONNECTION_OPTIONS = 'DRIVER={SQL Anywhere 17};
ServerNode = jjjjj.nnnn.iiii.com:xxxxx;
Database = report;
Port = xxxxx',
CREDENTIAL = [PolyFriend] );
but when I try to create a table using the data source
CREATE EXTERNAL TABLE v_data(
event_id int
) WITH (
LOCATION='report.dbo.v_data',
DATA_SOURCE=external_data_source_name
);
I get this error:
105082;Generic ODBC error: [SAP][ODBC Driver][SQL Anywhere]Database
server not found.
you need to specify the Host & ServerName & DatabaseName properties (for SQL Anywhere) in the connection_options
CREATE EXTERNAL DATA SOURCE external_data_source_name
WITH (
LOCATION = 'odbc://jjjjj.nnnn.iiii.com:xxxxx',
CONNECTION_OPTIONS = 'DRIVER={SQL Anywhere 17};
Host=jjjjj.nnnn.iiii.com:xxxxx;
ServerName=xyzsqlanywhereservername;
DatabaseName=report;',
CREDENTIAL = [PolyFriend] );
Host == machinename:port, the machinename where SQLAnywhere resides and port most likely the default 2638 where the SQLAnywhere service is listening for connections.
ServerName == the name of the SQLAnywhere server/service which hosts the database (connect to the SQLAnywhere db and execute select ##servername).
I want to connect R to SQL Server on my MAC (El Capitan), I can do it very easy in Python, but in R I can't do it.
In Python it is easy as:
import pymssql
pymssql.connect(server = 'CHWN-DSX-DB02', user = 'XXXX',password ='XXXX',database = 'Info')
For R, I tried with RODBC library, but didn't work, I think the problem is the "Driver":
driver.name <- "SQL Server"
db.name <- "Info"
host.name <- "CHWN-DSX-DB02"
port <-""
server.name <-"XXX"
pwd <- "XXX"
# Use a full connection string to connect to a SAMPLE database
con.text <- paste("DRIVER=",RMySQL::MySQL(),
";Database=",db.name,
";Server=",host.name,
";Port=",port,
";PROTOCOL=TCPIP",
";UID=", server.name,
";PWD=",pwd,sep="")
con1 <- odbcDriverConnect(con.text)
This code in R never ends, and when I stop it, I have this warning:
Warning messages:
1: In odbcDriverConnect(con.text) :
[RODBC] ERROR: state 00000, code 0, message [iODBC][Driver Manager]dlopen(SQL Server, 6): image not found
There are several issues with your current setup:
General vs. Specific APIs: In Python you are using a specific SQL Server API: pymssql. In fair comparison to your R attempt, you should be using the ODBC API: pyodbc (to compare with RODBC). Remember there are multiple ways to connect to backend databases or data stores from client applications either by specific APIs or generalized APIs (ODBC, OLEDB, JDBC, etc.).
Required Drivers: To use any ODBC library be it Python, R, or other language, you need to have an installed ODBC driver on your client machine. Download such drivers before attempting connection. Nearly every RDBMS or data store maintains (usually free to download) ODBC drivers for Windows, Mac, and Linux OS's including SQL Server: 2013 or 2017.
Mixing R Libraries: In R, most APIs follow the DBI standard including ROracle, RJDBC, odbc, RMySQL, RPostgreSQL, RSQLite. Unfortunately, RODBC does not follow this standard. Your attempted connection appears to be an attempted DBI connection using the RMySQL::MySQL() object which even in DBI is not part of an ODBC connection string.
Note even though both require underlying ODBC drivers (see #2), RODBC is a different library and implementation than odbc. Additionally, do not conflate specific APIs such as RMySQL with an attempted general SQL Server ODBC. In fact, it is unclear why you use RMySQL when you have a variable for driver.name. Also such a value, 'SQL Server' is the Windows ODBC driver name and not macOS or Linux names. See below R ODBC connections:
RODBC
driver.name <- "ODBC Driver 13 for SQL Server" # REQUIRES DOWNLOAD
driver.name <- "ODBC Driver 17 for SQL Server" # REQUIRES DOWNLOAD
...
con.text <- paste0("driver=", driver.name,
";database=", db.name,
";server=", host.name,
";port=", port,
";protocol=TCPIP",
";UID=", user.name,
";PWD=", pwd)
conn <- odbcDriverConnect(con.text)
odbc
conn <- dbConnect(odbc::odbc(), driver = driver.name,
server = host.name, port = port, database = db.name
uid = user.name, pwd = pwd)
# ALTERNATIVE:
conn <- dbConnect(odbc::odbc(), .connection_string = con.text)
Using osm2pgsql I have loaded a data from .osm.pbf file to the postgreSQL database. Now I want to access the "planet_osm_polygons" table from the postgres database. My R code is as below:
con <- dbConnect("PostgreSQL", dbname = "osmdb",
host = "localhost", port = 5432,
user = "postgres", password = "root")
if(dbExistsTable(con, "planet_osm_polygon")){
df <- dbReadTable(con, "planet_osm_polygon")
head(df)
}else{
print("Table not found in the database!")
}
I have checked dbExistsTable() and it returns me TRUE for the indicated table name. Which means the table exists but for dbReadTable() it throws me this:
Warning message:
In postgresqlExecStatement(conn, statement, ...) :
RS-DBI driver warning: (unrecognized PostgreSQL field type geometry (id:80871) in column 68)
I have searched allot on many blogs but couldn't any solution of this problem. Kindly help me in solving this simple issue. Thank you very much for your time.
I want SQL Server permissions to conform to the ID I specify in the connection string, NOT those of whoever happens to be running the PowerShell script (usually me).
For example, I have a table called “MyTable” which I can update but which “TestIDRestricted” cannot.
I log onto the Microsoft SQL Server Management Studio as “TestIDRestricted” and run the following query
select MyField
from MyTable
where ID = 2
and I see it returns "4". I now try to update:
update MyTable
set MyField = 5
where ID = 2
As expected, I get the following failure result:
Msg 229, Level 14, State 5, Line 1
The UPDATE permission was denied on the object 'MyTable', database 'mydb', schema 'dbo'.
Next I run the following UpdateProblem.ps1 PowerShell script:
$ConnectionString = "Server=MyServer\MSSQL2012;Database=mydb;User Id= TestIDRestricted;Password=secretpasswd;"
$ConnectionString = $ConnectionString + "Trusted_Connection = yes;Persist Security Info=False"
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = $ConnectionString
$sqlConnection.Open()
$sqlCommand = $sqlConnection.CreateCommand()
$updateSQL ="update MyTable set MyField = 5 where ID = 2"
$sqlCommand.CommandText = $updateSQL
write-host $updateSQL
#following command should fail
$sqlCommand.ExecuteNonQuery()
$sqlConnection.close()
I expect this update to fail because I am NOT specifying integrated security in the connection string. But when I log back onto the Microsoft SQL Server Management Studio as “TestIDRestricted” and again run the following query
select MyField
from MyTable
where ID = 2
I see it returns "5". So the update succeeded even though I expect it to fail. From what I can tell, the reason it succeeded is because the SQL permissions are based on my ID using integrated security instead of the security associated with the ID I use in the connection string.
How can I get the script to use the permissions associated with the ID I specify in the connection string instead of using my ID’s permissions that seem to follow along with integrated security?
You're setting Trusted_Connection to TRUE; this almost certainly means you're connecting with the credentials of the current user, not those specified in the connection string.
Try:
$ConnectionString = "Server=MyServer\MSSQL2012;Database=mydb;User Id= TestIDRestricted;Password=secretpasswd;"
$ConnectionString = $ConnectionString + "Trusted_Connection = no;Persist Security Info=False"
You could probably just comment out the second line:
$ConnectionString = "Server=MyServer\MSSQL2012;Database=mydb;User Id= TestIDRestricted;Password=secretpasswd;"
' $ConnectionString = $ConnectionString + "Trusted_Connection = yes;Persist Security Info=False"
(Is this VB?)