Here is my code:
library('RODBC')
db.handle <- odbcDriverConnect('driver={SQL Server Native Client 11.0};server=server_name;database = db_name;trusted_connection=yes')
sql_table <- 'db_name.table_name'
sqlDrop(db.handle, sql_table, errors = TRUE)
sqlSave(db.handle,df_small,tablename = sql_table,safer=FALSE,append=TRUE,
rownames = FALSE)
close(db.handle)
When I execute line:
sqlDrop(db.handle, sql_table, errors = TRUE)
I get error message:
Error in odbcTableExists(channel, sqtable, abort = errors) :
‘db_name.table_name’: table not found on channel
When I execute line:
sqlSave(db.handle,df_small,tablename = sql_table,safer=FALSE,append=TRUE,
rownames = FALSE)
I get the following error message:
Error in sqlSave(db.handle, df_small, tablename = sql_table, safer =
FALSE, : 42S01 2714 [Microsoft][SQL Server Native Client 11.0][SQL
Server]
There is already an object named 'table_name' in the database.
[RODBC] ERROR: Could not SQLExecDirect 'CREATE TABLE
db_name.table_name ("State_rename" varchar(255), "CoverageType"
varchar(255))'
I execute code consecutively and cannot understand how both error messages can be true.
Consider removing the schema from the sqltable variable which SQL Server uses with period qualifier. Specifically, change db_name.table_name to table_name. Reason you do not need this schema is your connection handle already specifies the database. With this connection, you cannot access other database schemas in specified server.
library('RODBC')
db.handle <- odbcDriverConnect(paste0('driver={SQL Server Native Client 11.0};',
'server=server_name;database=db_name;trusted_connection=yes'))
sql_table <- 'table_name'
sqlDrop(db.handle, sql_table, errors = TRUE)
sqlSave(db.handle, df_small, tablename = sql_table, safer = FALSE,
append = TRUE, rownames = FALSE)
close(db.handle)
By the way, you can simply use append=FALSE which will overwrite table (first dropping it and then re-creating it) with no need to call sqlDrop:
sqlSave(db.handle, df_small, tablename = sql_table, safer = FALSE,
append = FALSE, rownames = FALSE)
Related
I am trying to connect to Azure SQL using Service Principle to create views, but it says
com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user ClientConnectionId: XXXXX-XXXX-XXXX
However, with the same SPN I was able to connect and create tables, read tables.
import adal
resource_app_id_url = "https://database.windows.net/"
service_principal_id = dbutils.secrets.get(scope = "XX", key = "XXX")
service_principal_secret = dbutils.secrets.get(scope = "XX", key = "spn-XXXX")
tenant_id = dbutils.secrets.get(scope = "XX", key = "xxId")
authority = "https://login.windows.net/" + tenant_id
azure_sql_url = "jdbc:sqlserver://xxxxxxx.windows.net"
database_name = "testDatabase"
encrypt = "true"
host_name_in_certificate = "*.database.windows.net"
context = adal.AuthenticationContext(authority)
token = context.acquire_token_with_client_credentials(resource_app_id_url, service_principal_id, service_principal_secret)
access_token = token["accessToken"]
using above code I am able to create and read tables. There is a requirement to create views so I am using sql_driver_manager to connect to Azure SQL
properties = spark._sc._gateway.jvm.java.util.Properties()
properties.setProperty("accessToken", access_token)
sql_driver_manager = spark._sc._gateway.jvm.java.sql.DriverManager
sql_con = sql_driver_manager.getConnection(azure_sql_url, properties)
query = """
create or alter view test_view as select * from dbo.test_table
"""
stmt = sql_con.createStatement()
stmt.executeUpdate(query)
stmt.close()
this is resulting in an error:
Py4JJavaError: An error occurred while calling
z:java.sql.DriverManager.getConnection. :
com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user token-identified principal. ClientConnectionId:
If I try the same with username and password instead of token, it works but I just need to use spn token for authenticating.
Working code:
sql_driver_manager = spark._sc._gateway.jvm.java.sql.DriverManager
sql_con = sql_driver_manager.getConnection(azure_sql_url, username, password)
query = """
create or alter view test_view as select * from dbo.test_table
"""
stmt = sql_con.createStatement()
stmt.executeUpdate(query)
stmt.close()
What is that I am missing, can someone help me understand the issue. Thanks.
You can not use that method since it was built to execute an update statement and return indexes.
See documentation. Use the prepare() and execute() methods.
https://learn.microsoft.com/mt-mt/sql/connect/jdbc/reference/executeupdate-method-java-lang-string?view=azuresqldb-current
#
# 5 - Upsert from stage to active table.
# Grab driver manager conn, exec sp
#
driver_manager = spark._sc._gateway.jvm.java.sql.DriverManager
connection = driver_manager.getConnection(url, user_name, password)
connection.prepareCall("exec [stage].[UpsertFactInternetSales]").execute()
connection.close()
This is sample code from an article I wrote. It calls a stored procedure to execute an UPSERT. However, any DML or DDL will work as long as it does not return a result set.
I am trying to connect to a SQL Server but somehow i'm getting the below error when trying to connect to db from databricks using Python:
Error:java.lang.ClassNotFoundException: com.microsoft.sqlserver.jdbd.SQLServerDriver
My connection code is the next one:
jdbcHostname = "hostname"
jdbcDatabase = "databasename"
jdbcPort = port
username = 'username'
password = 'password'
jdbcUrl = "jdbc:sqlserver://{0}:{1};database={2}".format(jdbcHostname, jdbcPort, jdbcDatabase)
connectionProperties = {
"user" : username,
"password" : password,
"driver" : "com.microsoft.sqlserver.jdbd.SQLServerDriver"
}
The last code works, but when I try to execute a query I got the mentioned error coming out from the second code line from the next block:
pushdown_query = "select * from table"
df = spark.read.jdbc(url=jdbcUrl, table=pushdown_query, properties=connectionProperties)
display(df)
I tried to install different connectors but I have not been lucky with it, could you help me?
I'm trying to fetch data which includes some German word with umlaut characters. following the bellow structure everything is fine in windows machine :
Sys.setlocale('LC_ALL','C')
library(RMySQL)
conn <- dbConnect(MySQL(), user = "user", dbname = "database",
host = "host", password = "pass")
sql.query <- paste0("some query")
df <- dbSendQuery(conn, sql.query)
names <- fetch(df, -1)
dbDisconnect(conn)
As an example I have :
names[1230]
[1] "Strübbel"
What should I change in order to get the same result in Linux Ubuntu ?
the query will run without problem, but the result is :
names[1230]
[1] "Str\374bbel"
I have checked This solution, but when I put the 'set character set "utf8"' inside of query I'm getting the following error :
df <- dbSendQuery(conn, sql.query, 'set character set "utf8"')
names <- fetch(df, -1)
Error in .local(conn, statement, ...) :
unused argument ("set character set \"utf8\"")
I should mention the encoding for the result is unknown :
Encoding(names[1230])
[1] "unknown"
and doing the :
Encoding(names[1230]) <- "UTF-8"
names[1230]
[1] "Str<fc>bbel"
does not solve the problem !
Instead of :
Sys.setlocale('LC_ALL','C')
You have to use :
Sys.setlocale('LC_ALL','en_US.UTF-8')
and in the sql query :
library(RMySQL)
conn <- dbConnect(MySQL(), user = "user", dbname = "database",
host = "host", password = "pass")
sql.query <- paste0("some query")
dbSendQuery(conn,'set character set "utf8"')
df <- dbSendQuery(conn, sql.query)
names <- fetch(df, -1)
dbDisconnect(conn)
Not sure if this solution will help you but you could try such approach:
con <- dbConnect(MySQL(), user = "user", dbname = "database",
host = "host", password = "pass", encoding = "ISO-8859-1")
If this encoding doesn't work then try "brute force" with different variants
On local PC this works.
$HmsDBuser = 'test';
$HmsDBpassword = 'password';
$HmsDBserver = 'Developer,1433';
$HmsDBdatabase = 'DBNAME';
$this->db = new PDO ("sqlsrv:Server=$HmsDBserver;Database=$HmsDBdatabase","$HmsDBuser","$HmsDBpassword", array(PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION));
My Slim Framework is on Server A: 121.55.0.25
My database is on another Server B: 121.55.0.21
$HmsDBuser = 'test';
$HmsDBpassword = 'password';
$HmsDBserver = '121.55.0.21\MYSERVER\MSSQLSERVER,1433';
$HmsDBdatabase = 'DBNAME';
$this->db = new PDO ("sqlsrv:Server=$HmsDBserver;Database=$HmsDBdatabase","$HmsDBuser","$HmsDBpassword", array(PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION));
and after I connect to the database, I call function with path "product" in Slim framework.
ERROR got on Console :
angular.js:8619 GET http://121.55.0.25/product-manager_servertest/api/v1/products 404 (Not Found)
Try it like this:
$HmsDBserver = '121.55.0.21\\MYSERVER\\MSSQLSERVER,1433';
Notes:
Escaped backslashes.
No space before port.
If it doesn't work, then use just the server name, like this:
$HmsDBserver = '<server-name>,1433';
Normally to query a sql-server database from R, I'd use:
library(RODBC)
con <- odbcConnect(dsn = "ESTUDIOS", uid = "estudios", pwd = "yyyy")
sql_trx <- "SELECT [Fecha], [IDServicio]
FROM [ESTUDIOS].[dbo].[TRX] where MONTH(Fecha) = MONTH('2016-08-01') and YEAR(Fecha) = YEAR('2016-08-01');"
trx.server <- sqlQuery(channel = con, sql_trx)
odbcClose(con)
But when the table of the database is too big, I could the use the libraries: ff and ETLUtils.
So, the normal thing to do must be:
library(RODBC)
library(ff)
library(ETLUtils)
sql2_trx <- read.odbc.ffdf(query = sql_trx, odbcConnect.args = list(con))
But this doesn't give me the desired result, instead this returned the following error.
1: In RODBC::odbcDriverConnect("DSN=11") :
[RODBC] ERROR: state IM002, code 0, message [Microsoft][Administrador de controladores ODBC] No se encuentra el nombre del origen de datos y no se especificó ningún controlador predeterminado
2: In RODBC::odbcDriverConnect("DSN=11") : ODBC connection failed
Can you point out what is wrong with the use of read.odbc.ffdf ?
Currently you are passing what seems to be the previous RODBC connection object, con, into read.odbc.ffdf() but remember the method is attempting to create an ODBC connection and call a query. The R docs mention the proper assignment of odbcConnect.args:
odbcConnect.args a list of arguments to pass to ODBC's odbcConnect
(like dsn, uid, pwd)
Consider passing your original DSN and credentials like you did in regular RODBC connection:
sql2_trx <- read.odbc.ffdf(query = sql_trx, odbcConnect.args = list(dsn = "ESTUDIOS", uid = "estudios", pwd = "yyyy"))