Query data from Hana database with R, dealing with quotes - sql

I use R to read data from a Hana database. Some of the table names include backslashes, which forces me to use quotation marks. I cannot read these tables using R. Let me show you an example ...
This SQL works in Hana:
SELECT COUNT(*) FROM P3O."/BBB/BBB";
When I try to use the same code to read the data with R from the Hana database I get these errors:
library("RODBC")
channel <- odbcConnect("xxx",uid="xxx",pwd="xxx")
query <-
paste("'","SELECT COUNT(*) FROM P30.", "\"/BBB/BBB\"","'",sep="")
RAW_dataHana <- sqlQuery(channel, query)
close(channel)
I get following errors:
Syntax error or access violation;257 sql syntax error: incorrect
syntax near \"SELECT COUNT() FROM ... [2] "[RODBC] ERROR: Could not
SQLExecDirect ''SELECT COUNT() FROM P30.\"/BBB/BBB\"''"
I think it has something to do with the quotation, but when I check the code with this, I think I get the correct query:
x = paste("'","SELECT COUNT(*) FROM P30.", "\"/BBB/BBB\"", "'",sep="")
cat(x)
> cat(x)
'SELECT COUNT(*) FROM P30."/BBB/BBB"'

Just got now to check my RODBC test code...
The easiest way to handle the quoting is to enclose the query string in single quotes, e.g.:
sales_fact<-sqlQuery (ch, 'SELECT TOP 200 "ORDERID", "VAR_INDICATOR",
sum("ORDER_CNT") AS "ORDER_CNT",
sum("VARIANCE") AS "VARIANCE",
sum("VARIANCE_PCT") AS "VARIANCE_PCT",
sum("BUDGET") AS "BUDGET",
sum("ACTUAL") AS "ACTUAL"
FROM "_SYS_BIC"."test/ODERS_CV"
GROUP BY "ORDERID", "VAR_INDICATOR"')
This also works with paste():
queryText <- 'SELECT TOP 200 "ORDERID", "VAR_INDICATOR",
sum("ORDER_CNT") AS "ORDER_CNT",...'
queryText <- paste(sep = '', queryText, ' "_SYS_BIC"."test/ODERS_CV"
GROUP BY "ORDERID", "VAR_INDICATOR"')

Related

R Pass required variable from ODBC/HANA connection to sql statement

I have a table I am trying to call with my usual method
sql <- 'SELECT TOP 10 *
FROM "_SYS_BIC"."data-path.self-service.DOIP/table_name"'
df <- dbGetQuery(jdbcConnection, sql)
and receive the error
Error in .verify.JDBC.result(r, "Unable to retrieve JDBC result set for ", :
Unable to retrieve JDBC result set for SELECT TOP 10 *
FROM "_SYS_BIC"."data-path.self-service.DOIP/table_name" (SAP DBTech JDBC: [2048]: column store error: search table error: [34023] Instantiation of calculation model failed;exception 306106: Undefined variable: $$IP_ExtractionWeekFrom$$. Variable is marked as required but not set in the query)
I've been trying to insert IP_ExtractionWeekFrom into the sql statement with a where clause with no luck
param1 <- 201943
sql <- 'SELECT TOP 10 *
FROM "_SYS_BIC"."ccf-edw.self-service.DOIP/R_CA_B_DemandPlan" where
"$$IP_ExtractionWeek$$" = ?'
SpringVisit <- dbGetQuery(jdbcConnection, sql, param1)
I've tried the term surrounded by the "$$" and without, and both with and without "$$" sourrounded in quotes and not. Usually am met with an "invalid column name" error.
Is this supposed to be called with something other than a where clause?
Consider maintaining your working Tableau query with the integration of parameters in R with properly handling of double quotes for identifiers and single quotes for literals.
Additionally, parameterization is not supported with the old ('PLACEHOLDER'= ('<varname>', <varvalue>)) syntax.
Instead, as explained in How to escape sql injection from HANA placeholder use the PLACEHOLDER."<varname>" => ? syntax.
param1 <- 201943
sql <- "SELECT TOP 10 *
FROM \"_SYS_BIC\".\"ccf-edw.self-service.DOIP/R_CA_B_DemandPlan\"(
PLACEHOLDER.\"$$IP_ExtractionWeekFrom$$\", ?),
PLACEHOLDER.\"$$IP_ExtractionWeekTo$$\",?)
)\"_SYS_BIC\".\"ccf-edw.self-service.DOIP/R_CA_B_DemandPlan\"
WHERE (1 <> 0)"
SpringVisit <- dbGetQuery(jdbcConnection, sql, param1, param1)
Additionally, if your JDBC already connects to the schema_SYS_BIC, use the synonymous qualifier :: as original query in order to reference package and calculation view:
sql <- "SELECT TOP 10 *
FROM \"ccf-edw.self-service.DOIP::R_CA_B_DemandPlan\"(
PLACEHOLDER.\"$$IP_ExtractionWeekFrom$$\", ?),
PLACEHOLDER.\"$$IP_ExtractionWeekTo$$\", ? )
)\"ccf-edw.self-service.DOIP::R_CA_B_DemandPlan\"
WHERE (1 <> 0)"

SQL "where" clause failing with R JDBC HANA connection

I've had nothing but trouble connecting to my company's HANA db through R but finally had a breakthrough, however now my sql statement is failing in subsetting data using a "where" statement.
The following returns a data frame of 10 observations across 9 variables
# Fetch all results
rs <- dbSendQuery(jdbcConnection, 'SELECT TOP 10
VISITTYPE,
ACCOUNT,
PLANNEDSTART,
PLANNEDEND,
EXECUTIONSTART,
EXECUTIONEND,
STATUS,
SOURCE,
ACCOUNT_NAME
FROM "_SYS_BIC"."cona-reporting.field-sales/Q_CA_R_SpringVisit"')
a <- dbFetch(rs)
However when I throw a where into it, I receive an error.
rs <- dbSendQuery(jdbcConnection, 'SELECT TOP 10
VISITTYPE,
ACCOUNT,
PLANNEDSTART,
PLANNEDEND,
EXECUTIONSTART,
EXECUTIONEND,
STATUS,
SOURCE,
ACCOUNT_NAME
FROM "_SYS_BIC"."cona-reporting.field-sales/Q_CA_R_SpringVisit" WHERE VISITTYPE = ZR')
Error in .verify.JDBC.result(r, "Unable to retrieve JDBC result set for ", :
Unable to retrieve JDBC result set for SELECT TOP 10
VISITTYPE,
ACCOUNT,
PLANNEDSTART,
PLANNEDEND,
EXECUTIONSTART,
EXECUTIONEND,
STATUS,
SOURCE,
ACCOUNT_NAME
FROM "_SYS_BIC"."cona-reporting.field-sales/Q_CA_R_SpringVisit" WHERE VISITTYPE = ZR (SAP DBTech JDBC: [260] (at 222): invalid column name: ZR: line 11 col 101 (at pos 222))
What does this mean? ZR is not a column, it is a value within the column. Tried placing ZR in quotes to no other effect.
My double and single quote syntax is based on this other question I've asked.
Issues connecting R to HANA db with many special characters
Never got it working with RODBC so tried JODBC.
Likely it is handling of quotes within an embedded quote-enclosed string, further complicated by the double quote symbols used in SQL for identifiers. However, consider parameterization (an industry best practice whenever running SQL in application layer such as R) to avoid the need of quote punctuation or concatenation. Like most JDBC APIs, RJDBC supports parameterization. Also note, dbGetQuery summarily equates to dbSendQuery + dbFetch:
sql <- 'SELECT TOP 10 VISITTYPE,
ACCOUNT,
PLANNEDSTART,
PLANNEDEND,
EXECUTIONSTART,
EXECUTIONEND,
STATUS,
SOURCE,
ACCOUNT_NAME
FROM "_SYS_BIC"."cona-reporting.field-sales/Q_CA_R_SpringVisit"
WHERE VISITTYPE = ?'
param <- 'ZR'
df <- dbGetQuery(jdbcConnection, sql, param)
To complete the previous answer (which is of course preferable as it uses bind variables) here is described the *root cause** of the problem:
The use of a single quote in a single quoted string must be of course escaped
Contrary to the Oracle escaping using doubling the quote R uses backslash.
i.e. the proper usage is as follows:
> df <- dbGetQuery(jdbcConnection,
+ 'select * from "DUAL" where "DUMMY" = \'X\'')
> df
DUMMY
1 X
alternative way using double quoted string
> df <- dbGetQuery(jdbcConnection,
+ "select * from \"DUAL\" where \"DUMMY\" = 'X'")
> df
DUMMY
1 X

dbReadTable won't pull data but dbListFields will see correct fields

I am trying to pull data from a SQL database that I have access to. I can connect to the database, see the tables and get the fields associated with a given table, but cannot read a table into an R variable.
I'm working in R Studio, in case this makes a difference.
I have tried using online code snippets (new to R) and these work, except for the dbReadTable() examples. I have used both "Payments" and name="Payments" as the second argument, and both with and without "" quotes.
library(DBI)
con<-(dbConnect(odbc::odbc(), .connection_string="Driver={SQL Server},
Server=example_1234
Database=exampleDB
TrustedConnection=TRUE")
testing123 <- dbListFields(con,"Payments")
testing456 <- dbReadTable(con,"Payments")
I expect a connection to the database which is now named con. This works.
I expect testing123 to contain all the fields in "Payments". This also works.
I expect testing456 to be a data.frame copy of Payments. This produces:
Error: 'SELECT * FROM "Payments"
nanodbc/nanobdc.cpp:1587 42s02 [Microsoft][ODBC SQL SERVER DRIVER][SQL SERVER]Invalid pbject name 'Payments'.
It's slightly different without "Payments" as the argument - simply saying "Object "Payments" not found".
Any help much appreciated.
I suspect that it's because your table is in a different catalog or schema.
Rationale: DBI::dbListFields is doing select * from ... limit 0 (which is not correct syntax for sql server), but odbc::dbListFields is really calling a C++ function connection_sql_columns that is SQL Server specific. It might be permitting you to be a touch sloppy in that it will find the table even if you do not specify the catalog and/or schema. This is why your dbListFields is working. However, DBI::dbReadTable is really doing select * from ... under the hood (and odbc:: is not overriding it), so it is not allowing you to omit the schema (and/or catalog).
First, find the specific table information for your case:
DBI::dbGetQuery(con, "select top 1 table_catalog, table_schema, table_name, column_name from information_schema.columns where table_name='events'")
# table_catalog table_schema table_name column_name
# 1 my_catalog dbo Payments Id
(I'm projecting what you'll find.)
From here, try one of the following until it works:
x <- DBI::dbReadTable(con, DBI::SQL("[Payments]")) # equivalent to the original
x <- DBI::dbReadTable(con, DBI::SQL("[dbo].[Payments]"))
x <- DBI::dbReadTable(con, DBI::SQL("[my_catalog].[dbo].[Payments]"))
My guess is that DBI::dbGetQuery(con, "select top 1 * from Payments") will not work, so for "regular queries" you'll need to use the same hierarchy of catalog.schema.table, such as one of
DBI::dbGetQuery(con, "select top 1 * from dbo.Payments")
DBI::dbGetQuery(con, "select top 1 * from [dbo].[Payments]")
DBI::dbGetQuery(con, "select top 1 * from [my_catalog].[dbo].[Payments]")
(The use of the [ and ] quoted-identifier brackets are often a personal preference, strictly required in only some corner cases.)
Try changing your con argument just slightly:
con <- DBI::dbConnect(odbc::odbc(),
Driver = "SQL Server",
Server = "example_1234",
Database = "exampleDB",
TrustedConnection = TRUE)
# read table to df
testing456 <- dbReadTable(con,"Payments")
# you can also use SQL queries directly, such as:
testing789 <- dbGetQuery(con, statement = "SELECT * FROM Payments WHERE ...")

Rmarkdown - Use table name as variable in dynamic sql chunk?

I need to execute an SQL-engine chunk in my Rmarkdown, where the table which is queried has a dynamic name, defined by R code.
I know that linking variables to the current R-environment is doable by using ?, but this works only for strings and numerics, not for "objects".
Of course I could just run the SQL query with DBI::dbGetQuery() but this would imply building all my request (which is very long) as a string which is not comfortable (I have many chunks to run).
Basically what I would need is :
`` {r}
mytable <- "name_of_table_on_sql_server"
``
then
`` {sql}
select * from ?mytable
``
This fails because the created query is select * from "name_of_table_on_sql_server" where SQL would need select * from name_of_table_on_sql_server (without quotes).
Using glue for defining mytable as mytable <- glue("name_of_table_on_sql_server") is not working neither.
Any idea ?
A slight variant on what you posted works for me (I don't have SQL Server so I tested with sqlite):
`` {r}
library(glue)
mytable <- glue_sql("name_of_table_on_sql_server")
``
then
`` {sql}
select * from ?mytable;
``
My only real changes were to use the function glue_sql and add a semicolon (;) to the end of the SQL chunk.

DB2 query error during the retrieval of a CLOB field

From Java I am doing the following query on DB2:
SELECT * FROM PRV_PRE_ACTIVATION WHERE TRANSACTION_ID = ?
The field TRANSACTION_ID is a VARCHAR of length 32. I set the parameter in the preparedStatement using the setString method.
I get the error:
com.ibm.db2.jcc.am.SqlSyntaxErrorException: DB2 SQL Error: SQLCODE=-270, SQLSTATE=42997, SQLERRMC=63, DRIVER=3.59.81
at com.ibm.db2.jcc.am.dd.a(dd.java:676)
at com.ibm.db2.jcc.am.dd.a(dd.java:60)
at com.ibm.db2.jcc.am.dd.a(dd.java:127)
at com.ibm.db2.jcc.am.bn.c(bn.java:2546)
at com.ibm.db2.jcc.am.bn.d(bn.java:2534)
at com.ibm.db2.jcc.am.bn.a(bn.java:2026)
at com.ibm.db2.jcc.t4.cb.g(cb.java:140)
at com.ibm.db2.jcc.t4.cb.a(cb.java:40)
at com.ibm.db2.jcc.t4.q.a(q.java:32)
at com.ibm.db2.jcc.t4.rb.i(rb.java:135)
at com.ibm.db2.jcc.am.bn.gb(bn.java:1997)
at com.ibm.db2.jcc.am.cn.pc(cn.java:3009)
at com.ibm.db2.jcc.am.cn.b(cn.java:3786)
at com.ibm.db2.jcc.am.cn.bc(cn.java:678)
at com.ibm.db2.jcc.am.cn.executeQuery(cn.java:652)
Where the sqstate means "Capability is not supported by this version of the DB2 application requester, DB2 application server, or the combination of the two." But I don't use any strange functionality.
I have tried using an squ client the query:
SELECT * FROM PRV_PRE_ACTIVATION where transaction_id='A'
And it goes ok.
What is the cause of the problem?
UPDATE: The code where the statement is prepared:
s = con.prepareStatement(sSQL,
ResultSet.TYPE_SCROLL_INSENSITIVE,
ResultSet.CONCUR_UPDATABLE);
Try changing to a specified list of columns in the select list -- my guess is you have a user defined column type (or some other type) which is not supported by your driver. For example, does the statement
SELECT TRANSACTION_ID FROM PRV_PRE_ACTIVATION WHERE TRANSACTION_ID = ?
work? If so then start adding columns in and you will find the problem column.
I've came across this problem lately, and after some searching on web, I've came across this link:
DB2 SQL error: SQLCODE: -270, SQLSTATE: 42997, SQLERRMC: 63
, which specifies this:
A column with a LOB type, distinct type on a LOB type, or
structured type cannot be specified in the select-list of an
insensitive scrollable cursor.
With help from an colleague, we came to this conclusion:
1, Q: When will you get this "SQLCODE=-204, SQLSTATE=42704" exception?
A: When a scrollable PreparedStatement is prepared & executed, yet there are [B|C]LOB fields exist in the select list. e.g.:
String strQuery = "SELECT NUMBER_FIELD, CHAR_FIELD, CLOB_FIELD FROM TABLE_NAME WHERE CONDITION IS TRUE;"
Statement stmt = conn.createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE, REsultSet.CONCUR_READ_ONLY);
rs = stmt.executeQuery(strQuery); //and this exception will be thrown here
2, Q: So what's the solution if we want to get rid of it when [B|C]LOB fields are queried?
A: Try to use ResultSet.TYPE_FORWARD_ONLY while creating the query statement.e.g.:
stmt = conn.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
Or simply try this one:
stmt = conn.createStatement();
Note that the same rules apply to conn.prepareStatement() too. You may refer to Java API doc for more information.