Related
I am working to make R more user friendly. I am using library(RODBC) function to have direct access to the SQL database from RStudio. In markdown I use code something like this:
sql <- "
Select [A]
,[year]
,[model]
....
from
(SELECT ....
Where [year] between 2010 and 2020 and model ='1A' and [A] between 11 and 22... "
I need to let the user specify parameters from the last line (for instance 2010...2020 , '1A' and 11 ...22) in the terminal by using readline() function.
It is not the best option, but you can simply ask for some inputs before,
range_start <- readline("Range start: ")
range_end <- readline("Range start: ")
literal_and <- "and"
sql_stmt <- "select year, model from where year between"
and then concatenate them in one query
final_sql <- paste(sql_stmt, range_start, literal_and, range_end)
I would like to iterate this SQL query over the 17 rows in my df. My df and code are below. I think I may need single quotes around dat$ptt_id, because I get a syntax error at the "IN" function. Any ideas how to correctly write this?
df looks like:
ptt_id
1 181787
2 181788
3 184073
4 184098
5 197601
6 197602
7 197603
8 197604
9 197605
10 197606
11 197607
12 197608
13 197609
14 200853
15 200854
16 200851
17 200852
#Load data----
dat <- read.csv("ptts.csv")
dat2<-list(dat)
#Send to database----
for(i in 1:nrow(dat)){
q <- paste("SELECT orgnl_pit, t_name, cap_date, species, sex, mass, cap_lat, cap_lon, sat_appld_id
FROM main.capev JOIN uid.turtles USING (orgnl_pit)
WHERE sat_appld_id IN", dat$ptt_id[i],";")
#Get query----
tags <- dbGetQuery(conn, q)
}
Error in postgresqlExecStatement(conn, statement, ...) :
RS-DBI driver: (could not Retrieve the result : ERROR: syntax error at or near "181787"
LINE 3: WHERE sat_appld_id IN 181787 ;
^
Thanks for any assistance.
Two options:
Parameter binding.
qmarks <- paste0("(", paste(rep("?", nrow(df)), collapse = ","), ")")
qmarks
# [1] "(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)"
qry <- paste(
"SELECT orgnl_pit, t_name, cap_date, species, sex, mass, cap_lat, cap_lon, sat_appld_id
FROM main.capev JOIN uid.turtles USING (orgnl_pit)
WHERE sat_appld_id IN", qmarks)
tags <- dbGetQuery(conn, qry, params = df[,1])
Temporary table. This might be more useful when you have a large number of ids to use. (This also works without a temp table if you get the ids from the database anyway, and can use that query in this sub-query.)
dbWriteTable(conn, df, "mytemp")
qry <- "SELECT orgnl_pit, t_name, cap_date, species, sex, mass, cap_lat, cap_lon, sat_appld_id
FROM main.capev JOIN uid.turtles USING (orgnl_pit)
WHERE sat_appld_id IN (select id from mytemp)"
tags <- dbGetQuery(conn, qry)
dbExecute(conn, "drop table mytemp")
(Name your temp table carefully. DBMSes usually have a nomenclature to ensure the table is automatically cleaned/dropped when you disconnect, often something like "#mytemp". Check with your DBMS or DBA.)
The IN operator requires a list. You can think of it as multiple OR conditions.
E.g. instead of WHERE sat_appld_id IN 181787 it should be WHERE sat_appld_id IN (181787)
And to that point, instead of a loop you could create a list from your dat$ptt_id column for just one sql query such as WHERE sat_appld_id IN (181787, 181788, 184073, ...) and do any additional wraggling within your R code instead of making multiple database queries.
I am using R to link HANA, so that I can use SQL query in R for data retrieve. My data includes a lot of store names. sometimes, my query is based on all the stores, which can be done easily without setting up any limits in WHERE. Sometimes, I just focus on a couple of stores, like store 1,2,3. I can use the answer from Providing lookup list from R vector as SQL table for RODBC lookup to do this.
For example:
IDlist <- c(23, 232, 434, 35445)
paste("WHERE idname IN (", paste(IDlist, collapse = ", "), ")")
But how can I combine these two situations, i.e., all names or name subset, together in a WHERE? I am looking forward for something like:
IDlist <- all
IDlist <- c(23, 232, 434, 35445)
paste("WHERE idname IN (", paste(IDlist, collapse = ", "), ")")
So, when IDlist is all, then the query will be for all the store names. When IDlist have some specific numbers, then the query will just focus on those stores.
This is just my idea. I am not sure if there is a better way to do it. Anyway, combining all names and some names together so that I can use them in one WHERE, therefore, I do not need to change my code.
Here, my WHERE is :
myOffice <- c(416,247,602,428)
WHERE "a"."/BIC/ZARTICLE"<>\'GIFTCARDPU\' AND "a"."/BIC/ZRETURN"=\'X\'
AND "a"."CALDAY" BETWEEN',StartDate,'AND',EndDate,'
AND "a"."/BIC/ZSALE_OFF" IN (',paste(myOffice, collapse = ", "),')
Maybe something like this:
office_clause = ""
if (IDlist != all) {
office_clause = paste(
'AND "a"."/BIC/ZSALE_OFF" IN (',
paste(IDlist, collapse = ', '),
')'
)
}
Then you can construct your query, just pasting office_clause at the end of the where. If it IDlist is all, then you will paste on a blank string, otherwise you will paste on the ID clause. (Note that I assume all is a variable because that's how you used it in the question.)
I am tring to using insert into sql syntax in R to insert row in data frame but is showing the following error:
(( error in sentax ))
Below is sample of my code:
Vector <- c("alex" ,"IT")
Tst <- data.frame( name.charcher(), major.charachter())
sqldf( c(" insert into Tst values (" , Vector[1] , "," ,Vector[2] , ")" , "select * from main.Tst "))
I hope my question is clear
A few edits to help address the syntax error:
use a lower case s in the function name (sqldf() instead of Sqldf())
add a comma between "," and Vector[2]
add quotes around select * from main.Tst
Also, to note:
the 1d data structure for the heterogeneous content types in your Vector <- c("alex", 32) should be a list (rather than an atomic vector where all contents are of the same type).
depending on what database driver you're using, sqldf() may return an error if you try to insert values into an empty R data frame as you have in your code. Creating the empty data frame within the sqldf() call is one approach to avoid this (used below in absence of knowing your database info).
For example, you could use the following to resolve the error message you're getting:
library(sqldf)
new <- list(name='alex', age=as.integer(32))
Tst <- sqldf(c("create table T1 (name char, age int)",
paste0("insert into T1 (name, age) values ('", new$name[1],"',", new$age[1],")",sep=''),
"select * from T1"))
Tst
# > Tst
# name age
# 1 alex 32
I am new to SQL and i am trying to figure out how i can "select all" in WHERE clause.
Let me make it bit more clearer why i would like to achieve that:
I do have reactive SQL in my Shiny App. The user can filter at least three different variables in my SQL:
In this case user can choose Kunde (Customer), Abmessung (Diameter) and Date. I do have hard time to figure out when the user wants to filter only one/two variables and not three (the combinations of possibilities inside is huge, and writing SQL Query in if statement for each is quite a thing). So for example the user would like to have Kunde (Customer) filtered but all the Abmessung (Diameters) kept.
Here is a sample SQL Query which i have used:
select * from x.xy where kdname IN (",paste0("'",paste(input$kunde,
collapse="', '"),"'"),") and abmessung IN (",paste0("'",paste(input$abm,
collapse="', '"),"'"),") and dati_create between
to_date('",format(input$date[1], '%d.%m.%Y'),"','dd.mm.yyyy') and
to_date('",format(input$date[2], '%d.%m.%Y'),"','dd.mm.yyyy') + (86399/86400)
Is there a possibility in SQL to use some kind of "*" in WHERE clause?
Why not simply do it using if to each condition and than paste collapse it
( i think there isn t way to do what your want without "if\esle" or "case" or "nvl" or "decode" so you need to hardcode it)
like:
input=list("kunde"="a","abm"="b",date=c("2016-01-01","2016-02-01"))
#input=list("kunde"="","abm"="",date=c("","")) # for test
sql_main="select * from x.xy "
sql_cond=list()
sql_cond[1]= if(input$kunde==""){NULL}else{paste0("kdname IN ('",paste(input$kunde,collapse="','"),"')")}
sql_cond[2]= if(input$abm==""){NULL}else{paste0("abmessung IN ('",paste(input$abm,collapse="', '"),"')")}
sql_cond[3]= if(input$date[1]==""|input$date[2]==""){NULL}else{paste0("dati_create between to_date('",as.character(as.Date(input$date[1]),'%d.%m.%Y'),"','dd.mm.yyyy') and
to_date('",as.character(as.Date(input$date[2]),'%d.%m.%Y'),"','dd.mm.yyyy') + (86399/86400)")}
sql_cond=sql_cond[!sapply(sql_cond,is.null)]# needed to del NULL in list
sql_cond_all=paste(sql_cond,collapse =" and ")
sql=if(sql_cond_all!=""){paste(sql_main,"where",sql_cond_all)}else{sql_main}
sql
If you use RMySQL you can just paste together a chr string as query.
So you could do something like this:
condition <- character()
if(input$kdname != "") condition <- append(condition, paste0("kdname IN '", paste(input$kdname, collapse = "', '"), "'"))
if(input$abm != "") condition <- append(condition, paste0("abmessung IN '", paste(input$abm, collapse = "', '"), "'"))
query <- paste("SELECT * FROM x.xy WHERE", paste(condition, collapse = " AND "))
I am not familiar with shiny. However, I think you could achieve your goal by using oracle's decode function.
The where clause will look something like this (pseudo-code)
where
decode(input_variable,null,1,input_variable)=decode(input_variable,null,1,table_column)
and dati_create between
decode(input_date1,null,'1/1/1900',input_date1)
and decode(input_date2,null,'1/1/3000',input_date2)
and table_column2 in (
SELECT decode(inputvarable2,null, (select table_column2 from dual)
,TRIM(REGEXP_SUBSTR(temp, '[^,]+', 1, level)) )
FROM (SELECT inputvarable2 temp FROM DUAL)
CONNECT BY level <= REGEXP_COUNT(temp, '[^,]+')
)
Here I assume that when input variable is not given - it has null value. Also, as user input sample was not provided, I assume that user input for in clause is comma separated string. (e.g. 'a,b,c')
And thus, if input_variable is null then where becomes 1=1 (always true), otherwise it is input_variable=table_column. With dates it is a little bit more tricky - so I give very early date (1/1/1900), or very far away one (1/1/3000). The logic behind in clause, is to convert user input into collection from comma separated string and then use the same trick with decode.
Though this might not be the most efficient way to do this.
Also I see that you are concatenating user input directly into your sql statement. This is highly risky - as your code would be prone to sql injection attacks.
I find sub function very useful when constructing queries, try this:
myquery <- 'select * from x.xy where kdname IN "NAME" and abmessung IN "ABM" and dati_create between DATE1 and DATE2 + (86399/86400)'
date1 <- input$date[1];date1 <- sub("-",".",date1);date1 <- sub("-",".",date1)
date2 <- input$date[2];date2 <- sub("-",".",date2);date2 <- sub("-",".",date2)
myquery <- sub("DATE1",date1,myquery)
myquery <- sub("DATE2",date2,myquery)
myquery <- sub("NAME",input$kunde,myquery)
myquery <- sub("ABM",input$abm,myquery)
myquery <- noquote(myquery)
myquery