Is there a better way to code this sqlQuery in R? - sql

I'm writing an R script to get some database data and then do stuff with it, using the RODBC package. Currently all my sqlQuery commands are one long string;
stsample<-sqlQuery(odcon, paste"select * from bob.DESIGNSAMPLE T1, bob.DESIGNSUBJECTGROUP T2, bob.DESIGNEVENT T3, bob.CONFIGSAMPLETYPES T4 WHERE T1.SUBJECTGROUPID = T2.SUBJECTGROUPID AND T1.TREATMENTEVENTID = T3.TREATMENTEVENTID AND T1.SAMPLETYPEKEY = T4.SAMPLETYPEKEY AND T1.STUDYID = T2.STUDYID AND T1.STUDYID = T3.STUDYID AND T1.STUDYID = ", chstudid, sep=""))
head(stsample)
which looks ugly and is hard to read/update. I've tried putting them multiline, but then new line characters get in the way, currently my best is this using lots of paste's;
stsample<-sqlQuery(odcon,
paste(
"select ",
"* ",
"from ",
"BOB.DESIGNSAMPLE T1, ",
"BOB.DESIGNSUBJECTGROUP T2, ",
"BOB.DESIGNEVENT T3, ",
"BOB.CONFIGSAMPLETYPES T4 ",
"WHERE ",
"T1.SUBJECTGROUPID = T2.SUBJECTGROUPID ",
"AND T1.TREATMENTEVENTID = T3.TREATMENTEVENTID ",
"AND T1.SAMPLETYPEKEY = T4.SAMPLETYPEKEY ",
"AND T1.STUDYID = T2.STUDYID ",
"AND T1.STUDYID = T3.STUDYID ",
"AND T1.STUDYID = ",chstudid,
sep="")
)
head(stsample)
But I don't like having to put quotes around everyline, and getting my whitespace correct. Is there a better way ?

I would use something like this:
stsample<-sqlQuery(odcon,
paste("
####DATASET CONSTRUCTION QUERY #########
select
*
from
BOB.DESIGNSAMPLE T1,
BOB.DESIGNSUBJECTGROUP T2,
BOB.DESIGNEVENT T3,
BOB.CONFIGSAMPLETYPES T4
WHERE
T1.SUBJECTGROUPID = T2.SUBJECTGROUPID
AND T1.TREATMENTEVENTID = T3.TREATMENTEVENTID
AND T1.SAMPLETYPEKEY = T4.SAMPLETYPEKEY
AND T1.STUDYID = T2.STUDYID
AND T1.STUDYID = T3.STUDYID
AND T1.STUDYID =
###################################
", as.character(chstudid), sep="")
)

What about using gsub("\n", " ", "long multiline select string") instead of paste?

This is a really old question, but thought this may be useful to someone.
One thing I have found useful is the GetoptLong package, which provides the qq() function. I think it is inspired by Perl, but essentially it provides a way to do a multiline string with easy variable interpolation. For example:
library(GetoptLong)
tableName <- "myTable"
id <- 42
sqlQuery(odcon, qq("
SELECT * FROM #{tableName}
WHERE id = #{id}
LIMIT 1
")
Obviously I should mention the usual caveat that this is a bad idea if you are working directly with user input and it would be better to use some kind of prepared statement in that case.

Related

How to create a dynamic query using collection-valued named parameters?

As the title suggests, i'm currently trying to add parts to the JPQL-query using collection-valued named parameters (:queryLst).
Function call:
List<PanelSet> psetLst = setRepository.getMaxZchnrGroupByLeftEight(p.getCustomerNumber(), p.getDrawingNumber(), queryLst);
queryLst:
// Is used to store values from scanned and convert them into parts of a query
ArrayList<String> queryLst = new ArrayList<>();
for (int i = 0; i < size1; i++) {
scanEdvRev = scanned.get(i).toString();
queryLst.set(i, "and left(a.drawingnumber, 8) != left('" + scanEdvRev + "', 8)");
}
SetRepository:
public interface SetRepository extends CrudRepository<PanelSet, Integer> {
#Query("select distinct max(a.drawingNumber) from PanelSet a "
+ "where a.customerNumber = :customerNumber "
+ "and a.drawingNumber != :drawingNumber (:queryLst) "
+ "group by left(a.drawingNumber, 8)")
List<PanelSet> getMaxZchnrGroupByLeftEight(#Param("customerNumber") String customerNumber,
#Param("drawingNumber") String drawingNumber,
#Param("queryLst") ArrayList<String> queryLst);
}
When i run the project i get the following exception:
org.hibernate.hql.internal.ast.QuerySyntaxException: unexpected token: ( near line 1, column 159 [select distinct max(a.drawingNumber) from com.asetronics.qis2.model.PanelSet a where a.customerNumber = :customerNumber and a.drawingNumber != :drawingNumber (:queryLst) group by left(a.drawingNumber, 8)]
I'm unsure whether my approach to this problem is the correct way of doing this and whether this exception is caused by a simple syntax error or by my usage of collection-valued named parameters.
I've followed this guide trying to solve the problem.
EDIT: I'm basically trying to add each String from ArrayList<String> queryLst to the parametrized query inside setRepository.
#Query("select distinct max(a.drawingNumber) from PanelSet a "
+ "where a.customerNumber = :customerNumber "
+ "and a.drawingNumber != :drawingNumber (:queryLst) "
+ "group by left(a.drawingNumber, 8)")
If successful, the query behind the function
List<PanelSet> getMaxZchnrGroupByLeftEight(#Param("customerNumber") String customerNumber,
#Param("drawingNumber") String drawingNumber,
#Param("queryLst") ArrayList<String> queryLst);
should look like this:
queryStr = "select distinct max(a.drawingNumber) from PanelSet a "
+ "where a.customerNumber = " + customerNumber + ""
+ "and a.drawingNumber != " + drawingNumber + "";
for (String s : queryLst) {
queryStr = queryStr + s;
}
queryStr = queryStr + " group by left(a.drawingNumber, 8)";
I hope this clarifies what i'm trying to do with queryLst.
It can't be done using your approach of passing in a list of query chunks.
The closest you'll get is by adding every possible condition to the query and provide values for all those conditions in a way that allows conditions to be ignored, typically by providing a null.
You code might look like this:
#Query("select distinct max(a.drawingNumber) from PanelSet a "
+ "where a.customerNumber = :customerNumber "
+ "and a.drawingNumber != :drawingNumber "
+ "and a.myTextColumn = coalesce(:myTextColumn, a.myTextColumn) "
+ "and a.myIntegerColumn = coalesce(:myIntegerColumn, a.myIntegerColumn) "
// etc for all possible runtime conditions
+ "group by left(a.drawingNumber, 8)")
List<PanelSet> getMaxZchnrGroupByLeftEight(
#Param("customerNumber") String customerNumber,
#Param("drawingNumber") String drawingNumber,
#Param("myTextColumn") String myTextColumn,
#Param("myIntegerColumn") Integer myIntegerColumn);
Passing null for myTextColumn or myIntegerColumn will allow that column to be any value (except null).
You'll have to find SQL that works for the type of conditions you have and the data type of the columns involved and whether nulls are allowed.
If passing nulls doesn't work, use a special value, perhaps blank for text columns and some "impossible" date like 2999-01-01 fir date columns etc and code the condition like:
and (a.myCol = :myCol or :myCol = '2999-01-01')

How to properly clean column header in Power Query and capitalize first letter only without changing other letter?

I would like to clean a column Header of the table so that my column header that has a name like below:
[Space][Space][Space]First Name[Space][Space]
[Space]MaintActType[Space]
TECO date[Space]
FIN Date
ABC indicator
COGS
Created On
And my desired Column Header Name to be like below:
First Name
Main Act Type
TECO Date
FIN Date
ABC Indicator
COGS
Created On
my code is as below:
let
Source = Excel.Workbook(File.Contents("C:\RawData\sample.xlsx"), null, true),
#"sample_Sheet" = Source{[Item="sample",Kind="Sheet"]}[Data],
#"Promoted Headers" = Table.PromoteHeaders(#"sample_Sheet", [PromoteAllScalars=true]),
#"Trim ColumnSpace" = Table.TransformColumnNames(#"Promoted Headers", Text.Trim),
#"Split CapitalLetter" = Table.TransformColumnNames(#"Trim ColumnSpace", each Text.Combine(Splitter.SplitTextByPositions(Text.PositionOfAny(_, {"A".."Z"},2)) (_), " ")),
#"Remove DoubleSpace" = Table.TransformColumnNames(#"Split CapitalLetter", each Replacer.ReplaceText(_, " ", " ")),
#"Capitalise FirstLetter" = Table.TransformColumnNames(#"Remove DoubleSpace", Text.Proper),
#"Remove Space" = Table.TransformColumnNames(#"Capitalise FirstLetter", each Text.Remove(_, {" "})),
#"Separate ColumnName" = Table.TransformColumnNames(#"Remove Space", each Text.Combine(Splitter.SplitTextByCharacterTransition({"a".."z"}, {"A".."Z"}) (_), " "))
in
#"Separate ColumnName"
However, i get the result as below. Which is not what i wanted as all the capital letter we combined together. How do i change the code so that i get the result as wanted? I would really appreciate your help, please.
First Name
Main Act Type
TECODate
FINDate
ABCIndicator
COGS
Created On
Alternatively, i changed the code to:
let
Source = Excel.Workbook(File.Contents("C:\RawData\sample.xlsx"), null, true),
#"sample_Sheet" = Source{[Item="sample",Kind="Sheet"]}[Data],
#"Promoted Headers" = Table.PromoteHeaders(#"sample_Sheet", [PromoteAllScalars=true]),
#"Trim ColumnSpace" = Table.TransformColumnNames(Input, Text.Trim),
#"Separate ColumnName" = Table.TransformColumnNames(#"Trim ColumnSpace", each Text.Combine(Splitter.SplitTextByCharacterTransition({"a".."z"}, {"A".."Z"}) (_), " ")),
#"Capitalise FirstLetter" = Table.TransformColumnNames(#"Separate ColumnName", Text.Proper)
in
#"Capitalise FirstLetter"
Unfortunately it return the result like so:
First Name
Main Act Type
Teco Date
Fin Date
Abc Indicator
COGS
Created On
I have no idea how to play around the code anymore.
One way is to mark the existing spaces with something (I used "ZZZ") and restore them back to spaces at the end. Here's your code with a couple of tweaks. Thanks for your code. I was trying to do Text.Proper and your sample helped me!
let
Source = Input,
#"Replaced Value" = Table.ReplaceValue(Source,"[Space]"," ",Replacer.ReplaceText,{"Headers"}),
#"Transposed Table" = Table.Transpose(#"Replaced Value"),
#"Promoted Headers" = Table.PromoteHeaders(#"Transposed Table", [PromoteAllScalars=true]),
#"Trim ColumnSpace" = Table.TransformColumnNames(#"Promoted Headers", Text.Trim),
#"Change space to ZZZ" = Table.TransformColumnNames(#"Trim ColumnSpace", each Replacer.ReplaceText(_, " ", " ZZZ ")),
#"Split CapitalLetter" = Table.TransformColumnNames(#"Change space to ZZZ", each Text.Combine(Splitter.SplitTextByPositions(Text.PositionOfAny(_, {"A".."Z"},2)) (_), " ")),
#"Capitalise FirstLetter" = Table.TransformColumnNames(#"Split CapitalLetter", Text.Proper),
#"Remove Space" = Table.TransformColumnNames(#"Capitalise FirstLetter", each Text.Remove(_, {" "})),
#"Separate ColumnName" = Table.TransformColumnNames(#"Remove Space", each Text.Combine(Splitter.SplitTextByCharacterTransition({"a".."z"}, {"A".."Z"}) (_), " ")),
#"Change ZZZ to space" = Table.TransformColumnNames(#"Separate ColumnName", each Replacer.ReplaceText(_, "ZZZ", " ")),
#"Remove DoubleSpace" = Table.TransformColumnNames(#"Change ZZZ to space", each Replacer.ReplaceText(_, " ", " "))
in
#"Remove DoubleSpace"

Talend - Read a SQL from .txt and pass variables to it

I need to read an Excel file and excecute a different SQL Query (Oracle) for each row of the Excel, depending on the value that a column ("tipo") of the table has. Also, I need to pass variables to the SQL query that comes from others columns of the same excel.
I have accomplished this with a tjava, creating a string that generates de query.. somthing like this (simplified):
String var1 = input_row.var1;
String var2 = input_row.var2;
String sql_query = ""
if (tipo.toString().equals(String.valueOf("Option1"))) {
query_ev = "select " + var1 + " as variable1, " + var2 + " as var 2 from dual";}
if (tipo.toString().equals(String.valueOf("Option2"))) {
query_ev = "select " + var1+ " from dual";}
context.sql = sql_query;
Nevertheless, my "problem" is that my queries are long, and this method only allows me to put them in the tjava whithout line breaks, so its very difficult to edit them since they remain a "one-row-query" in the tjava.
Is there a way to accomplish the same but having the queries formatted ?
for example, for Option1 having this
select " + var1 + " as variable1, " + var2 + " as var 2
from dual
where
...
I appreciate any clue.
Try this:
query_ev = "select "
" + var1 + " as variable1, " + var2 + " as var 2 "+
from dual";
I am without Java right now to verify but I think this approach could help formatting. Just check that you have enough spaces - don't do this
"as var 2"+"from dual"
which would result in SQL like
"as var 2from dual"

Oracle SQL Select Query Not Working in VB.NET Works Fine in SQL +

When my query executes within my VB application I am receiving the following error:
ORA-00942: table or view does not exist
All table names in my query are spelt correctly, and do in fact exist.
If I dump the query from my VB.net app and then run the query manually in Oracle SQL Plus it executes just fine.
In both cases I am logged in with the exact same credentials and have selected the same database. For my connection within visual basic I am using OleDb.
From within my visual basic application I also ran a query to dump all tables which the user has access to using
select table_name from all_tables
And the table names which I am querying show up.
Any idea what would be causing this?
SELECT
RSS.STEP_STATUS_DATE,
RSS.VALUE_RECORDED
FROM
ITR,
REPORT R,
INSTRUCTION I,
INSTRUCTION_STEP INS,
REPORT_STEP RS,
REPORT_STEP_STATUS RSS
WHERE
ITR.ITR_NO = '1' AND
I.INSTRUCTION_ID = '12345' AND
INS.STEP_NO = '2' AND
R.INSTRUCTION_ID = I.INSTRUCTION_ID AND
RS.REPORT_ID = R.REPORT_ID AND
RS.INSTRUCTION_STEP_ID = INS.INSTRUCTION_STEP_ID AND
RSS.REPORT_STEP_ID = RS.REPORT_STEP_ID AND
RSS.MEASUREMENT_NAME = 'ESN'
My visual basic code is as follows:
strQuery = "SELECT RSS.STEP_STATUS_DATE, " +
" RSS.VALUE_RECORDED " +
"FROM ITR, " +
" REPORT R, " +
" INSTRUCTION I, " +
" INSTRUCTION_STEP INS, " +
" REPORT_STEP RS, " +
" REPORT_STEP_STATUS RSS " +
"WHERE ITR.ITR_NO = '%01' AND " +
" I.INSTRUCTION_ID = '%02' AND " +
" INS.STEP_NO = '%03' AND " +
" R.INSTRUCTION_ID = I.INSTRUCTION_ID AND " +
" RS.REPORT_ID = R.REPORT_ID AND " +
" RS.INSTRUCTION_STEP_ID = INS.INSTRUCTION_STEP_ID AND " +
" RSS.REPORT_STEP_ID = RS.REPORT_STEP_ID AND " +
" RSS.MEASUREMENT_NAME = '%04'"
I'm looking at this portion (and others like it):
RSS.MEASUREMENT_NAME = '%04'
I expect you meant to do this:
RSS.MEASUREMENT_NAME LIKE '%04'
While I'm here, almost no one uses the "A,B" join syntax any more. It's extremely out-dated, and can lead to errors where join conditions are applied to the wrong tables, such that the query doesn't return any results... in other words, it might even be causing the problem you have right now.

Entity Framework - how to join tables without LINQ and with only string?

I have a question about Entity Framework. Please answer if you know answer on this. I have such query :
String queryRaw =
"SELECT " +
"p.ProductName AS ProductName " +
"FROM ProductEntities.Products AS p " +
"INNER JOIN CategoryEntities.Categories AS c " +
"ON p.CategoryID = c.CategoryID ";
ObjectQuery<DbDataRecord> query = new ObjectQuery<DbDataRecord>(queryRaw, entityContext);
GridView1.DataSource = query;
GridView1.DataBind();
Particularly I want to join few tables in one query, but I can NOT use LINQ and can NOT use ObjectQuery with objects mapped to DB fields inside my query. Because each entity creates dynamically. So this is what i can NOT use :
msdn.microsoft.com/en-us/library/bb425822.aspx#linqtosql_topic12
msdn.microsoft.com/en-us/library/bb896339%28v=VS.90%29.aspx
The question is can I use something like this instead of using objects?
query.Join ("INNER JOIN CategoryEntities.Category ON p.CategoryID = c.CategoryID ");
The purpose is to use Join method of ObjectQuery with syntax as in Where method :
msdn.microsoft.com/en-us/library/bb338811%28v=VS.90%29.aspx
Thanks, Artem
Any decision i see right now is to temporary convert ObjectQuery to string, add joined table as string and then convert it back to ObjectQuery :
RoutesEntities routesModel = new RoutesEntities(entityConnection);
String queryRaw = "SELECT " +
"rs.RouteID AS RouteID, " +
"rs.LocaleID AS LocaleID, " +
"rs.IsSystem AS IsSystem " +
"FROM RoutesEntities.Routes AS rs ";
_queryData = new ObjectQuery<DbDataRecord>(queryRaw, routesModel);
var queryJoin = _queryData.CommandText + " INNER JOIN LocalesEntities.Locales AS ls ON ls.LocaleID = rs.LocaleID ";
_queryData = new ObjectQuery<DbDataRecord>(queryJoin, routesModel);
Maybe someone has more consistent suggestions?
Finally I Found a better solution for this, we can use Sub Query inside main Query. For example :
var db = CustomEntity();
ObjectQuery<Categories> query1 = db.Categories.Where("it.CategoryName='Demo'").Select ("it.CategoryID");
var categorySQL = query1.ToTraceString().Replace("dbo", "CustomEntity"); // E-SQL need this syntax
ObjectQuery<Products> query2 = db.Categories.Where("it.CategoryID = (" + categorySQL + ")");
Some example is here :
http://msdn.microsoft.com/en-us/library/bb896238.aspx
Good luck!