How best to process dynamic query parameters using Java & Spring - sql

The problem:
I have an api endpoint which handles multiple query parameters. It is implemented using Spring, and the query parameters are used to query data from a postgres database, which I query with a JDBC Template.
I am searching for a mature query builder technology to solve my problem.
Example:
A trivial query could look something like this:
api/book?name=LOTR&cover=hardback
The query parameters are added to a map, and a query string is build from the maps data:
String sqlQuery += (String) map.entrySet().stream()
.map(entry -> entry.getKey() + "='" + entry.getValue() + "' AND ")
.collect(Collectors.joining());
Its not the most efficient, as I must always remove the trailing "AND" clause from the string, but it works.
However, if the query where to look something like
api/book?name=LOTR&name=Ulysses&cover=hardback
there is now the addition of an "OR" clause, which the above code would not handle. I can see myself quickly getting into the territory of with tedious string parsing to create SQL statements.
So now that I have presented my problem, I wonder if there is a technology I can use which handles this kind of problem nicely?
I would like to avoid the use of any ORM for this project, so Hibernate and MyBatis are out of the question. I have looked at some JOOQ examples, but they do not look compatible with JDBC Template.

For trivial implementations like your first case where you have to remove last AND after your query is built there is a simple hack - immediately after WHERE you add 1 = 1 and then for every WHERE predicate you add AND [COLUMN] = [VALUE].
Note: most databases optimise use of constants in WHERE clause before execution, so performance will not be an issue
/*
select <columns> from <tables> where 1 = 1
[dynamically built Where predicates will come here from following code]
*/
String sqlQuery += (String) map.entrySet().stream()
.map(entry -> "AND " + entry.getKey() + "='" + entry.getValue() + "'")
.collect(Collectors.joining());
However for serious production implementations you may want to use frameworks like myBatis that gives you possibilities of templating a query and then passing parameters at runtime to build final queries.
You can find a good tutorial here.
/* An example */
<select id = "getName_Id_phone" parameterType = "Student" resultType = "Student">
SELECT * FROM STUDENT
<where>
<if test = "id != null">
id = #{id}
</if>
<if test = "name != null">
AND name LIKE #{name}
</if>
</where>
</select>

Came across here with the same question. Maybe you want to take a look at RSQL 1

Related

What this mean " Incorrectly Handled Query Assembly " in SQL injection?

im reading book is about " SQL-injection " so i defaced a title
"Incorrectly Handled Query Assembly" what does this mean? and can you give me a example code ? Thanks.
I think you might have understood SQL injection. Incorrectly Handled Query Assembly seems to mean incorrect construction of query string.
Think of a simple scenario where you have written a query to list the whole details of the (logged in) user. Let us
String part1="SELECT * FROM TRANSACTIONS WHERE TAG=' ";
part2=" ' AND ID=' ";
part3=" ';";
//Constructing query with user inputted tag and user ID
String query=part1+ user_entered_tag + part2 + user_id + part3;
//This is an unsafe construction of query.
If the user enters tag like this:
Abc' OR '2'='2' OR '1'='1
The query will become like this:
SELECT * FROM TRANSACTIONS WHERE TAG='Abc' OR '2'='2' OR '1'='1' AND ID='544678';
If the query is then executed, all the transactions will be fetched.
Thus unauthorized person will have access to data. This happens because a loophole is left in the construction of query. The developer should avoid such injection by adding type checking or using built-in features like prepared statements .

Wildcard, '%', within ColdFusion cfscript query LIKE statement?

Is it possible to use a wildcard in a SQL LIKE statement within a ColdFusion cfscript query?
An example that doesn't work:
local.q = new Query();
local.q.setDatasource(variables.dsn);
local.q.addParam(name='lastname', value='%' & arguments.lastname, cfsqltype="cf_sql_varchar");
local.qString = 'SELECT name FROM users WHERE lastname LIKE :lastname';
local.q.setSQL(local.qString);
local.result = local.q.execute().getResult();
I also tried these, which didn't work:
local.qString = 'SELECT name FROM users WHERE lastname LIKE %:lastname';
local.qString = "SELECT name FROM users WHERE lastname LIKE '%:lastname'";
UPDATE:
I am using MS SQL Server 2008.
The query works fine within SQL Server Mgmt Studio... I think it has something to do with how to format the query within cfscript tags?
Yes, it is possible. You're setting it in the param, which is correct. I'm not sure why it's not working with you.
I did the following and it worked.
var qryArgsCol = {};
qryArgsCol.datasource = variables.datasource;
qryArgsCol.SQL = "
SELECT ID
FROM Users
WHERE LastName LIKE :searchStringParam
";
var qryGetID = new query(argumentCollection=qryArgsCol);
qryGetID.addParam(name="searchStringParam", value="%" & searchString, cfsqltype="cf_sql_varchar");
qryGetIDResult = qryGetID.execute().getResult();
There's a response here from Adam Cameron, which was apparently deleted by an overzealous mod.
Rather than repeat what he says, I've just copied and pasted (with emphasis added to the key parts):
Just to clarify that the syntax you tried in your first example does work. That is the correct approach here. To clarify / explain:
The <cfquery> version of the example you have would be along the lines of:
<cfqueryparam value="%foo">
So in the function version, the param would be ? or :paramName and the value of the param would continue to be "%foo".
The % is part of the param value, not the SQL string.
So given that "doesn't work" for you, it would help if you posted the error, or whatever it is that causes you to think it's not working (what your expectation is, and what the actual results are). Then we can deal with the actual cause of your problem, which is not what you think it is, I think.
Does the query work fine as a <cfquery>?
Depending on the dbms used, that single and double quotes may be interpreted when the sql statement is run. What dbms are you using? Your statement now doesn't select for the value in the variable, but for any user whose lastname is "lastname". It should be something like:
lastname like '%#lastname#'
Just remember that you ultimately need to see what CF gives the DB server. In this instance, you can try this mockup to get close and find the same error in SSMS by messing with the quotes/value in the param declaration:
declare #param1 varchar(max) = '%Eisenlohr';
SELECT name FROM users WHERE lastname LIKE #param1
I just ran into the same problem as the original poster where it "wasn't working" and I didn't get any results from the query of queries.
The problem for me is that the wildcard search is case-sensitive.
local.q = new Query();
local.q.setDatasource(variables.dsn);
local.q.addParam(name='lastname', value='%' & LCase(arguments.lastname), cfsqltype="cf_sql_varchar");
local.qString = 'SELECT name FROM users WHERE LOWER(lastname) LIKE :lastname';
local.q.setSQL(local.qString);
local.result = local.q.execute().getResult();
So what I did was made sure the incoming argument was lower case and made sure the comparing field in the SQL was lower case as well and it worked.
Use like this.
local.q = new Query();
local.q.setDatasource(variables.dsn);
local.q.addParam(name="lastname", cfsqltype="cf_sql_varchar",value='%ARGUMENTS.lastname' );
local.qString = 'SELECT name FROM users WHERE lastname LIKE :lastname';
local.q.setSQL(local.qString);
local.result = local.q.execute().getResult();
I would suggest using the CFQuery tag instead of attempting to run queries within CFScript. Unless you REALLY know what you are doing. I say this because the CFQuery tag has some built-in functionality that not only makes building queries easier for you but may also protect you from unforeseen attacks (the SQL injection type). For example, when using CFQuery it will automatically escape single-quotes for you so that inserting things like 'well isn't that a mess' will not blow up on you. You also have the benefit of being able to use the CFQueryParam tag to further battle against SQL injection attacks. While you may be able to use the CFQueryParam functionality within CFScript it is not as straight forward (at least not for me).
See this blog post from Ben Nadel talking about some of this.
So in CFQuery tags your query would look something like this:
<cfquery name="myQuery" datasource="#variables.dsn#">
SELECT name
FROM users
WHERE lastname LIKE <cfqueryparam cfsqltype="cf_sql_varchar" value="%:#arguments.lastname#" maxlength="256" />
</cfquery>

NHibernate SQL query issue with string parameter

I got a problem with this code:
string sql = "select distinct ruoli.c_tip_usr"
+ " from vneczx_ute_app_trn ruoli"
+ " join vnecyd_ent_prf ind"
+ " on ruoli.c_ent = ind.c_ent"
+ " where ruoli.c_app = :appCode"
+ " and ruoli.c_ute_mat = :matricola"
+ " and ind.t_des_ent = :indirizzo";
var ruoli = session.CreateSQLQuery(sql)
.SetString("appCode", Config.Configurator.Istance.ApplicationCode)
.SetString("matricola", user.Matricola)
.SetString("indirizzo", indirizzoCasella)
.List<string>();
This code is correctly executed, the query logged is correct, and the parameter passed correctly evaluated... but it doesn't return any result at all.
Copying the query from the debug console and executing it directly in an Oracle client application (SQL Developer), it gets 2 results (the results I expect to be right).
I found out that the problem is in the last parameter indirizzo, and should depend on the fact that it contains a special char # (indirizzo is an email address).
So I ended up using this solution:
string sql = "select distinct ruoli.c_tip_usr"
+ " from vneczx_ute_app_trn ruoli"
+ " join vnecyd_ent_prf ind"
+ " on ruoli.c_ent = ind.c_ent"
+ " where ruoli.c_app = :appCode"
+ " and ruoli.c_ute_mat = :matricola"
+ " and ind.t_des_ent = '" + indirizzoCasella + "'";
var ruoli = session.CreateSQLQuery(sql)
.SetString("appCode", Config.Configurator.Istance.ApplicationCode)
.SetString("matricola", user.Matricola)
.List<string>();
But it gives me thrills! Aren't the parameters in a query supposed to handle specifically this situation, and thus handle themselves situations with special char, and so on?
Why here a string concatenation works better that a parametric query?
Isn't there a way to force the NHibernate engine to escape some special char?
Update:
I found out how to solve this particular issue: usign the trim command on the field who raise the problem, the problem disappears.
So last line of my sql string now is:
+ " and trim(ind.t_des_ent) = :indirizzo";
I can't understand why it solves the problem thought. Not the field, nor the variable contains empty chars and copying th query on SQL Developer works in both way.
So we have some luck soving the problem, but we have no idea why now it works?
Any thoughts?
even I was facing the same issue, but your hint using TRIM for column saved my day. Since I was looking for a long time, but not able to figure it out.
Along with that, I was able solve my issue by doing the below changes as well:
We were using CHAR datatype some of the columns which used in the query where clause. This was causing the issue to fetch the data from NHibernate. We changed the data type of that column from CHAR to VARCHAR2 and even updated the data with actual size and removed the TRIM from Where clause, the TRICK WORKED!!!! :)
So, any one face this kind of issue, first check whether you are having any column with CHAR and then change to VARCHAR2.
But one more thing you have to remember is: if you are running your application from ASP.Net Development Server, close it and re-run your application. Since if the opening Asp.Net Development Server and if you make any changes to datatype that will NOT be refreshed in your oracle connection. So, better you close the ASP.Net Development server and then re run the application.
Hope my points will help somebody in future!!
Regards,
Sree Harshavardhana
You are not using parameters in a SQL query if you want SQL parameters use a SQL stored proc

SQL Injection: is this secure?

I have this site with the following parameters:
http://www.example.com.com/pagination.php?page=4&order=comment_time&sc=desc
I use the values of each of the parameters as a value in a SQL query.
I am trying to test my application and ultimately hack my own application for learning purposes.
I'm trying to inject this statement:
http://www.example.com.com/pagination.php?page=4&order=comment_time&sc=desc' or 1=1 --
But It fails, and MySQL says this:
Warning: mysql_fetch_assoc() expects parameter 1 to be resource,
boolean given in /home/dir/public_html/pagination.php on line 132
Is my application completely free from SQL injection, or is it still possible?
EDIT: Is it possible for me to find a valid sql injection statement to input into one of the parameters of the URL?
The application secured from sql injection never produces invalid queries.
So obviously you still have some issues.
Well-written application for any input produces valid and expected output.
That's completely vulnerable, and the fact that you can cause a syntax error proves it.
There is no function to escape column names or order by directions. Those functions do not exist because it is bad style to expose the DB logic directly in the URL, because it makes the URLs dependent on changes to your database logic.
I'd suggest something like an array mapping the "order" parameter values to column names:
$order_cols = array(
'time' => 'comment_time',
'popular' => 'comment_score',
... and so on ...
);
if (!isset($order_cols[$_GET['order'])) {
$_GET['order'] = 'time';
}
$order = $order_cols[$_GET['order']];
Restrict "sc" manually:
if ($_GET['sc'] == 'asc' || $_GET['sc'] == 'desc') {
$order .= ' ' . $_GET['sc'];
} else {
$order .= ' desc';
}
Then you're guaranteed safe to append that to the query, and the URL is not tied to the DB implementation.
I'm not 100% certain, but I'd say it still seems vulnerable to me -- the fact that it's accepting the single-quote (') as a delimiter and then generating an error off the subsequent injected code says to me that it's passing things it shouldn't on to MySQL.
Any data that could possibly be taken from somewhere other than your application itself should go through mysql_real_escape_string() first. This way the whole ' or 1=1 part gets passed as a value to MySQL... unless you're passing "sc" straight through for the sort order, such as
$sql = "SELECT * FROM foo WHERE page='{$_REQUEST['page']}' ORDER BY data {$_REQUEST['sc']}";
... which you also shouldn't be doing. Try something along these lines:
$page = mysql_real_escape_string($_REQUEST['page']);
if ($_REQUEST['sc'] == "desc")
$sortorder = "DESC";
else
$sortorder = "ASC";
$sql = "SELECT * FROM foo WHERE page='{$page}' ORDER BY data {$sortorder}";
I still couldn't say it's TOTALLY injection-proof, but it's definitely more robust.
I am assuming that your generated query does something like
select <some number of fields>
from <some table>
where sc=desc
order by comment_time
Now, if I were to attack the order by statement instead of the WHERE, I might be able to get some results... Imagine I added the following
comment_time; select top 5 * from sysobjects
the query being returned to your front end would be the top 5 rows from sysobjects, rather than the query you try to generated (depending a lot on the front end)...
It really depends on how PHP validates those arguments. If MySQL is giving you a warning, it means that a hacker already passes through your first line of defence, which is your PHP script.
Use if(!preg_match('/^regex_pattern$/', $your_input)) to filter all your inputs before passing them to MySQL.

SQL Select Like Keywords in Any Order

I am building a Search function for a shopping cart site, which queries a SQL Server database. When the user enters "Hula Hoops" in the search box, I want results for all records containing both "Hula" and "Hoop", in any order. Furthermore, I need to search multiple columns (i.e. ProductName, Description, ShortName, MaufacturerName, etc.)
All of these product names should be returned, when searching for "Hula hoop":
Hula hoop
Hoop Hula
The Hoopity of xxhula sticks
(Bonus points if these can be ordered by relevance!)
It sounds like you're really looking for full-text search, especially since you want to weight the words.
In order to use LIKE, you'll have to use multiple expressions (one per word, per column), which means dynamic SQL. I don't know which language you're using, so I can't provide an example, but you'll have to produce a statement that's like this:
For "Hula Hoops":
where (ProductName like '%hula%' or ProductName like '%hoops%')
and (Description like '%hula%' or Description like '%hoops%')
and (ShortName like '%hula%' or ShortName like '%hoops%')
etc.
Unfortunately, that's really the only way to do it. Using Full Text Search would allow you to reduce your criteria to one per column, but you'll still have to specify the columns explicitly.
Since you're using SQL Server, I'm going to hazard a guess that this is a C# question. You'd have to do something like this (assuming you're constructing the SqlCommand or DbCommand object yourself; if you're using an ORM, all bets are off and you probably wouldn't be asking this anyway):
SqlCommand command = new SqlCommand();
int paramCount = 0;
string searchTerms = "Hula Hoops";
string commandPrefix = #"select *
from Products";
StringBuilder whereBuilder = new StringBuilder();
foreach(string term in searchTerms.Split(' '))
{
if(whereBuilder.Length == 0)
{
whereBuilder.Append(" where ");
}
else
{
whereBuilder.Append(" and ");
}
paramCount++;
SqlParameter param = new SqlParameter(string.Format("param{0}",paramCount), "%" + term + "%");
command.Parameters.Add(param);
whereBuilder.AppendFormat("(ProductName like #param{0} or Description like #param{0} or ShortName like #param{0})",paramCount);
}
command.CommandText = commandPrefix + whereBuilder.ToString();
SQL Server Full Text Search should help you out. You will basically create indexes on the columns you want to search. in the where clause of your query you will use the CONTAINS operator and pass it your search input.
you can start HERE or HERE to learn more
You might want to check out SOLR too - if you're going to be doing this type of searching. Super cool.
http://lucene.apache.org/solr/