I have been reading some recipes in the Perl Hacks book. Recipe #24 "Query Databases Dynamically without SQL" looked interesting. The idea is to use SQL-Abstract to generate the SQL statement for you.
The syntax to generate a select statement looks something like this:
my($stmt, #bind) = $sql->select($table, \#fields, \%where, \#order);
To illustrate further, an example could look like this (taken from the perldoc):
my %where = (
requestor => 'inna',
worker => ['nwiger', 'rcwe', 'sfz'],
status => { '!=', 'completed' }
);
my($stmt, #bind) = $sql->select('tickets', '*', \%where);
The above would give you something like this:
$stmt = "SELECT * FROM tickets WHERE
( requestor = ? ) AND ( status != ? )
AND ( worker = ? OR worker = ? OR worker = ? )";
#bind = ('inna', 'completed', 'nwiger', 'rcwe', 'sfz');
Which you could then use in DBI code like so:
my $sth = $dbh->prepare($stmt);
$sth->execute(#bind);
Now, sometimes the order of the columns in the WHERE clause is very important, especially if you want to make good use of indexes.
But, since the columns to the WHERE clause generator in SQL-Abstract are specified by means of a hash - and as is known, the order that data is retrieved out of perl hashes cannot be guaranteed - you seem to loose the ability to specify the order of the columns.
Am i missing something? Is there an alternate facility to guarantee the order that columns appear in the WHERE clause when using SQL-Abstract ?
I originally misinterpreted your question.
You can use -and to achieve the desired ordering.
For example:
#!/usr/bin/perl
use strict; use warnings;
use SQL::Abstract;
my $sql = SQL::Abstract->new;
my ($stmt, #bind) = $sql->select(
tickets => '*',
{
-and => [
requestor => 'inna',
status => { '!=', 'completed' },
worker => ['nwiger', 'rcwe', 'sfz'],
],
}
);
print "$stmt\n";
See Nested conditions, -and/-or prefixes.
This module cannot do everything -- it is meant as a convenience for constructing queries that will do the job "most of the time". Sometimes you still may need to write a query by hand. I use SQL::Abstract in my main $work::app and have never run into the situation that you describe. A good SQL engine will know which keys are indexed, and optimize the query to use those first, no matter the ordering you specify. Are you sure that your engine is not the same, and that the order you specify in the query is really significant?
If you really need to order your WHERE clauses in a special order, you may find it easier to write subqueries instead. SQL::Abstract can make this easier too.
Related
Former sequelize person here.
What is the best method to extract data from MariaDB using a raw Knex statement? Here are the two methods i have tried:
MariaDb:
SELECT JSON_OBJECT(
...........
) 'JSON_OBJECT'
jscript to interpret results:
for ( tmp of result[0] ) { console.log (JSON.parse(tmp.JSON_OBJECT)) ; }
and this MariaDb:
SELECT JSON_ARRAYAGG(
JSON_OBJECT(
..........
)
) 'JSON_ARRAYAGG'
jscript:
for ( tmp of JSON.parse(result[0][0]['JSON_ARRAYAGG'] )) { console.log(tmp) ; }
Both of these methods work, but there may be a much cleaner way rather than using JSON.parse.
Suggestions?
note: i certainly understand there is a controversy using raw - i have a number of very large sql statements that were in a previous application, and i would rather (for now) just use them as is rather than rewrite them in pure Knex.
EDIT: this may be "good enough" since it appears that knex raw does not necessarily return objects.
node:
SELECT JSON_ARRAYAGG(
JSON_OBJECT(
..........
)
) 'JSON_ARRAYAGG'
.....................................
const result = await knex.raw( sqlStatement, sqlQuery);
return result[0][0].JSON_ARRAYAGG ;
browser:
(async () => { try { let result = await app.service('raw-service').find({query: sqlQuery}) ; newResult = result; } catch (e) { console.log(e); } } )() ;
console.log(JSON.parse(newResult));
but at least the really ugly stuff is hidden away, inside of node.
SQL databases are designed to return rows of columns; any query where you route everything through json instead is not going to be "the best method" you ask for.
Your first try at least uses rows, so is better than the second.
That said, if you are just modifying existing code to work with Knex rather than rewriting it, whatever requires minimum changes will lead to fewer bugs and should be preferred.
To extract data from a MariaDB database using Knex, you can use the .select() method on a Knex query builder object. This method allows you to specify the columns that you want to retrieve from the database, as well as any conditions or filters using the .where() method.
I want to query a table and only need one cell returned. Right now the only way I can think to do it is:
$query = $this->db->query('SELECT id FROM crops WHERE name = "wheat"');
if ($query->num_rows() > 0) {
$row = $query->row();
$crop_id = $row->id;
}
What I want is, since I'm select 'id' anyway, for that to be the result. IE: $query = 'cropId'.
Any ideas? Is this even possible?
Of course it's possible. Just use AND in your query:
$query = $this->db->query('SELECT id FROM crops WHERE name = "wheat" AND id = {$cropId}');
Or you could use the raw power of the provided Active Record class:
$this->db->select('id');
$this->db->from('crops');
$this->db->where('name','wheat');
$this->db->where('id',$cropId);
$query = $this->db->get();
If you just want the cropId from the whole column:
foreach ($query->result()->id as $cropId)
{
echo $cropId;
}
Try this out, I'm not sure if it will work:
$cropId = $query->first_row()->id;
Note that you want to swap your quotes around: use " for your PHP strings, and ' for your SQL strings. First of all, it would not be compatible with PostgreSQL and other database systems that check such things.
Otherwise, as Christopher told you, you can test the crop identifier in your query. Only if you define a string between '...' in PHP, the variables are not going to be replaced in the strings. So he showed the wrong PHP code.
"SELECT ... $somevar ..."
will work better.
Yet, there is a security issue in writing such strings: it is very dangerous because $somevar could represent some additional SQL and completely transform your SELECT in something that you do not even want to think about. Therefore, the Active Record as mentioned by Christopher is a lot safer.
I have this code for query:
$repository = $em->getRepository('AcmeCrawlerBundle:Trainings');
$query = $repository->createQueryBuilder('p')
->where('p.title LIKE :word')
->orWhere('p.discription LIKE :word')
->setParameter('word', $word)
->getQuery();
$trainings = $query->getResult();
The problem is: even if matches exist, they not found by this query. I used this code to see full sql:
print_r(array(
'sql' => $query->getSQL(),
'parameters' => $query->getParameters(),
));
And what I've got:
FROM Trainings t0_ WHERE t0_.title LIKE ? OR t0_.discription LIKE ? [parameters] => Array ( [word] => Spoken )
(last part of query)
Tell me please what to change?
You forgot the % signs around the word:
->setParameter('word', '%'.$word.'%')
Below are some additional steps you can take to further sanitise input data.
You should escape the term that you insert between the percentage signs:
->setParameter('word', '%'.addcslashes($word, '%_').'%')
The percentage sign '%' and the symbol underscore '_' are interpreted as wildcards by LIKE. If they're not escaped properly, an attacker might construct arbitrarily complex queries that can cause a denial of service attack. Also, it might be possible for the attacker to get search results he is not supposed to get. A more detailed description of attack scenarios can be found here: https://stackoverflow.com/a/7893670/623685
I have this site with the following parameters:
http://www.example.com.com/pagination.php?page=4&order=comment_time&sc=desc
I use the values of each of the parameters as a value in a SQL query.
I am trying to test my application and ultimately hack my own application for learning purposes.
I'm trying to inject this statement:
http://www.example.com.com/pagination.php?page=4&order=comment_time&sc=desc' or 1=1 --
But It fails, and MySQL says this:
Warning: mysql_fetch_assoc() expects parameter 1 to be resource,
boolean given in /home/dir/public_html/pagination.php on line 132
Is my application completely free from SQL injection, or is it still possible?
EDIT: Is it possible for me to find a valid sql injection statement to input into one of the parameters of the URL?
The application secured from sql injection never produces invalid queries.
So obviously you still have some issues.
Well-written application for any input produces valid and expected output.
That's completely vulnerable, and the fact that you can cause a syntax error proves it.
There is no function to escape column names or order by directions. Those functions do not exist because it is bad style to expose the DB logic directly in the URL, because it makes the URLs dependent on changes to your database logic.
I'd suggest something like an array mapping the "order" parameter values to column names:
$order_cols = array(
'time' => 'comment_time',
'popular' => 'comment_score',
... and so on ...
);
if (!isset($order_cols[$_GET['order'])) {
$_GET['order'] = 'time';
}
$order = $order_cols[$_GET['order']];
Restrict "sc" manually:
if ($_GET['sc'] == 'asc' || $_GET['sc'] == 'desc') {
$order .= ' ' . $_GET['sc'];
} else {
$order .= ' desc';
}
Then you're guaranteed safe to append that to the query, and the URL is not tied to the DB implementation.
I'm not 100% certain, but I'd say it still seems vulnerable to me -- the fact that it's accepting the single-quote (') as a delimiter and then generating an error off the subsequent injected code says to me that it's passing things it shouldn't on to MySQL.
Any data that could possibly be taken from somewhere other than your application itself should go through mysql_real_escape_string() first. This way the whole ' or 1=1 part gets passed as a value to MySQL... unless you're passing "sc" straight through for the sort order, such as
$sql = "SELECT * FROM foo WHERE page='{$_REQUEST['page']}' ORDER BY data {$_REQUEST['sc']}";
... which you also shouldn't be doing. Try something along these lines:
$page = mysql_real_escape_string($_REQUEST['page']);
if ($_REQUEST['sc'] == "desc")
$sortorder = "DESC";
else
$sortorder = "ASC";
$sql = "SELECT * FROM foo WHERE page='{$page}' ORDER BY data {$sortorder}";
I still couldn't say it's TOTALLY injection-proof, but it's definitely more robust.
I am assuming that your generated query does something like
select <some number of fields>
from <some table>
where sc=desc
order by comment_time
Now, if I were to attack the order by statement instead of the WHERE, I might be able to get some results... Imagine I added the following
comment_time; select top 5 * from sysobjects
the query being returned to your front end would be the top 5 rows from sysobjects, rather than the query you try to generated (depending a lot on the front end)...
It really depends on how PHP validates those arguments. If MySQL is giving you a warning, it means that a hacker already passes through your first line of defence, which is your PHP script.
Use if(!preg_match('/^regex_pattern$/', $your_input)) to filter all your inputs before passing them to MySQL.
I was wondering if there was a way to use "find_by_sql" within a named_scope. I'd like to treat custom sql as named_scope so I can chain it to my existing named_scopes. It would also be good for optimizing a sql snippet I use frequently.
While you can put any SQL you like in the conditions of a named scope, if you then call find_by_sql then the 'scopes' get thrown away.
Given:
class Item
# Anything you can put in an sql WHERE you can put here
named_scope :mine, :conditions=>'user_id = 12345 and IS_A_NINJA() = 1'
end
This works (it just sticks the SQL string in there - if you have more than one they get joined with AND)
Item.mine.find :all
=> SELECT * FROM items WHERE ('user_id' = 887 and IS_A_NINJA() = 1)
However, this doesn't
Items.mine.find_by_sql 'select * from items limit 1'
=> select * from items limit 1
So the answer is "No". If you think about what has to happen behind the scenes then this makes a lot of sense. In order to build the SQL rails has to know how it fits together.
When you create normal queries, the select, joins, conditions, etc are all broken up into distinct pieces. Rails knows that it can add things to the conditions without affecting everything else (which is how with_scope and named_scope work).
With find_by_sql however, you just give rails a big string. It doesn't know what goes where, so it's not safe for it to go in and add the things it would need to add for the scopes to work.
This doesn't address exactly what you asked about, but you might investigate 'contruct_finder_sql'. It lets you can get the SQL of a named scope.
named_scope :mine, :conditions=>'user_id = 12345 and IS_A_NINJA() = 1'
named_scope :additional {
:condtions => mine.send(:construct_finder_sql,{}) + " additional = 'foo'"
}
sure why not
:named_scope :conditions => [ your sql ]