I have a PostgreSQL RPC that aims to select filtered rows of a view.
This RPC requires some parameters (name_article, catg_article, color_article, etc).
Most of these parameters are int[]/bigint[] because I want the user to be able to request "all blue articles or all red articles, etc" but I want the user to be able to post empty parameters as well, and that the request considers he doesn't care about which color or category so it will return all possibilities.
The problem is that from what I saw after many topics on Internet, the ANY () or IN () can't be empty, which I'd like to allow it otherwise my filters system would have to manage all possibilities and I really don't want to cry.
This is what I've readen on Internet to try ( param is null or in()/any() ) but it doesn't work, not returning any article (the first where is fine, also don't pay attention to the cast thing, it's just that catg_and_type is json so I have to say id_catgarticle from this json is a bigint so it works fine) :
SELECT *
FROM dev.get_all_articles
WHERE get_all_articles.lib_article ILIKE '%' || $1 || '%'
AND ($2 is null or CAST(get_all_articles.catg_et_type->>'id_catgarticle' AS BIGINT) = any ($2));
Do you have any idea how I could allow empty arrays that will be processed with IN/ANY commands ?
Thanks a lot.
Problem solved, as mentionned into my answer to #LaurenceIsla's answer to the topic.
When having to send an array parameter into a PostgREST API endpoint, the syntax is like this : /rpc/endpoint?param={1,2,3}. So in order to make the request understand an empty param in URL (endpoint?param={}), I had to say, in the WHERE clause this : OR $2 = '{}'. That's all. Kind of tricky syntax when you don't know it.
Related
I am using the Google Analytics API to automatically fetch stats from eccomerce sites. I need to query a dynamic segment with the sessions that spent more than 0 and less than 50USD in ecommerce.
I tried this:
segment=users::condition::perSession::ga:transactionRevenue>0;users::condition::perSession::ga:transactionRevenue<50
But it looks like the API is ignoring the ga:transactionRevenue < 50 condition, returning all the sessions with ga:transactionRevenue>0. I tried some other metrics in the > 0 condition ( like uniquePurchases , ga:transactionTax...) with the same results.
The fun part is that if I use transactionShipping it works OK ( returning the sessions with purchases including shipping costs and with less than 50USD revenue):
segment=users::condition::perSession::ga:transactionShipping >0;users::condition::perSession::ga:transactionRevenue<50
But this is not OK, because I need to include the free shippingtransactions on the segment.
Anybody nows a workarround for this?
Check how you are processing the segment for sending to the API.
The rule is that you should escape a semicolon in a value expression (\;). I suspect you are escaping the semicolon between the conditions by accident because you encode/escape everything after the 'segment=' like segment=<encoded/escaped segment definition>
What you need to to send segment=<encoded/escaped condition1>;<encoded/escaped condition2>
I am using twilio to provide audio conference functionality in my rails app. When I call my conference number, twilio passes on a couple of values - including 'From' which contains the caller's phone number in international format.
I have a profile for every user in my system and in my controller I am querying the profile to provide a personalised welcome message. Every profile contains between 0 and 3 numbers (primary, secondary and cellphone) and I need to check the caller's ID against those three fields in all profiles.
When I use the console on my dev machine, the following code finds the correct profile:
Profile.find_by('+44000000000')
When I upload to heroku, I use following code instead:
name = Profile.find_by(params['From']) || 'there'
Which causes an error in my app:
2014-04-03T19:20:22.801284+00:00 app[web.1]: PG::DatatypeMismatch: ERROR: argument of WHERE must be type boolean, not type bigint
2014-04-03T19:20:22.801284+00:00 app[web.1]: LINE 1: SELECT "profiles".* FROM "profiles" WHERE (+4400000000) ...
Any suggestion how that could be solved?
Thanks!
Additional information:
I think my problem is that I don't know how to query either the whole profile or three columns at once. Right now the code:
name = Profile.find_by(params['From'])
is not correct (params['From'] contains a phone number) because I am not telling rails to query the columns primary phone number, secondary phone number and cellphone. Neither am I querying the whole profile which would also be an option.
So the question basically is:
How can I change this code:
Profile.find_by(params['From'])
so that it queries either all fields in all profiles or just the three columns with phone numbers which each profile contains?
Is there something like Profile.where(:primary_number).or.where(:secondary_number)or.where(:cellphone) => params['From']
?
I am not familiar with twilio and not sure if this helps but find and find_by_attribute_name accepts array of values as options:
name = Profile.find_by([params['From'], 'there'] )
suppose params['From'] was here , This should generate:
SELECT `profiles`.* FROM `profiles` WHERE `profiles`.`attribute` IN ('here', 'there')
Or:
If you are trying to build dynamic matcher at run time , which is called Meta-programming , you can try this code:
name = eval("Profile.find_by_#{params['From']) || 'there'}(#rest of query params here) ")
Update
First of all, i think you are not using find_by correctly!! the correct syntax is:
Model.find_by(attribute_name: value)
#e.g
Profile.find_by(phone_number: '0123456')
Which will call where and retrive one record, but passing a value will generate a condition that always passes, for example:
Model.find_by('wrong_condition')
#will generate SQL like:
SELECT `models`.* FROM `models` WHERE ('wrong_condition') LIMIT 1
#which will return the first record in the model since there is no valid condition here
Why don't you try:
Profile.where('primary_number = ? OR secondary_number = ? OR cellphone = ?', params['From'], params['From'], params['From'])
You can write your query like:
Profile.where("primary_number = ? or secondary_number = ? or cellphone = ?", params['From'])
Just double check the syntax, but that should do it.
I have a small index with ~1000 documents with only two fields:
- id (string)
- content (text_general)
I noticed that when I do MLT search by id for similar content, the original document(which id is the searched id) have a score 5.241327.
There is 1:1 duplicated document and for the duplicated content it is returning score = 1.5258181. Why? Why it is not 5.241327 when it is 100% duplicate.
Another question is can I in any way to get similarity documents by content by passing some text in the query.
Example:
/mlt/?q=content:Some encoded long text&mlt.fl=content
I am trying to check if there is similar content uploaded and the check must be performed at new content upload time.
It might be worth to try some different parameters. I also use MLT on only one field, I use the following parameters:
'mlt.boost': 'true',
'mlt.fl': 'my_field_name',
'mlt.maxqt': 1000,
'mlt.mindf': '0',
'mlt.mintf': '0',
'qt': 'mlt',
'rows': '10'
See http://wiki.apache.org/solr/MoreLikeThis for an explanation of the parameters. I think with a small index mindf might be important and I see the default mintf (term frequency) is 2, so I assume an ID is only one term, so this is probably ignored!
First, how does Solr More-Like-This works?
A regular Solr query is conducted (e.g. "?q=content:Some encoded long text&.....".
For each document returned by the above query, More-Like-This conduct More like this query...
So, the first result set "response", is just like any Solr query results set.
The More-Like-This appears below and start with something like that (Json format):
"moreLikeThis":{
"57375":{"numFound":18155,"start":0,"docs":["
For an explanation about More Like This algorithm, please read that:
http://blog.brattland.no/node/18
and: http://cephas.net/blog/2008/03/30/how-morelikethis-works-in-lucene/
If you didn't solved the problem yet, please let me know and I will guide you through.
I have this code for query:
$repository = $em->getRepository('AcmeCrawlerBundle:Trainings');
$query = $repository->createQueryBuilder('p')
->where('p.title LIKE :word')
->orWhere('p.discription LIKE :word')
->setParameter('word', $word)
->getQuery();
$trainings = $query->getResult();
The problem is: even if matches exist, they not found by this query. I used this code to see full sql:
print_r(array(
'sql' => $query->getSQL(),
'parameters' => $query->getParameters(),
));
And what I've got:
FROM Trainings t0_ WHERE t0_.title LIKE ? OR t0_.discription LIKE ? [parameters] => Array ( [word] => Spoken )
(last part of query)
Tell me please what to change?
You forgot the % signs around the word:
->setParameter('word', '%'.$word.'%')
Below are some additional steps you can take to further sanitise input data.
You should escape the term that you insert between the percentage signs:
->setParameter('word', '%'.addcslashes($word, '%_').'%')
The percentage sign '%' and the symbol underscore '_' are interpreted as wildcards by LIKE. If they're not escaped properly, an attacker might construct arbitrarily complex queries that can cause a denial of service attack. Also, it might be possible for the attacker to get search results he is not supposed to get. A more detailed description of attack scenarios can be found here: https://stackoverflow.com/a/7893670/623685
I have this site with the following parameters:
http://www.example.com.com/pagination.php?page=4&order=comment_time&sc=desc
I use the values of each of the parameters as a value in a SQL query.
I am trying to test my application and ultimately hack my own application for learning purposes.
I'm trying to inject this statement:
http://www.example.com.com/pagination.php?page=4&order=comment_time&sc=desc' or 1=1 --
But It fails, and MySQL says this:
Warning: mysql_fetch_assoc() expects parameter 1 to be resource,
boolean given in /home/dir/public_html/pagination.php on line 132
Is my application completely free from SQL injection, or is it still possible?
EDIT: Is it possible for me to find a valid sql injection statement to input into one of the parameters of the URL?
The application secured from sql injection never produces invalid queries.
So obviously you still have some issues.
Well-written application for any input produces valid and expected output.
That's completely vulnerable, and the fact that you can cause a syntax error proves it.
There is no function to escape column names or order by directions. Those functions do not exist because it is bad style to expose the DB logic directly in the URL, because it makes the URLs dependent on changes to your database logic.
I'd suggest something like an array mapping the "order" parameter values to column names:
$order_cols = array(
'time' => 'comment_time',
'popular' => 'comment_score',
... and so on ...
);
if (!isset($order_cols[$_GET['order'])) {
$_GET['order'] = 'time';
}
$order = $order_cols[$_GET['order']];
Restrict "sc" manually:
if ($_GET['sc'] == 'asc' || $_GET['sc'] == 'desc') {
$order .= ' ' . $_GET['sc'];
} else {
$order .= ' desc';
}
Then you're guaranteed safe to append that to the query, and the URL is not tied to the DB implementation.
I'm not 100% certain, but I'd say it still seems vulnerable to me -- the fact that it's accepting the single-quote (') as a delimiter and then generating an error off the subsequent injected code says to me that it's passing things it shouldn't on to MySQL.
Any data that could possibly be taken from somewhere other than your application itself should go through mysql_real_escape_string() first. This way the whole ' or 1=1 part gets passed as a value to MySQL... unless you're passing "sc" straight through for the sort order, such as
$sql = "SELECT * FROM foo WHERE page='{$_REQUEST['page']}' ORDER BY data {$_REQUEST['sc']}";
... which you also shouldn't be doing. Try something along these lines:
$page = mysql_real_escape_string($_REQUEST['page']);
if ($_REQUEST['sc'] == "desc")
$sortorder = "DESC";
else
$sortorder = "ASC";
$sql = "SELECT * FROM foo WHERE page='{$page}' ORDER BY data {$sortorder}";
I still couldn't say it's TOTALLY injection-proof, but it's definitely more robust.
I am assuming that your generated query does something like
select <some number of fields>
from <some table>
where sc=desc
order by comment_time
Now, if I were to attack the order by statement instead of the WHERE, I might be able to get some results... Imagine I added the following
comment_time; select top 5 * from sysobjects
the query being returned to your front end would be the top 5 rows from sysobjects, rather than the query you try to generated (depending a lot on the front end)...
It really depends on how PHP validates those arguments. If MySQL is giving you a warning, it means that a hacker already passes through your first line of defence, which is your PHP script.
Use if(!preg_match('/^regex_pattern$/', $your_input)) to filter all your inputs before passing them to MySQL.