I am using Apache Calcite to execute queries on different data sources.
The model file that I am using is
inline: {
version: '1.0',
defaultSchema: 'sakila',
schemas: [
{
name: 'sakila',
type: 'custom',
factory: 'org.apache.calcite.adapter.jdbc.JdbcSchema$Factory',
operand: {
jdbcDriver: 'org.postgresql.Driver',
jdbcUrl: 'jdbc:postgresql://localhost:5432/sakila',
jdbcUser: 'postgres',
jdbcPassword: 'postgres'
}
}
]
}
And the query is
select
"sakila"."actor"."first_name" as "actor_first_name"
from
"sakila"."actor"
The above query is not working due to the double quotes applied for tables and columns. So, I had to remove the quotes and the following query works fine.
select
sakila.actor.first_name as actor_first_name
from
sakila.actor
Here, the question is the query parser is not allowing some queries if they don't have double quotes. And in some cases like above it is not requiring quotes to execute properly. Can anyone throw some insight on why exactly it is so?
I suspect that the cause is case-sensitivity. Assuming that Calcite is in its default lexical mode, if you remove the quotes around identifiers, Calcite will convert them to upper case before trying to find tables and columns with those names. You say that it works without quotes, so I presume that your schema, table and column are upper case (SAKILA, ACTOR, FIRST_NAME).
You can ask Calcite to be non-case-sensitive by passing caseSensitive=false as part of the Calcite JDBC connect string.
Related
I need make a query for a postgresdb without identify accents (á, í,ö, etc).
I'm already use Knex.js as query builder, and postgresql have a unaccent extension that works fine in sql querys directly to db, but in my code i use knex and unaccent function throws error in querys.
Can anyone help me, ¿is possible make querys with knex.js that use unaccent function of postgresql?
My solution is to process the string before submitting the query using the following code:
const normalize = (str) => str.normalize('NFD').replace(/[\u0300-\u036f]/g, '');
console.log(normalize('Ấ Á Ắ Ạ Ê')) -> 'A A A A A'.
Or if you use postgresql version 13 or later it already supports that functionality.
select normalize('hồ, phố, ầ', NFC) → 'ho, pho, a' -- NFC (the default), NFD, NFKC, or NFKD.
Document: https://www.postgresql.org/docs/13/functions-string.html
I'm attempting to query against a materialized path stored with postgres ltree type from a Grails application. Unfortunately, my query uses the "?" operator which is being captured by GORM as a parameter
sqlRestriction("materialized_path ? (SELECT ARRAY(SELECT CAST(CAST(subpath(?,0,generate_series) AS text) ||'.*{1}' AS lquery) FROM generate_series(1,nlevel(CAST(? AS lquery)))))"
,[vertex.materializedPath,vertex.materializedPath])
Where that first question mark should be escaped and the error being thrown is
org.postgresql.util.PSQLException: No value specified for parameter 4.
at org.postgresql.core.v3.SimpleParameterList.checkAllParametersSet(SimpleParameterList.java:246)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:272)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:430)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:356)
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:168)
at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:116)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:70)
Found it myself with a little experimentation. It just takes a double-question mark. So,
"materialized_path ? (SELECT ARRAY(...
becomes
"materialized_path ?? (SELECT ARRAY(
I have a SQL column that contains JSON payloads. I'm writing a SQL query that will extract part of that column w/ regex.
In the following field:
{
"myContent":{
"fieldG":null,
"valuable":"this is the text",
"fieldH":[
"a4a6ba1c2e0e4a9c89dac46f1092b505"
],
"fieldI":"1"
},
"fieldJ":"1441375349399"
}
I want to scrape the string immediately after "valuable":
I was experimenting w/ something like this:
\"valuable\":\".*\"
... but that doesn't help. For starters, I only want what's AFTER valuable and colon. Also, that particular regex matches from "valuable:" all the way through the entire end of the line. That's way more text than I need.
In SQL Server 2016 CTP3 you could use JSON_VALUE function, e.g.:
SELECT JSON_VALUE(column, '$.mycontent.valuable').
See http://blogs.msdn.com/b/jocapc/archive/2015/05/16/json-support-in-sql-server-2016.aspx
If you cannot wait for Sql Server 2016, you can use some JSON CLR library e.g. http://www.codeproject.com/Articles/1000953/JSON-for-SQL-Server-Part
Otherwise yo would need to use PATINDEX and substring.
I've a problem with escaping colons and dashes in SQL queries when I use dbFit with Fitnesse.
Such statement doesn't work:
!|Query|select to_char(my_birthday,'YYYY-MM-DD HH24:MI:SI') from family|
I need to replace colons and dashes with some other acceptable characters, ex.
!|Query|select to_char(my_birthday,'YYYY_MM_DD HH24_MI_SI') from family|
Do you know how to solve it properly without using the 2nd approach ?
Cheers,
foxrafi
I think this is what you need. From http://dbfit.github.io/dbfit/docs/reference.html
Avoiding parameter mapping
If you want to prevent DbFit from mapping parameters to bind variables
(eg to execute a stored procedure definition that contains the #
symbol in Sql Server), disable bind symbols option before running the
query.
|set option|bind symbols|false|
|execute| insert into users (name, username) values ('#hey','uuu')|
|query|select * from users| |name|username| |#hey|uuu|
Remember to re-enable the option after the query is executed. You can
use the same trick with the Execute command.
In addition to Mike's answer, you can also solve this by using bind parameters. This is useful if you have to use bind parameters at other places in the same query.
!|DatabaseEnvironment|ORACLE|
|Connect|${HOSTNAME}|${USERNAME}|${PASSWORD}|
!|Query|!-select current_timestamp ts from dual-!|
|ts?|
|>>my_birthday_ts|
|set parameter|my_birthday_ts| <<my_birthday_ts|
#Set bind parameter :MI to string ':MI'
|set parameter|MI|:MI|
# and do it in the same way with :SS.
|set parameter|SS|:SS|
!|Query|!-select to_char(:my_birthday_ts, 'YYYY-MM-DD HH24'|| :MI || :SS) bds from dual-!|
|bds?|
|>>birthday_string|
Note that you have to use !- -! around your query, otherwise FitNesse will expand the concatenation operator to table cells. The main drawback of this manner is that you cannot use ordinary FitNesse variables (${varname}) in the query.
!|DatabaseEnvironment|ORACLE|
|Connect|${HOSTNAME}|${USERNAME}|${PASSWORD}|
!|Query|!-select current_timestamp ts from dual-!|
|ts?|
|>>my_birthday_ts|
!|Query|!-select to_char(:my_birthday_ts, 'YYYY-MM-DD HH24:'||'MI:'||'SS) bds from dual-!|
|bds?|
|>>birthday_string|
I have this site with the following parameters:
http://www.example.com.com/pagination.php?page=4&order=comment_time&sc=desc
I use the values of each of the parameters as a value in a SQL query.
I am trying to test my application and ultimately hack my own application for learning purposes.
I'm trying to inject this statement:
http://www.example.com.com/pagination.php?page=4&order=comment_time&sc=desc' or 1=1 --
But It fails, and MySQL says this:
Warning: mysql_fetch_assoc() expects parameter 1 to be resource,
boolean given in /home/dir/public_html/pagination.php on line 132
Is my application completely free from SQL injection, or is it still possible?
EDIT: Is it possible for me to find a valid sql injection statement to input into one of the parameters of the URL?
The application secured from sql injection never produces invalid queries.
So obviously you still have some issues.
Well-written application for any input produces valid and expected output.
That's completely vulnerable, and the fact that you can cause a syntax error proves it.
There is no function to escape column names or order by directions. Those functions do not exist because it is bad style to expose the DB logic directly in the URL, because it makes the URLs dependent on changes to your database logic.
I'd suggest something like an array mapping the "order" parameter values to column names:
$order_cols = array(
'time' => 'comment_time',
'popular' => 'comment_score',
... and so on ...
);
if (!isset($order_cols[$_GET['order'])) {
$_GET['order'] = 'time';
}
$order = $order_cols[$_GET['order']];
Restrict "sc" manually:
if ($_GET['sc'] == 'asc' || $_GET['sc'] == 'desc') {
$order .= ' ' . $_GET['sc'];
} else {
$order .= ' desc';
}
Then you're guaranteed safe to append that to the query, and the URL is not tied to the DB implementation.
I'm not 100% certain, but I'd say it still seems vulnerable to me -- the fact that it's accepting the single-quote (') as a delimiter and then generating an error off the subsequent injected code says to me that it's passing things it shouldn't on to MySQL.
Any data that could possibly be taken from somewhere other than your application itself should go through mysql_real_escape_string() first. This way the whole ' or 1=1 part gets passed as a value to MySQL... unless you're passing "sc" straight through for the sort order, such as
$sql = "SELECT * FROM foo WHERE page='{$_REQUEST['page']}' ORDER BY data {$_REQUEST['sc']}";
... which you also shouldn't be doing. Try something along these lines:
$page = mysql_real_escape_string($_REQUEST['page']);
if ($_REQUEST['sc'] == "desc")
$sortorder = "DESC";
else
$sortorder = "ASC";
$sql = "SELECT * FROM foo WHERE page='{$page}' ORDER BY data {$sortorder}";
I still couldn't say it's TOTALLY injection-proof, but it's definitely more robust.
I am assuming that your generated query does something like
select <some number of fields>
from <some table>
where sc=desc
order by comment_time
Now, if I were to attack the order by statement instead of the WHERE, I might be able to get some results... Imagine I added the following
comment_time; select top 5 * from sysobjects
the query being returned to your front end would be the top 5 rows from sysobjects, rather than the query you try to generated (depending a lot on the front end)...
It really depends on how PHP validates those arguments. If MySQL is giving you a warning, it means that a hacker already passes through your first line of defence, which is your PHP script.
Use if(!preg_match('/^regex_pattern$/', $your_input)) to filter all your inputs before passing them to MySQL.