A python library for SQL that I've used provided sanitation of query parameters to protect against injection attacks if you used the ('query == $0', param) approach, but not if you created the string yourself, i.e. ('query == param'). Is this the same in Realm for React Native?
Is there any functional difference between results.filtered(`query == ${param}`) and results.filtered('query == $0', param) ?
Yes, those are different, and filtered('query == $0', param) will work regardless of the contents of param, while filtered(`query == ${param}`) will not.
The substitution parameter syntax does not sanitize the input per se. The interface for the low-level query engine isn't string-based, so unlike with an SQL library there never is a query string with the parameters substituted in involved at all, so there isn't actually any sanitization needed. The end result is largely the same, though.
Related
Here I have a scala UDF that checks if a url is one of my domains. To check if 'to_site' is one of my domains, I'm using indexOf in javascript.
CREATE TEMPORARY FUNCTION our_domain(to_site STRING)
RETURNS BOOLEAN
LANGUAGE js AS """
domains = ['abc.com', 'xyz.com'];
if (to_site == null || to_site == undefined) return false;
for (var i = 0; i < domains.length; i++){
var q= DOMAIN('XYZ');
if (String.prototype.toLowerCase.call(to_site).indexOf(domains[i]) !== -1)
return true;
}
return false;
""";
SELECT our_domain('www.foobar.com'), our_domain('www.xyz.com');
This returns false, then true.
It would be much nicer if I could use the DOMAIN(url) function from javascript. indexOf is not very good because it will match www.example.com?from=www.abc.com, when actually example.com is not one of my domains. Javascript also has a (new URL('www.example.com/q/z')).hostname to parse the domain component, but it includes the subdomain like 'www.' which complicates the comparison. Bigquery's DOMAIN(url) function only gives the domain and knowing google it's fast C++.
I know I can do this
our_domain(DOMAIN('www.xyz.com'))
But in general it would be nice to use some of the bigquery API functions in javascript. Is this possible?
I also tried this
CREATE TEMPORARY FUNCTION our_domain1(to_site String)
AS (our_domain(DOMAIN(to_site));
but it fails saying DOMAIN does not exist.
DOMAIN() function is supported in BigQuery Legacy SQL whereas Scalar UDFs are part of BigQuery Standard SQL.
So, unfortunatelly, no, you cannot use DOMAIN() function with code that uses Scalar UDF as of now at least.
And, no, you cannot use SQL Functions within JS [Scalar] UDFs, but you can use them in SQL UDFs
Finally, as I suggested in my answer on your previous question - in somple scenario like this your particular one - you better use SQL Scalar SQL vs. JS Scalar UDFs - they do not have LIMITs that JS UDFs have
The DOMAIN function in legacy SQL is more or less just a regular expression. Have you seen this previous question about DOMAIN? As Mikhail points out, you should be able to define a SQL UDF that uses a regex to extract the domain and then checks if it's in your list.
I have an application for which I need to allow the user to perform full text search on documents, and use the Lucene Query Parser syntax if desired. The eXist database is queried from a Django backend that uses eulexistdb to talk to eXist.
The problem is that when the user uses an incorrect syntax for the full text search, this is discovered late in the game. The Django application has to query a SQL database to determine some of the parameters of the search. By the time the complete XQuery is built and eXist is accessed, the SQL query has already run, which means that the cost of the SQL query has already been spent. (I know I could marshal the data queried on the SQL side into eXist so that only eXist is queried. It's just not an option for now.)
I'd like to know ahead of time whether the Lucene query has a syntactical error to that I can avoid starting querying the SQL database for nothing.
I've checked the documentation of eXist, but I've not found anything in the API which would be a simple function that checks whether a full-text query is syntactically valid or not.
Here is a simple function that will return True if a Lucene query is fine, or False if there is a syntax error in the query. db must be an instance of eulexistdb.db.ExistDB and query is the Lucene query:
def check(db, query):
try:
db.query(safe_interpolate("ft:query(<doc/>, {lucene_query})",
lucene_query=query))
except ExistDBException as ex:
if ex.message().startswith(
"exerr:ERROR Syntax error in Lucene query string"):
return False
raise ex # Don't swallow other problems that may occur.
return True
This should be adaptable to any language for which there is a library that provides access to eXist. The idea is to run the query of interest against a bogus document (<doc/>). Using the bogus document avoids having to actually search the database. (An empty node sequence might seem better, but we're not running ft:query against an empty node sequence because then the XQuery optimizer could skip trying to parse and run the Lucene query since a valid query on an empty sequence will necessarily return an empty sequence, irrespective of the actual Lucene query.) It does not matter whether it returns any results or not. If the query has no errors, then there won't be an exception. If the query has a syntax error, then an exception will be raised. I've not found a more robust way than checking the error message stored with the exception to detect whether it is a Lucene syntax error or something else.
(The safe_interpolate function is a function that should interpolate lucene_query so as to avoid injections. It is up to you to decide what you need in your application.)
Here is an approach I consider complementary to the one I posted earlier. I'm using lucene-query-parser to perform the check client-side (i.e. in the browser):
define(function (require, exports, _module) {
"use strict";
var lqp = require("lucene-query-parser");
function preDrawCallback() {
// We get the content of the search field.
var search = this.api().search();
var good = true;
try {
lqp.parse(search); // Here we check whether it is syntactically valid.
}
catch (ex) {
if (!(ex instanceof lqp.SyntaxError)) {
throw ex; // Don't swallow exceptions.
}
good = false;
}
// Some work is performed here depending on whether
// the query is good or bad.
return good; // And finally we tell DataTables whether to inhibit the draw.
}
// ....
});
preDrawCallback is used with a DataTables instance. Returning false inhibits drawing the table, which also inhibits performing a query to the server. So if the query is syntactically incorrect, it won't ever make it to the backend. (The define and require calls are there because both my code and lucene-query-parser are AMD modules.)
Potential issues:
If the library that performs the check is buggy or otherwise does not support the entire syntax that Lucene supports, it will block queries that should go through. I've found a few buggy (or at best severely obsolete) libraries before I settled on lucene-query-parser.
If the client-side library happens to support a construct introduced in a later version of Lucene but which is not supported in the version used with eXist. Keeping the backend check I show in my other answer allows to make sure that anything that would slip through is caught there.
I have an EF Code First Db context that I'm using to query the database. I noticed some performance issues when passing in queries as Func<Product, bool>s from my Aggregate Repository and on investigating further it turned out that the queries were not being translated into SQL Queries.
After a little more digging I discovered the following.
var results = _context.Products
.Where(p => p.ProductCode.Contains("AAA"))
.Where(p => p.CategoryId == 1)
.ToList();
This works exactly as expected. It generates some parametrized SQL with a Where Clause.
==================================================================
var results2 = _context.Products
.Where(p => p.ProductCode.Contains("AAA") && p.CategoryId == 1)
.ToList();
This also works as expected. It generates the same sql as above
==================================================================
Func<Product, bool> pred = (p => p.ProductCode.Contains("AAA") && p.CategoryId == 1);
var results3 = _context.Products.Where(pred).ToList();
This is broken. It doesn't generate the where clause in the SQL, it returns everything and then filters it in code.
Because in order to translate into SQL, it has to be an Expression<...>, not a Func<...>.
This is done automatically for you by the compiler, and since the overloads on the Linq-to-SQL classes takes expressions, not delegates, the compiler will automagically translate your code (which looks like a lambda or an anonymous method) into an expression object and pass that.
However, if you take care of building the function yourself, the compiler cannot do this, and Linq-to-SQL does not take anonymous methods, it only takes expressions.
What you can do is to execute the parts of your query that you can, and then filter the results through your function, but I would look into just changing the type of your value into an expression instead.
No sooner than I posted this ReSharper helped answer my question by showing me the overload method signature for the Where() extension method.
It takes both Func<T, bool> and Expression<Func<T, bool>>. If your declaring your predicates externally, you must use the Expression variation as the former is not translated into sql.
Here's why the query reads the whole table.
When a Func is used instead of an Expression, the compiler chooses methods on System.Linq.Enumerable - instead of System.Linq.Queryable. The Enumerable methods iterate the source collection (sometimes lazily) while the Queryable methods build up the expression tree.
Since the call to Where isn't part of the expression tree, the sql generator doesn't see it during the query translation.
i have a question regarding operator overloading in cli/c++ environment
static Length^ operator++(Length^ len)
{
Length^ temp = gcnew Length(len->feet, len->inches);
++temp->inches;
temp->feet += temp->inches/temp->inchesPerFoot;
temp->inches %= temp->inchesPerFoot;
return temp;
}
(the code is from ivor horton's book.)
why do we need to declare a new class object (temp) on the heap just to return it?
ive googled for the info on overloading but theres really not much out there and i feel kinda lost.
This is the way operator overloading is implemented in .NET. Overloaded operator is static function, which returns a new instance, instead of changing the current instance. Therefore, post and prefix ++ operators are the same. Most information about operator overloading talks about native C++. You can see .NET specific information, looking for C# samples, for example this: http://msdn.microsoft.com/en-us/library/aa288467(v=vs.71).aspx
.NET GC allows to create a lot of lightweight new instances, which are collected automatically. This is why .NET overloaded operators are more simple than in native C++.
Yes, because you're overloading POST-increment operator here. Hence, the original value may be used a lot in the code, copied and stored somewhere else, despite the existance of the new value. Example:
store_length_somewhere( len++ );
While len will be increased, the original value might be stored by the function somewhere else. That means that you might need two different values at the same time. Hence the creation and return of a new value.
This is probably fairly straightforward but i can't seem to find a reasonable explanation in any documentation.
I'm trying to use an NHibernate.ISQLQuery and using SetResultTransformer() to return a custom set of results from a custom SQL query. Like so:
public virtual IList<T> GetSQLObject<T>(string sql, IDbParameter[] parameters = null)
{
ISQLQuery qry = _sess.CreateSQLQuery(sql);
qry.SetResultTransformer(Transformers.AliasToBean(typeof(T)));
if (parameters != null) {
foreach (IDbParameter parameter in parameters) {
qry.SetParameter(parameter.Name, parameter.Value);
}
}
return qry.List<T>();
}
From looking at the examples, it seems that in the sql query I have to use parameters in the format :param1 instead of #param1 as I would in a standard SQL query. If i use the latter syntax in the query, it throws an error at qry.SetParameter().
Is there a reason why ISQLQuery/NHibernate requires them in this format and won't work with the normal syntax?
SQL Server uses #param, but not every other database does. For example, MySQL uses ?param
NHibernate allows you to swap out 1 database implementation for another with little to no reworking of your DAL. It sets the parameters based on the database you configured when you setup the NH Configuration.
Edit: Also I think :param came about from Hibernate being targeted at Oracle when it was initially developed, since Oracle uses :param
Phil has answered the "why"; so perhaps I can recommend a "how"; why not just add a new extension method to the IDbParameter type (something like .GetNHibernateName() ) that will return the parameter name with the "#" replaced with a ":"; that should be trivial to implement.