Is it possible to use a JOIN clause when using a raw SQL ? If so, how should I map the result as there are at least 2 models involved ?
you can use any clause your database provides. When running a non-standard query, use method Model.findBySQL(...).
However, before using, read its documentation:
http://javalite.github.io/activejdbc/snapshot/org/javalite/activejdbc/Model.html#findBySQL-java.lang.String-java.lang.Object...-
Specifically, this:
Ensure that the query returns all columns associated with this model, so that the resulting models could hydrate themselves properly. Returned columns that are not part of this model will be ignored, but can be used for clauses like above.
In other words, whatever query you execute, the columns it returns should match that of this model. Technically speaking, your query can span multiple tables, views, even execute custom functions, but the resultset needs to have the columns that match this model. Why is that? Because your model is 'O' in the ORM. In other words, it represents a single record from a table as an object.
If your results are a mix of more than one model, or a partial model, or anything in between, do not use a model API. Use Base:
List<Map> results = Base.findAll("complex custom query");
http://javalite.github.io/activejdbc/snapshot/org/javalite/activejdbc/Base.html#findAll-java.lang.String-
Look at variations of find(...) and findAll(...) methods
Related
I have this type of model(schema)
1 product hasOffer (many) Offers
1 offer hseRule (many) shipment rules
like
Product(1)--->Offer(N)----->Rules(M)
How can I query
one product with all offers and all shipping rules.
In simple words.
How can I query one-many related records ?
This can be achieved using a simple SPARQL query with multiple triples in the WHERE clause. This works because the graph model over which SPARQL queries naturally joins data together.
eg.
SELECT ?product ?offer ?rules
WHERE { {?product ns:hasOffer ?offer} {?offer ns:hasRules ?rules} }
Source: Running the same example in Protege with this query.
EDIT:
Question: is there any way to construct as product{ offers:{ rules:{} }, productAttributes } ?
Answer: Yes there is, you can add ?attribute to the SELECT clause the triple ?product ns:productAttribute ?attribute to the WHERE clause (or multiple such clauses if you have attributes across multiple data properties). However I would strongly recommend you refrain from doing so. The query is going to return a potentially large set of data similar to an SQL result set. Thus you are going to see multiple rows with the same product and having a product attribute there will make your life unnecessarily difficult, since that attribute will appear in all rows with that product. Instead run a separate query where you only get the products and their attributes.
If we take a database table, we can query all the rows or we can choose to apply a filter on it. The filter can vary depending on the user input. In cases when there are few options we can specify different queries for those few specific conditions. But if there are lots and lots of options that user might or might not specify, aforementioned method does not come handy. I know, I can compose the filter based upon the user input and send it as a string to the corresponding stored procedure as a parameter, build the query with that filter and finally execute the query string with the help of EXECUTE IMMEDIATE(In Oracle's case). Don't know why but I really don't like this way of query building. I think this way I leave the doors open for SQL injectors. And besides, that I always have trouble with the query itself as everything is just a string and I need to handle dates and numbers carefully.What is the best and most used method of forming the WHERE clause of a query against a database table?
Using database parameters instead of attempting to quote your literals is the way forward.
This will guard you against SQL injection.
A common way of approaching this problem is building expression trees that represent your query criteria, converting them to parameterized SQL (to avoid SQL injection risks), binding parameter values to the generated SQL, and executing the resultant query against your target database.
The exact approach depends on your client programming framework: .NET has Entity Framework and LINQ2SQL that both support expression trees; Java has Hibernate and JPA, and so on. I have seen several different frameworks used to construct customizable queries, with great deal of success. In situations when these frameworks are not available, you can roll your own, although it requires a lot more work.
I'm currently implementing a Solr solution where a user is able to select various options to search for a product. I can now take all those options and put them together into one single long query, or I can use a query that fetches everything (*:*) and applies query filters to it.
Regular query:
q=color:blue AND price:500
Query using filter queries:
q=*:*&fq=color:blue&fq=price:500
The result is exactly the same. So what is the difference? When should I use one or the other?
Filter queries do not influence scores of the document.
Further they are useful in Caching, the queries specified with fq are cached independently from the main query
Document for solr query parameters
Typically in any production system you would use a variant of the Dismax request handler which doesn't support the former syntax, hence filtering must be performed using filter queries in that case.
I am a newbie in using functions and it appears to me that an inline function is very similar to a view. Am I correct?
Also, can I have UPDATE statements within a function?
After reading many of the answers here, I'd like to note that there is a big difference between an inline table-valued function and any other kind of function (scalar or multi-line TVF).
An inline TVF is simply a parameterized view. It can be expanded and optimized away just like a view. It is not required to materialize anything before "returning results" or anything like that (although, unfortunately, the syntax has a RETURN.
A big benefit I've found of an inline TVF over a view is that it does force required parameterization whereas with a view, you have to assume that the caller will appropriately join or restrict the usage of the view.
For example, we have many large fact tables in DW with a typical Kimball star model. I have a view on a fact table-centered model, which called without any restriction, will return hundreds of millions of rows. By using an inline TVF with appropriate parameterization, users are unable to accidentally ask for all the rows. Performance is largely indistinguishable between the two.
No difference. They are both expanded/unnested into the containing query.
Note: indexed views are considered differently but still may be expanded, and multi-valued table functions are black boxes to the containing query.
Tony Rogerson: VIEWS - they offer no optimisation benefits; they are simply inline macros - use sparingly
Adam Machanic: Scalar functions, inlining, and performance: An entertaining title for a boring post
Related SO question: Does query plan optimizer works well with joined/filtered table-valued functions?
Scary DBA (at the end)
Finally, writes to tables are not allowed in functions
Edit, after Eric Z Beard's comment and downvote...
The question and answers (not just mine) are not about scalar udfs. "Inline" means "inline table valued functions". Very different concepts...
Update: Looks like I missed the "inline" part. However, I'm leaving the answer here in case someone wants to read about difference between VIEWs and regular functions.
If you only have a function that does SELECT and output the data, then they are similar. However, even then, they are not the same because VIEWs can be optimized by the engine. For example, if you run SELECT * FROM view1 WHERE x = 10; and you have index on table field that maps to X, then it will be used. On the other hand, function builds a result set prior to searching, so you would have to move WHERE inside it - however, this is not easy because you might have many columns and you cannot ORDER BY all of those in the same select statement.
Therefore, if you compare views and functions for the same task of giving a "view" over data, then VIEWs are a better choice.
BUT, functions can do much more. You can do multiple queries without needing to join tables with JOINS or UNIONs. You can do some complex calculations with the results, run additional queries and output data to the user. Functions are more like stored procedures that are able to return datasets, then they are like views.
Nobody seems to have mentioned this aspect.
You can't have Update statements in an inline function but you can write Update statements against them just as though they were an updatable view.
One big difference is that a function can take parameters whereas a VIEW cannot.
I tend to favour VIEWs, being a Standard and therefore portable implementation. I use functions when the equivalent VIEW would be meaningless without a WHERE clause.
For example, I have a function that queries a relatively large valid-time state table ('history' table). If this was a VIEW and you tried to query it without a WHERE clause you'd get a whole lot of fairly data (eventually!) Using a function establishes a contract that if you want the data then you must supply a customer ID, a start date and an end date and the function is how I establish this contact. Why not a stored proc? Well, I expect a user to want to JOIN the resultset to further data (tables, VIEWs, functions, etc) and a function is IMO the best way of doing this rather then, say, requiring the user to write the resultset to a temporary table.
Answering your question about updates in a function (msdn):
The only changes that can be made by
the statements in the function are
changes to objects local to the
function, such as local cursors or
variables. Modifications to database
tables, operations on cursors that are
not local to the function, sending
e-mail, attempting a catalog
modification, and generating a result
set that is returned to the user are
examples of actions that cannot be
performed in a function.
A view is a "view" of data that is returned from a query, almost a pseudo-table. A function returns a value/table usually derived from querying the data. You can run any sql statement in a function provided the function eventually returns a value/table.
A function allows you to pass in parameters to create a more specific view. Lets say you wanted to have grab customers based on state. A function would allow you to pass in the state you are looking for and give you all the customers by that state. A view can't do that.
A function performs a task, or many tasks. A view retrieves data via a query. What ever fits in that query is what you are limited too. In a function I can update, select, create table variables, delete some data, send an email, interact with a CLR that I create, etc. Way more powerful than a lowly view!
A "static" query is one that remains the same at all times. For example, the "Tags" button on Stackoverflow, or the "7 days" button on Digg. In short, they always map to a specific database query, so you can create them at design time.
But I am trying to figure out how to do "dynamic" queries where the user basically dictates how the database query will be created at runtime. For example, on Stackoverflow, you can combine tags and filter the posts in ways you choose. That's a dynamic query albeit a very simple one since what you can combine is within the world of tags. A more complicated example is if you could combine tags and users.
First of all, when you have a dynamic query, it sounds like you can no longer use the substitution api to avoid sql injection since the query elements will depend on what the user decided to include in the query. I can't see how else to build this query other than using string append.
Secondly, the query could potentially span multiple tables. For example, if SO allows users to filter based on Users and Tags, and these probably live in two different tables, building the query gets a bit more complicated than just appending columns and WHERE clauses.
How do I go about implementing something like this?
The first rule is that users are allowed to specify values in SQL expressions, but not SQL syntax. All query syntax should be literally specified by your code, not user input. The values that the user specifies can be provided to the SQL as query parameters. This is the most effective way to limit the risk of SQL injection.
Many applications need to "build" SQL queries through code, because as you point out, some expressions, table joins, order by criteria, and so on depend on the user's choices. When you build a SQL query piece by piece, it's sometimes difficult to ensure that the result is valid SQL syntax.
I worked on a PHP class called Zend_Db_Select that provides an API to help with this. If you like PHP, you could look at that code for ideas. It doesn't handle any query imaginable, but it does a lot.
Some other PHP database frameworks have similar solutions.
Though not a general solution, here are some steps that you can take to mitigate the dynamic yet safe query issue.
Criteria in which a column value belongs in a set of values whose cardinality is arbitrary does not need to be dynamic. Consider using either the instr function or the use of a special filtering table in which you join against. This approach can be easily extended to multiple columns as long as the number of columns is known. Filtering on users and tags could easily be handled with this approach.
When the number of columns in the filtering criteria is arbitrary yet small, consider using different static queries for each possibility.
Only when the number of columns in the filtering criteria is arbitrary and potentially large should you consider using dynamic queries. In which case...
To be safe from SQL injection, either build or obtain a library that defends against that attack. Though more difficult, this is not an impossible task. This is mostly about escaping SQL string delimiters in the values to filter for.
To be safe from expensive queries, consider using views that are specially crafted for this purpose and some up front logic to limit how those views will get invoked. This is the most challenging in terms of developer time and effort.
If you were using python to access your database, I would suggest you use the Django model system. There are many similar apis both for python and for other languages (notably in ruby on rails). I am saving so much time by avoiding the need to talk directly to the database with SQL.
From the example link:
#Model definition
class Blog(models.Model):
name = models.CharField(max_length=100)
tagline = models.TextField()
def __unicode__(self):
return self.name
Model usage (this is effectively an insert statement)
from mysite.blog.models import Blog
b = Blog(name='Beatles Blog', tagline='All the latest Beatles news.')
b.save()
The queries get much more complex - you pass around a query object and you can add filters / sort elements to it. When you finally are ready to use the query, Django creates an SQL statment that reflects all the ways you adjusted the query object. I think that it is very cute.
Other advantages of this abstraction
Your models can be created as database tables with foreign keys and constraints by Django
Many databases are supported (Postgresql, Mysql, sql lite, etc)
DJango analyses your templates and creates an automatic admin site out of them.
Well the options have to map to something.
A SQL query string CONCAT isn't a problem if you still use parameters for the options.