I'm currently looking at some light-weight SQL abstraction modules. My workflow is such that i usually write SELECT queries manually, and INSERT/UPDATE queries via subs which take hashes.
Both of these modules seem perfect for my needs and i have a hard time deciding. SQL::Interp claims SQL::Abstract cannot provide full expressivity in SQL, but discusses no other differences.
Does it have any disadvantages? If so, which?
I can't speak to SQL::Interp, but I use SQL::Abstract and it's pretty good. In conjunction with DBIx::Connector and plain old DBI, I was able to totally eliminate the use of an ORM in my system with very little downside.
The only limitations I have run into is that it's not possible to write GROUP BY queries directly (although it's easy to do by simply appending to the generated query, and LIMIT queries are handled by the extension SQL::Abstract::Limit.
I used SQL::Abstract for a over a year, and then switched to SQL::Interp, which I've stuck with since.
SQL::Abstract had trouble with complex clauses. For the ones it could support, you would end up with a nest of "(" "[" and {" characters, which you were mentally translate back to meaning "AND", "OR" or actually parentheses.
SQL::Interp has no such limitations and uses no middle representation. Your SQL looks like SQL with bind variables where you want them. It works for complex queries as well as simple ones. I find SQL::Interp especially pleasant to use in combination with DBIx::Simple's built-in support for it. DBIx::Simple+SQL::Interp is a friendly and intuitive replacement for using raw DBI. I use the combination in a 100,000k+ LoC mod_perl web app.
Related
I like to use keyword where, for example.
I simply do it like this
Dim orderFromThisGridTrading = _orders1.Where(Function(x) x.customIdentifier = ASimpleGridTradeSettings.name).ToArray
However, most samples would say I have to use from
Dim customersForRegion = From cust In customers Where cust.Region = region
Which is weird af and don't follow normal vb.net format. That looks a like like SQL languages and not a real programming language.
I wonder if I can always avoid using Form and just use like I am using? Are there any cases where that is not possible?
Is this even a good idea?
There is nothing bad in using query syntax in general. Especially in VB.NET it is very powerful and supports a lot of LINQ methods(more than in C#). Many prefer that because it can make your code more readable. But it seems that the opposite is true for you, you don't like it. Then use the method syntax, it supports all methods. I don't like it in VB.NET because of that ugly and verbose Function keyword, especially with GroupBy or Join i prefer query syntax.
I wonder if I can always avoid using From and just use like I am
using? Are there any cases where that is not possible?
No, method syntax is always possible. Actually query syntax gets compiled to method syntax.
The second example is called "Query Comprehension Syntax". It's optional. A lot of people find it makes their code more readable, but not everyone likes it. Personally, I find it adds another extra layer of indirection for what you need to know to get the computer to do what you want.
In the first example, the "From" is implied by the intial IEnumerable or IQueryable item in the expression.
But I do have one issue:
SQL is definitely a real programming language, and you'll likely get even better results from learning it well vs needing to rely on an ORM to construct queries. Eventually, usually sooner than later, you'll find you want to do something where you need to know advanced SQL anyway. Then you have two problems, because you need to the know the SQL and you need to know how to construct the ORM syntax to produce that SQL.
Articles on the internet say user-defined functions can either burden or increase the performance.
Now, I know that standard SQL is pretty limited, however, some of the behavior can still be written as in T-SQL built-in functions.
For example, adddays() vs. dateadd() . Another point I've heard that its also better to use coalesce() - the ANSI standard function rather than isNull().
What is the performance difference between using the ANSI SQL standard functions vs T-SQL functions?
Does T-SQL adds any burden what so ever on the performance with it trying to make the job easier, or not?
My research does not seem to indicate a trend.
You will need to approach this on a case-by-case basis and do actual testing. There is no general rule, other than Microsoft tries to make the entire stack perform as well as possible. TESTING is what you need to do - we can't tell you that always a certain thing would be faster. That would be really bad advice.
It is important to do this testing on your actual production data, prefereably a copy of it. Do not rely on tests done against data sets that aren't yours. When you're talking about performance differences of functions, some very subtle things can make a big difference. Things like the size of the table, the data types involved, the indexing, and SQL Server versions, can change the result of these tests. That is why "no one has done this" for you. We can't.
I've recently implemented a package that uses the database/sql package. By limiting the SQL to very simple select/update/insert statements I assumed the package would work with all the DBMS supported by database/sql.
However, it turns out that some databases use ? as placeholder value while others use $1, $2, etc., which means the prepared statements will work with some DBMS but not with others.
So I'm wondering is there any technique to make this work in a generic way with all the supported drivers? Or is it necessary to have DBMS-specific code everywhere? (which I think would make the abstraction provided by database/sql a bit pointless). I guess using non-prepared statements is not an option either since different DBMS have different ways to escape parameters.
Any suggestion?
I assume this behaviour was left out specifically because SQL dialects vary significantly between databases, and the Go team wanted to avoid writing a preprocessor for each driver to translate 'GoSQL' into native SQL. The database/sql package mostly provides connection wrangling, which is an abstraction that falls under 'pretty much necessary' instead of statement translation, which is more 'nice to have'.
That said, I agree that re-writing every statement is a major nuisance. It shouldn't be too hard to wrap the database/sql/driver.Prepare() method with a regex to substitute the standard placeholder with the native one, though, or provide a new interface that specifies an additional PrepareGeneric method that guesses a wrapped sql.DB flavour, and provides similar translation.
Gorp uses a dialect type for this, which may be worth a look.
Just throwing out ideas.
I am new to ANTLR & grammar writing. My requirement is to parse insert/update/delete SQL queries to get details like which table is being update/inserted/deleting a row, list of columns & their values etc. Yes there is a good documentation of ANTLR, but if anyone can help me with specific grammar for query parsing then it would be great help.
There are various SQL grammars on the Wiki: http://www.antlr.org/grammar/list
Beware though: they are contributed by ANTLR-users, chances are that they've not been properly tested and/or contain bugs.
But why generate an SQL parser yourself? It would probably be easier to use some exiting SQL parser. Just do a search on "SQL parser Java" (assuming you're working with Java), and you're bound to get dozens of hits.
Implementing a "decent" SQL parser is actually fairly hard. In SQL, one can write all kinds of complex statements (nested joins, ...), and people really do this, so you have to implement the "full" language. (I've seen SQL queries that cover tens of pages. Stupid, yes, but then you have to work with what you encounter).
I suggest checking out SQL2011 (the standard) as being rather comprehensive. However, that grammar may not be ANTLR friendly, so be prepared for a fair bit of work.
You also have to worry about database/vendor specific extensions to standard SQL. PL/SQL (just the SQL sublanguage part) contains lots of Oracle-specific extensions. If you are facing PL/SQL stored procedures, and you want to do that table/column tracing, you may have to do the procedural part of PL/SQL too, and that's also pretty big.
My experience of using Adobe ColdFusion, even if still somewhat limited, was absolutely joyful and pleasant.
Of all good things I could say about ColdFusion, one feature completely blew me off my feet. It might be neither very effective, or useful in production, but anyway, I am talking about the so-called "query of queries" feature, or the dbtype="query" attribute of cfquery. It allows you to run SQL statements against arbitrary datasets, not just a database connection. You can, for example, join a resultset, that you've just retrieved from the database and an in-memory structure (that is, of course, subject to certain limitations). It provides a quick-and-dirty way to kind of "post-process" the data, which can sometimes be much more readable (and flexible, too!), than, say, iterating through the dataset in a loop.
However, ColdFusion is not a very popular product and I am not going to go over the reasons why it is like that. What I am asking is, is there any support for this technique in other languages (like a library, that does more or less the same)? Python? Perl? Ruby? PHP? Anything? Because, to me it seems, that the potential of this feature is huge, maybe not in production code, but it is an absolute life-saver if you need to test something quickly. Needless to say, the SQL ColdFusion uses for this is somewhat limited, to my knowledge, but still, the idea is still great.
If you don't find anything that handles data as well as ColdFusion then remember it plays very well with other programming languages. You can always do the heavy query processing in CF then just wrap your processing logic in remote CFCs and expose them as web services serving up JSON.
That will let you benefit from what you find great about ColdFusion while trying out some other languages.
If you need to get away from CF try SqlAlchemy in Python, or like other posters said Rails and LINQ are worth playing with.
i can't for python, ruby, perl, php. however .Net has something called LINQ which is essentially QoQ on steroids.
Lots of frameworks use object-relational mapping (ORM), which will convert your database tables to objects.
For example, using Rails you fetch data from a model instead of directly talking to the database. Queries, or finds, are returned as array objects, which can in turn be queried themselves.
You can also accomplish this in .NET by using LINQ. LINQ will let you query objects as well as databases.
In doing performance analysis of Query of Queries, I was surprised by their execution time, I could not get them to return in less than 10ms in my tests, where simply queries to the actual database would return in 1ms or less. My understanding (at least in CF MX 7) is that while this is a useful function, it is not a highly optimized one. I found it to be much faster to loop over the query manually doing conditional logic to replace what I was attempting to do with my query of queries.
That being said, it is faster than going to the database if the initial query is slow. Just don't use it thinking that it is going to always be faster than doing a more creative sort or initial query as each QofQ is far from instantaneous.
For Java, there's three projects worth taking a look at, each with it's own positives and negatives, some more SQL like than others. JoSQL JoSQL, JXPath, and MetaModel.
Maybe one of these day's I'll figure out how to call a QoQ directly from the Java underneath CF. ;)
This technique (ColdFusion's query-of-queries) is one of the worst ideas out there. Not only does it keep business logic in the database, but it takes what little business logic you have left in your code and shoves it out to the database just for spite.
What you need is a good language, not bad techniques to make up for deficiencies.
Python and Ruby, as well as other languages not on your list such as C# and Haskell, have exceptional support for writing arbitrary and powerful queries against in-memory objects. This is in fact the technique that you want, not ColdFusion's query-of-queries. The technique of writing queries against in-memory objects is an aspect of a general style of programming called functional programming.