I would like to create my own query language for a web api I wrote. But I have no idea where to start for it.
The language should be like SQL. For that I looked up the NHibernate code, cause I know, that they have the HQL (Hibernate Query Language) but it didn't help.
Are there any instructions or sth.? If this question is wrong here please move and/or tell me where I should ask else.
The first step is a lot of design work, starting be answering the following question:
Is this new Query Language going to be converted to SQL which will be
executed by a standard database engine, or are you going to write your
own database server as well?
If it's going to be converted to SQL (just like HQL) then map out the translations from your language to SQL on paper (and make sure you know the possible SQL constructs you may have to use). Once you've got that, you can start implementing it. Yes, this sounds like BDUF, but the language should be defined in one pass, I think, as it will be more consistent and easier to use if you do it that way. You can always implement it in a more Agile way once you've got that.
If you're going to write own database server, you're on your own....
Related
I'm trying to gain a deeper understanding of SQL, and I was just wondering - what was SQL originally written in? I did some research, and according to this stackoverflow page, it seems that SQL was originally written in C. If so, is there any way I could access the original source code to see how SQL really works under the hood - its algorithms, data structures etc? If SQL was indeed written in C, its inner workings must be imperative since C is an imperative language, and since I am more versed in C++ than in SQL, it would be really rewarding to explore SQL in terms of an imperative paradigm. Thanks in advance for any insight you may have!
SQL is a language definition, which itself is not written in anything but plain text.
Also, you can implement a compiler/engine for SQL or Java or whatever high level language in C++, C, Assembly, or even directly in machine code if you like, so the language in which it is implemented doesn't tell you anything about its possibilities.
Which SQL? There are a lot of different competing SQL database flavors out there: Sql Server, Oracle, MySql, PostgreSQL, Access, SqlLite, Sybase... and that's just the tip of the iceberg. Fortunately, several of those are open source... you can go look at the code for yourself.
You can also read the Sql Language Specifications.
Is it possible to implement own database server taking Oracle PL/SQL syntax as the bases or i would like to ask why different database solutions have different syntax eg: SQL server, MySql, Sqlite etc. can't they have some specific standard of syntax for basic operations including PL/SQL(excluding SQLite) why everyone is having a different syntax, sorry for diversion of question into patent issues but i could not find a better place to ask this question.
Of course you can, but you have to parse the PL/SQL yourself into something other platforms understand. (You can use ANTLR for example as parser tool. There is even a full featured grammar for PL/SQL) This is possible for small solutions with a small instruction set, but for large, full support of PL/SQL you need to be Oracle-sized.
To answer the why: two reasons:
There is no standard, so everyone picks his own;
You don't want customers to leave, so your own 'best' framework that is incompatible with others, that is your USP, and it prevents users from just porting their code to the other platform. They are stuck on yours.
I would like to build arbitrary queries to a database, by allowing the user to build queries "on the fly". For every object/table, being able to select its attributes, and then "building" the query (that would translate into a SQL statement) and finally launching it, all through a web interface.
The ticketing system "rt" does that, for example, and another example would be the http://gatherer.wizards.com/Pages/Advanced.aspx webpage.
I'm currently programming in rails but any existing solution that implements this (or something similar) would be welcome.
Just be careful when creating dynamically generated queries like this that will need to be executed via sp_executesql (example: ms sql server), etc..... make sure you cover all of your bases to ensure that your application isnt vulnerable to SQL injection attacks as this type of development will essentially get one in a lot of trouble if its done incorrectly.. I would recommend storing all queries in a table and only reading queries from this table to help isolate the queries that are being ran in your application. Just identify them with a label, and allow the EU to choose the label from a dropdown list control on the frontend.
Good luck and I'm not sure of any software that will help assist
Not quite sure what your use case is here but i would say check out the
Doctrine ORM ( Object Relational Mapper )
**Edit
After reading more and looking at the example. I would only suggest Doctrine for a large website.
Then use Doctrines DQL syntax with some javascript/jquery magic for the forms.
Note that the queries you're referencing aren't arbitrary: they're on a very specific problem domain, on a specific set of sql tables.
That said, if I were you I'd look into how people are building sql queries with javascript. Something like these:
http://code.google.com/p/django-querybuilder/
http://css.dzone.com/articles/sqlike-sql-querying-engine?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+zones%2Fria+(RIA+Zone)
http://thechangelog.com/post/4914956307/rel-arel-ported-to-node-js-with-some-changes
That'll at least get you a good idea of the underlying data structures.
Can you advise on whether I can use just the Query functionality from Lucene to generate SQL queries? Something like an SQLQueryBuilder?
I have a massive SQL database of logs from a webserver cluster containing the original request and response strings plus some other useful/less bits and bobs. What I need to do is analyse the parameters in the original request and compare with the generated responses, looking at ratios, volatility, variability, consistency etc.
This question does not relate to the analysis stage, but only the retrieval of data from database which matches the parameters I'm interested in. So, I could just do this in good old sql queries, manually building the exact queries I need on a case-by-case basis. But that's kinda lame; I reckon we can be a bit smarter than that. Particularly as I can already see large numbers of similar but subtly different queries being useful. And as I'm hoping that I can expose a single search box via a web interface to non-technical end-users, adding sql queries seems like a bad idea... and a recipe for permanent maintenance requests (and can I be the first to say, er no thanks!).
In an ideal world I expose a search form, with the option to write simple queries like
request:"someAttribute=\"someValue\"" AND response="some hoped for result" AND daterange:30
which would then hopefully find all instances of requests which contain someAttribute="someValue" over the last 30 days. The results will then be put through standard statistical analyses on the given response text and printed out on-screen. At least, that's the idea.
Much of the actual logic to determine how to handle custom field definitions or special words I'll need to write myself, and that's ok. And NB, my non-technical end users are familiar enough with xml that they can handle a bit of attr="value" syntax, at least for the first iteration of the tool :D
In summary, I want to:
1) allow users to use google-like search syntax (e.g. via Lucene's QueryAPI) to specify text to match in the logs
2) allow a layer to manipulate the query based on special words or fields (e.g. this layer could be during a Java object phase)
3) convert the final query into an sql query appropriate for my database schema
4) query the database and spit back the resultset for statistical analysis
5) pretty-print on website:)
Am I completely barking up the wrong tree? It looks like it should be possible, but I can't seem to find much on it. I've been googling for a bit on this, for example trying "Lucene SQLQueryBuilder" as a possible start but didn't really find much by way of a lead.
So, my questions are:
Has anyone tried using Lucene's QueryAPI like this before? Did it work? Any gotchas?
Are there better query api libraries out there?
Examples, finished discussions and open-source implementations would be most helpful.
Many thanks.
NB: I don't think I want Lucene's search capabilities as such, as I'm only ever looking for exact matches. I just need a query layer on top of the database.
Lucene and SQL have very little in common as they're using totally different syntax (as HefferWolf mentioned) and different underlying data models. As you said yourself, I'm afraid you're barking the wrong tree.
There are however attempts, such as Hibernate Search to bridge this gap. These are interesting experiments as such, but I would be very careful to use any of that code in production.
You could possibly use Full Text Search features available in some SQL databases, or reindex all data in Lucene and use it without database.
I doubt you can reuse any code from lucene for this. Lucene does an internal rewrite of such queries but into a syntax which wouldn't be of much help for SQL I think.
name: Phil AND lastname: Miller AND NOT age: 26
would be rewritten to
+name Phil +lastname: Miller -age: 26
So I think you would have to write your on transition into a SQL Query syntax.
But maybe you can use Lucene as such for this. Have a look into hibernate-search which is quite handy to easily create a lucene index of a sql table.
I know that in most cases it`s more useful to learn more complicated technology/language and then easy one than vice versa.
But, actually, time to do university tasks is limited. If I first learn LINQ, then go for SQL, would it be hard for me to use and learn sql?
EDIT
The task I need to do is to work with database and get some data from it, so the question is almost about LINQ to SQL.
It is a bad idea.
And if today's universities teach you LINQ instead of giving you foundation to build your knowledge upon, I can only feel sorry and pity for their students.
Time is always limited. Don't waste it on things that are subject to constant change.
SQL will be there tomorrow, LINQ.... well who knows.
SQL is applicable anywhere, LINQ only in .NET world.
Either LINQ or something else, it will be easy to "learn" it afterwards. When you have the knowledge of SQL, it will just me a matter of hours/days/weeks, hardly longer.
Well, the 2 things are very different. LINQ (in the pure sense) isn't actually related to databases at all - it can be used quite happily just with in memory objects, or against web services, etc.
If you are mainly interested in writing better .NET code, then learn LINQ - but learn it properly - perhaps pick up C# in Depth, for example - which covers it very well in the last few chapters.
If you want to know about databases, then sure: learn SQL (such as TSQL) - but understand the differences. Databases are good if you need to write enterprise software, but not necessarily if you just want to do simple tasks.
edit re edit to the question
If you are just getting simple data in and out of the database, then you probably don't need to know much about SQL. Just use LINQ-to-SQL (or whatever tool), and let the ORM tooling worry about it.
Learn SQL first, then LINQ.
That way you'll understand how LINQ-to-SQL is working behind the scenes, but you'll also know enough to be able to cope when LINQ can't do what you need.
SQL is a standard, learn the standard.
More precisely :
learn database theory
learn CODD algebra
then pick up a "common database", do some tutorials on it, ...
I personnaly really like PostgreSQL tutorial
I submit that you cannot effectively use LINQ unless you have SQL knowledge. If you don't understand, at a minimum, the following, you cannot effectively query a database in any fashion:
select
insert
delete
update
joins
group by
boolean algebra
relational theory
set theory
Learning SQL will give you the concepts you need even if you use LINQ later.
Sql. However you could play with linq pad for a while - it is a freeware and realize that LINQ is a nice hybrid between SQL and C#