I'm currently working on a project for a web application that may be installed on several different servers with various software configurations. I want to make my application as flexible as possible by allowing the user to have various SQL servers installed. The problem is the SQL syntax used by any two server vendors does not match up. For a simple example, here is the same SELECT statement for MS SQL and MySQL:
MS SQL - SELECT TOP 1 * FROM MyTable ORDER BY DateCreated DESC
MySQL - SELECT * FROM MyTable ORDER BY DateCreated DESC LIMIT 1
Are there any standard way to abstract the statement creation for various vendors? Any online resources or books discussing this problem? Any hints or smart-alec remarks that I'd find useful?
Further information: I'm writing my we application in vanilla ASP running on a Windows server.
Thanks, Spara
You can conform to ANSI SQL 92. All major RDBMS I know will support that.
However, there are tons of things individual RDBMS makers have added to enhance their own flavor of SQL. That is where you get into a lurch.
You may have to branch out in code depending on the RDBMS you are connecting to and generate / choose the appropriate SQL statement at that point.
A better option would be to create a DAL for each supported RDBMS. Implement a DAL interface across the DALs to make them uniform. This should be easier than switching in code.
I suggest that instead of trying to please everybody, you should write your code such that you support the top one or two systems that you expect to deploy on, and add support for other RDBMS as and when required.
I suggest you use an ORM (linq, nhibernate etc) to abstract the SQL dialect away rather than trying to write plain vanilla SQL.
Edit: Is there an OR/M for Classic ASP?
You know, I bet you could get by with strictly ansi sql, it will just take some effort and probably extra work. i.e.
SELECT MAX(*) FROM mytable ORDER BY datecreated DESC;
There will be workarounds in ansi because really all of the db specific constructs are ways to shorten and or shortcut existing ways of getting at or describing data. Another option might be to restrict access to the various databases to stored procs and user-defined functions. That way, you could write scripts for a bunch of the dbs you know will be used with the requirement that your db specific script be run before the app will work.
Just an idea.
Related
I'm writing instructions for people who are completely new to SQL and who are probably using an interface to talk to the database. The interface might be a SQL editor provided by their workplace, or it might be software (e.g. a BI reporting system) that allows them to write a query rather than use the gui. These are business people, not IT.
The first thing I want them to do is identify which RDBMS they have, so we can establish which type of SQL they need to learn. I.e. is it Oracle, Microsoft, MySQL, PostGre et al. I feel silly asking this: but is there an SQL command that works across all RDBMS to tell them this? The closest I've got is select version(), but this only gives the version number, not the RDBMS name. And anyway it isn't consistently implemented.
If the ANSI information schema is available:
SELECT * FROM information_schema.sql_implementation_info;
In particular the "DBMS NAME" and "DBMS VERSION" fields.
Usually, if they are using some sort of BI system, you can simply check the configuration of the software where the connection parameters are set
So, I haven't had any luck finding any articles or forum posts that have explained to me how exactly a query language works in conjunction with a general use programming language like c++ or vb. So I guess it wont hurt to ask >.<
Basically, I've been having a hard time understanding what the roles of the query language are ( we'll use SQL as an example for query language and VB6 for norm language) if i'm creating a simple database query that fills a table with normal information (first name, last name, address etc). I somewhat know the steps in setting up a program like this using ado objects for the connection and whatnot, but how do we decide which language of the 2 gets used for certain things ? Does vb6 specifically handle the basics like loops, if else's, declarations of your vars, and SQL specifically handles things like connecting to the database and doing the searching, filtering and sorting ? Is it possible to do certain general use vb6 actions (loops or conditionals) in SQL syntax instead ? Any help would be GREATLY appreciated.
SQL is a language to query a database. SQL is an ISO standard and relational database vendors implement to the ISO standard and then add on their own customizations. For example in SQL Server it is called T-SQL and in Oracle it is called PL-SQL. They both implement ISO standards and so each will have identical queries for a simple select like
select columname from tablename where columnname=1
However, each have different syntax for string functions, date functions, etc....
The ISO SQL standard by design is not a full procedural language with looping, subroutines, ect as in a full procedural language like VB.
However, each vendor has added capabilities to their version to add some of this functionality in.
For example both T-SQL and PL-SQL can "loop" through records using various constructs in their language.
There is also a difference when working with data that many developers are not well in tuned with. That is set based operations vs. procedural based.
Databases can work with procedural constructs but are often more performant with set based. A developer who is not versed in this concept may end up creating a very innefficient query. Here's an example of this discussion.
With any situation you have to weight out the pro's/con's of where it is best to do this work.
I tend to favor using procedural constructs such as loops in the language I am using over SQL. I find it easier to maintain and the language I am using offers more powerful syntax for me to get the job done.
However, I keep both options as a tool in the toolbox. For example, I have written data conversion scripts in SQL and in this case I have used the looping constructs in SQL.
Usually programming language are executed in the client side (app server too), and query languages are executed in the db server, so in the end it depends where you want to put all the work. Sometimes you can put lot of work in the client side by doing all the calculations with the programming language and other times you want to use more the db server and you end up using the query language or even better tsql/psql or whatever.
Relational databases are designed to manage data. In particular, they provide an efficient mechanism for managing memory, disk, and processors for large quantities of data. In addition, relational databases can handle multiple clients, guarantee transactional integrity, security, backups, persistence, and numerous other functions.
In general, if you are using an RDBMS with another language, you want to design the data structure first and then think about the API (applications programming interface) between the two. This is particularly true when you have an app/server relationship.
For a "simple" type of application, which uses a lot of data but with minimal or batch changes to it, you want to move as much of the processing into the database as is reasonable. Here are things you do not want to do:
Use queries to load things into arrays, and then do array manipulations at the language level. SQL provides joins for this.
Load data into an array and do manipulations and summaries on the array. SQL provides aggregations for this.
Save data into a file to have a backup. Databases provide backup mechanisms.
If you data fits into an array or on an Excel spreadsheet, it is often sufficient to get started with the data stored there. Only when you start to expand the needs (multiple clients, security, integration with other data) do the advantages of a database become more apparent.
These are just for guidance and to give you some ideas.
In terms of doing what where, do as much as is sensible in SQL (given it runs on a server) as you can.
So for instance don't do stuff like this (psuedo code)
foreach(row in "Select * from Orders")
if (row[CustomerID] = 876)
Display(row)
Do
foreach(row in "Select * from Orders where CustomerId = 876")
Display(row)
First it's likely Orders is indexed by CustomerID so it will find all 876s order way quicker.
Second to do the first one you just sucked every record in that table into the client's memory space probably across your network.
What language is used is essentially irrelevant, you could invent your own DBMS with it's own language.
It's where you do what processing that matters. It's Rule with exceptions, but the essential idea is let your backend do as much as it can.
I'm looking for a SQL Implementation (and its Editor) that can be used for translating it to many other(s) SQL Languages.
For example, when i code in that SQL Language to script file(s), and then i translate to other(s) SQL Language script file(s) (for ex: MS SQL's , MySQL's , ...).
If you're sure to use only ANSI SQL to construct your scripts, you should be good to go.
I agree with #Justin Niessner: all SQL vendors pay attention to the SQL Standards, notably core SQL-92. To take SQL Server as an example, although they find Sybase legacy code is tricky to deprecate they are not afraid to do so and entirely new features (e.g. MERGE in MSSQL2008) tend to extend their Standard SQL equivalents, rather than reinventing the wheel.
For a product that has good Standards compliance, take a look at Mimer
Here at Mimer Information Technology, we pride ourselves on conforming
to the SQL standard and we play an active role in the Database
Languages standardization group which determines exactly what is SQL
standard.
Mimer also provide extremely useful SQL validators for SQL-92, SQL-99 and SQL:2003 respectively.
I've been researching the same thing a while ago. What I've found is that there is a project liquibase. It is aimed at change tracking but also converting between different DBMS. You can download source code and see different datatypes conversions across databases. Source at github browse for java files there, probably you'll find something helpful
If all you want are basic operations, these are fairly universal. For instance:
SELECT
INSERT
DELETE
UPDATE
FROM
WHERE
JOIN
...are all at the most basic level the same across implementations.
However, the more complicated your scripts get, the more difficult it becomes to make them "universal". Things like aggregation, subqueries, cursors, while loops, functions, indexes, constraints, temp tables, variables, string manipulation, window operations etc. are all pretty much database-specific.
Some of these do have "universal" equivalents but the more generic you make your code the worse it will perform.
I am working on an application that will need to communicate with many different applications running on different database platforms. I will know the table schema before runtime but I won't know the database platform (MS SQL 200X, Oracle 9i, 10g, etc, MySQL 4.0.1, 5.x, etc, sybase, etc) until runtime.
It's my understanding that each of these systems have a slightly different dialect. Do I need to use nhibernate to handle the differences when connecting to these systems or can I use ADO.NET and pass raw SQL strings (select * from table)?
If you only need to use ANSI SQL statements, which should be implemented by all of the databases then yes, you can just use ADO.NET.
In my experience the main problem with database-agnostic code is the use of surrogate keys, like sequences or autonumber fields, as all databases implement these differently.
If you do need to use features that differ across databases then I don't think that it is reason enough to go to an object relational mapper like NHibernate - only do that if you have other reasons to do so. You can implement your own handling of syntax differences by generating different SQL for different databases easily enough.
SQL should be standardized for all dbs but they don't all use the same syntax so it really depends on what SQL you're calling. For example, SQL Server uses TOP while Oracle uses rownum. Even if they're all DDL, some syntactically differences between DBMSes can be an issue.
If select * from table is all you want, then there shouldn't be a problem, other than performance hits.
I need to give users the ability to build a simple SQL query against our database. Our application is written in Delphi.
I am assuming only moderate levels of knowledge by the user, but they need the ability to build a simple select statement to be able to query against a couple of tables. If I can make this easy for them, that would be most wondrous.
Does anyone know of a tool or a set of components that I can use to help the users build SQL SELECT statements...
I've used the DevExpress ExpressFilter control to allow users to specify SQL where constraints before.
I've come across FastQueryBuilder http://fast-report.com/en/products/visual-query-builder.html
and
OpenQueryBuilder http://fast-report.com/en/products/free-query-builder.html
(apparently differing versions of the same tool) at Fast-Report.
It looks like the right idea. Anyone have any experience with it?
I've used SimpleQuery for years and am quite happy with it. (http://devtools.korzh.com/eq/vcl/) It's not completely intuitive, but once you get the hang of it, you can present fields to the user from your database(s) and they can combine any series of them with ANDs and ORs to make very complex queries. You get a separate SQL window to save the resulting code. I make The resulting dataset available for printout with PrintDat! or to save as CSV data from the Woll2Woll Infopower grid that I use for display. I occasionally allow for XLS export too with XLSReadWrite. It's all quite automatic at this point. Just take the SimpleQuery example and mold that to your particular needs. You should be able to have something very usable in a day. There is a trial and an example free query builder standalone program that will give you an idea of the final look.
TMS Query Studio? Good value at 75 Euro..
Try EMS Advanced Query Builder.
is a powerful component suite for
Borland® Delphi® and C++ Builder®
intended for visual building SQL
statements for the SELECT, INSERT,
UPDATE and DELETE clauses. It allows
you to build new queries visually
and/or graphically represent the
existing queries in your own
applications. The suite includes
components for working with standard
SQL, MS SQL, InterBase/Firebird,
MySQL, PostgreSQL and many more
dialects. Advanced Query Builder
enables users to make up large and
complicated SQL queries with unions
and subqueries for different servers
without any knowledge of the SQL
syntax.
(source: sqlmanager.net)
(source: sqlmanager.net)
I think the most powerful one is ActiveQueryBuilder, which now included with Delphi IDE (Since 2007 I think)
(source: activequerybuilder.com)
I've written a couple of similar things in Delphi. It's easy enough to allow the user to pick table and column names by querying the metadata and using lists and tree views for display. The difficulty comes when trying to implement things like joins. I've never come up with a good interface for this, and in my experience neither have many of the major data tools players.
You may want to try this free builder:
GSC Visual Query Builder v.0.7
Visual Query Builder v.1.0
and some other in Torry:
http://www.torry.net/pages.php?id=546