I am working on an application that will need to communicate with many different applications running on different database platforms. I will know the table schema before runtime but I won't know the database platform (MS SQL 200X, Oracle 9i, 10g, etc, MySQL 4.0.1, 5.x, etc, sybase, etc) until runtime.
It's my understanding that each of these systems have a slightly different dialect. Do I need to use nhibernate to handle the differences when connecting to these systems or can I use ADO.NET and pass raw SQL strings (select * from table)?
If you only need to use ANSI SQL statements, which should be implemented by all of the databases then yes, you can just use ADO.NET.
In my experience the main problem with database-agnostic code is the use of surrogate keys, like sequences or autonumber fields, as all databases implement these differently.
If you do need to use features that differ across databases then I don't think that it is reason enough to go to an object relational mapper like NHibernate - only do that if you have other reasons to do so. You can implement your own handling of syntax differences by generating different SQL for different databases easily enough.
SQL should be standardized for all dbs but they don't all use the same syntax so it really depends on what SQL you're calling. For example, SQL Server uses TOP while Oracle uses rownum. Even if they're all DDL, some syntactically differences between DBMSes can be an issue.
If select * from table is all you want, then there shouldn't be a problem, other than performance hits.
Related
I need some clarification on databases and backend.
I am currently using microsoft sql server with a database connection. Does that mean, that the sql server uses an sql database? Or is it using a proprietary sql server database?
I don't understand the concept of database and it's associated platforms used. SQL server uses T-sql to connect with the database. But, can i use sqlite or mysql to connect with the same database?
Applications are often split into "front-end" and "back-end"; this is often an informal definition, but it might also refer to "client/server" architecture, or "n-tier" architecture.
The "front-end" is typically the user interface; the "backend" is everything else.
One of the key tasks of "backend" components is storing information.
There are lots of ways to do that; if you application domain is a good fit for relational data, it's common to use a relational database. The most common technology for relational databases (rdbms) is SQL, which is a standardized language for defining relational databases through data definition language (DDL), and retrieving and modifying data (DML).
There are many implementations of SQL - SQLite, MySQL, Postgres, Oracle, MS SQL Server are all examples. Each implementation has its strengths and weaknesses; they all adhere to the SQL specification to some degree, and many have extensions beyond the SQL specification such as stored procedures, support for XML, free text search etc. Theoretically, it is possible to migrate between SQL implementations (e.g. from Oracle to MS SQL Server); in practice, this is usually very hard. To the best of my knowledge, there is no RDBMS which adheres 100% to the standard; it would probably not be hugely useful because the standard defines the language, rather than the implementation, and the implementation is what makes the database useful. For instance, the way the database implements indexes is a huge influence on performance.
T-SQL is a proprietary language, originally developed by Sybase (MS SQL Server was originally Sybase); it supports things like stored procedures etc. As far as I know, MS SQL Server is the only platform with support for T-SQL.
Your database connection specifies a few things:
- which driver to use to connect to the database. The driver understands how to connect across the physical hardware (e.g. network), and how to convert your instructions into something the server understands.
- the location of the server, and which particular database to use (one server can contain multiple databases)
- credentials to for authentication and authorization.
After many years working with SQL databases, it feels unconformable working with a database that doesn't rely on a schema to model the data.
I understand that SQL and NoSQL solutions have their places for different business needs and goals, but I don't have any experience with NoSQL databases.
But since I discovered that Microsoft SQL Server has support to also work with JSON data (https://learn.microsoft.com/en-us/sql/relational-databases/json/json-data-sql-server?view=sql-server-2017), I wonder:
Can I always default to SQL Server for any (new) application I might need to create and use this flexibility of JSON querying when needed?
That would mean I don't have to wrap my head around considering between SQL Server OR MongoDB OR both. I could just use SQL Server always and be good to go.
A similar consideration of mine is about graph-databases. SQL Server vs Neo4j for graph databases.
(https://learn.microsoft.com/en-us/sql/relational-databases/graphs/sql-graph-architecture?view=sql-server-2017).
Sure SQL Server support for graph is inferior compared to Neo4j which is specialized for that task, but it seems that Microsoft is trying to create a one-for-all database solution that every project could rely on.
Now a days mostly all database providing the datatype of any field in a table as json type.
But relational database is not providing the solutions as nosql database.
I'm looking for a SQL Implementation (and its Editor) that can be used for translating it to many other(s) SQL Languages.
For example, when i code in that SQL Language to script file(s), and then i translate to other(s) SQL Language script file(s) (for ex: MS SQL's , MySQL's , ...).
If you're sure to use only ANSI SQL to construct your scripts, you should be good to go.
I agree with #Justin Niessner: all SQL vendors pay attention to the SQL Standards, notably core SQL-92. To take SQL Server as an example, although they find Sybase legacy code is tricky to deprecate they are not afraid to do so and entirely new features (e.g. MERGE in MSSQL2008) tend to extend their Standard SQL equivalents, rather than reinventing the wheel.
For a product that has good Standards compliance, take a look at Mimer
Here at Mimer Information Technology, we pride ourselves on conforming
to the SQL standard and we play an active role in the Database
Languages standardization group which determines exactly what is SQL
standard.
Mimer also provide extremely useful SQL validators for SQL-92, SQL-99 and SQL:2003 respectively.
I've been researching the same thing a while ago. What I've found is that there is a project liquibase. It is aimed at change tracking but also converting between different DBMS. You can download source code and see different datatypes conversions across databases. Source at github browse for java files there, probably you'll find something helpful
If all you want are basic operations, these are fairly universal. For instance:
SELECT
INSERT
DELETE
UPDATE
FROM
WHERE
JOIN
...are all at the most basic level the same across implementations.
However, the more complicated your scripts get, the more difficult it becomes to make them "universal". Things like aggregation, subqueries, cursors, while loops, functions, indexes, constraints, temp tables, variables, string manipulation, window operations etc. are all pretty much database-specific.
Some of these do have "universal" equivalents but the more generic you make your code the worse it will perform.
I'm currently working on a project for a web application that may be installed on several different servers with various software configurations. I want to make my application as flexible as possible by allowing the user to have various SQL servers installed. The problem is the SQL syntax used by any two server vendors does not match up. For a simple example, here is the same SELECT statement for MS SQL and MySQL:
MS SQL - SELECT TOP 1 * FROM MyTable ORDER BY DateCreated DESC
MySQL - SELECT * FROM MyTable ORDER BY DateCreated DESC LIMIT 1
Are there any standard way to abstract the statement creation for various vendors? Any online resources or books discussing this problem? Any hints or smart-alec remarks that I'd find useful?
Further information: I'm writing my we application in vanilla ASP running on a Windows server.
Thanks, Spara
You can conform to ANSI SQL 92. All major RDBMS I know will support that.
However, there are tons of things individual RDBMS makers have added to enhance their own flavor of SQL. That is where you get into a lurch.
You may have to branch out in code depending on the RDBMS you are connecting to and generate / choose the appropriate SQL statement at that point.
A better option would be to create a DAL for each supported RDBMS. Implement a DAL interface across the DALs to make them uniform. This should be easier than switching in code.
I suggest that instead of trying to please everybody, you should write your code such that you support the top one or two systems that you expect to deploy on, and add support for other RDBMS as and when required.
I suggest you use an ORM (linq, nhibernate etc) to abstract the SQL dialect away rather than trying to write plain vanilla SQL.
Edit: Is there an OR/M for Classic ASP?
You know, I bet you could get by with strictly ansi sql, it will just take some effort and probably extra work. i.e.
SELECT MAX(*) FROM mytable ORDER BY datecreated DESC;
There will be workarounds in ansi because really all of the db specific constructs are ways to shorten and or shortcut existing ways of getting at or describing data. Another option might be to restrict access to the various databases to stored procs and user-defined functions. That way, you could write scripts for a bunch of the dbs you know will be used with the requirement that your db specific script be run before the app will work.
Just an idea.
SO has plenty of Q&As about how to accomplish various tasks using dynamic SQL, frequently the responses are accompanied with warnings and disclaimers about the advisability of actually using the approach provided. I've worked in environments ranging from "knowing how to use a cursor is a bad sign" to "isn't sp_executesql neat!".
When it comes to production systems should dynamic sql always be avoided or should it have a valid place in the programming toolbox. If not/so, Why?
Answers to some of your questions can be found here The Curse and Blessings of Dynamic SQL
Sometimes you have to use dynamic SQL because it will perform better
It depends on what you mean by dynamic sql. Queries built where parameter values substituted directly into the sql string are dangerous and should be avoided except for the very rare case where there is no other option.
Dynamic SQL using the appropriate parameterized query mechanism supported by your platform can be very powerful.
There are several considerations when evaluating the use of dynamic SQL:
performance can vary widely, and specific to which database engine is targeted
maintenance can vary widely, in both directions (e.g., dynamic SQL can employ
modularization techniques that static SQL cannot)
dynamic SQL is vulnerable to SQL injection attacks, but static SQL is (mostly) not
sometimes your needs make static SQL (nearly) impossible, but that should be rare
Remember, SQL is code, and an RDBMS is another API. It is NOT special, or at least not much; deal with it like you would any other code and API. In particular, don't just code directly against the API: modularize your code and write some helper methods to make it easier and reusable.
The cost of 'dynamic SQL' varies between DBMS.
In IBM DB2, where query plans can be pre-compiled down to machine code, the cost of Dynamic SQL is significant.
In IBM Informix (IDS - Informix Dynamic Server), most queries are effectively 'dynamic SQL' (the exception is queries in stored procedures) in that the SQL is interpreted at run-time, even though the client-side notation uses static SQL.
I believe - though a DB2 expert might be able to contradict me - that the CLI (ODBC, aka CCC) and JCC (JDBC) systems for DB2 do all SQL as dynamic SQL.
I don't know what Oracle, Sybase, MS SQL Server do - my suspicion is that they hew closer to the line adopted by IDS than the line adopted by DB2. For MySQL and PostgreSQL, I'd be surprised if they did not behave more like IDS than DB2.
As a result, with IDS, there is no particular overhead to using Dynamic SQL; at the server level, your SQL is dynamic anyway. Other DBMS may have other factors at play.
One issue for all servers is 'how do you identify the pre-compiled query from the SQL sent over the wire'. With DB2, the pre-compilers identify the package which is used, and the communications protocol between application and server identifies that. My understanding is that the DB2 clients such as ODBC and JDBC do not use a pre-compiled package - hence I think that they effectively do dynamic SQL all the time.
Beware SQL injection with Dynamic SQL!
My previous shop would never allow such things to execute against the database (SQL Server).
It was prohibited as practice, and the DB was locked down to prevent it.
All work goes through objects (SPs, etc).
This is the right way to do it always, IMHO.
There are edge cases where dynamic SQL is both easier and faster than the alternative. As long as you keep them few and far between and they are dynamically generated prepared parameterized SQL I see no big problem with them.
Dynamic SQL has a place - even in production code. Executing arbitrary code with security holes does not.
In general, I would avoid using dynamic SQL if there is not a good reason to use it. Dynamic SQL does not get checked until it runs, so you obviously have a bigger testing burden. However, there are plenty of good times to use it when dealing with administrative tasks, static code-generation, accommodating changing systems without excessive maintenance based on querying metadata, using DRY to avoid redundancy, etc.
Our system has what I think is a lot of dynamic SQL in it, mostly because we have a lot of dynamically created objects (tables, indexes, views mostly). A lot of this is legacy; things like partitioned tables in 2k5 help somewhat in some of our use cases. But as mentioned you need to make a good case for it; on-the-fly SQL inside the procs usually has a better (static) solution.