Our databases are on SQL Server 2000. We are migrating to SQL Server 2008 R2.
We want to update the compatibility level to 100.
But the join operator *= and =* are no more supported with that compatibility level.
The advised solution is to update our source code and change the operators to OUTER JOIN.
But we cannot afford the modification of thousand exe and dll. So we cannot update our source code; there are too many projects.
My question is:
Is there in SQL Server 2008 R2 any system which will allow me to:
catch the SQL statement before it's execution
update the statement (for instance replace *= by LEFT OUTER JOIN)
give back the modified statement to the SQL Server engine for execution
This method would allow us to set the compatibility level to 100 even though our programs still use the old syntax
Thanks
(P.S. This question has been cross-posted on dba.stackexchange)
No. That would require two things that do not exist:
Something akin to an INSTEAD OF SELECT trigger
and
1000 billable hours (or more depending on how good you and/or your team is at QA) to come up with the text parsing / regular expressions / etc needed to capture all of the syntax variations that are allowed in queries: block comments in the middle of values , inline comments at the end, conditions broken up into multiple lines, table aliases, outer joins of inner-joined tables, and so on and so on.
You will actually spend less time updating your source code (as others have suggested), even across "thousand exe and dll", than you will trying to implement this and then spending the rest of your life debugging it and taking support calls for problems that eventually turn out to be caused by bugs in this.
(P.S. this answer has been cross-posted on DBA.StackExchange, but with a bit more detail there)
UPDATE
I submitted a Microsoft Feedback Suggestion for this ability: Intercept query batch from client to rewrite it or cancel it.
I agree that updating the apps would be the right thing to do, long-term.
But sometimes doing the right thing is not an option, and you have to consider alternatives.
In your case, there is the option to add a smart database proxy (e.g. Gallium Data) to rewrite the query as needed. You can define a filter in the proxy that will intercept the problematic queries and rewrite them as desired. Then all you need is to get those apps to talk to the proxy instead of directly to the database.
Given your problem, is this preferable to modifying all these apps? That's a decision you need to make, given your particular situation. But you have that option.
Related
I am writing an OpenSQL query, including joins and sub-selects, and I am unsure whether it works. To find out, I'd like to run it somewhere, look at the results, then change it, it an iterative way.
Traditional tools like transactions SE16 or SQVI don't help, because they have too little functionality (only one table, only joins but no sub-selects, etc.). Other threads with similar questions, like this one remained unanswered.
Of course, I could write a report or a unit test, paste my code there, and run it, but this makes the iterative cycle so slow because I need to change input and output data formats all the time. Optimal solution would be something like the SQL Console in SAP HANA Studio, but for OpenSQL instead of native SQL.
(on behalf of #Florian) Use the SQL console in Eclipse. it allows Open SQL SELECT, except FOR ALL ENTRIES and variables.
(on behalf of #JozsefSzikszai) software "OpenSQL Editor for SAP", by STA Consulting Kft. There is a 30 days trial period, so might be a short term solution only (unless your company buys it)
Use the tool ZTOAD, created by Sébastien Hermann : http://quelquepart.biz/article7/ztoad-requeteur-open-sql.
You may indirectly test the Open SQL statements by creating test classes (ABAP Unit), in which you use the class CL_OSQL_TEST_ENVIRONMENT (ABAP >= 7.52), which is used to mock the database, so you may provide false data, enter the expected result, execute the open SQL in test mode, and verify that actual result = expected result. It supports all Open SQL features.
All the above solutions allow testing Open SQL directly. Another solution is to use native SQL editors, and to write the Open SQL statements in the Strict mode (ABAP >= 7.40) so that they are closer to the native SQL syntax (especially the comma is to be used to separate the column names; but the alias separator has to be manually changed from ~ to .). There are two native SQL editors: one is a tool in ST04 or ST04N, one is the standard program RSDU_EXEC_SQL. Both display the result of SELECT statements in an ALV grid.
Good morning,
you could take a look into queries (SQ01/SQ02) instead of QuickViews in SQVI. In Queries you have a chance to enhance your Joins with coding and you are still able to test your "code". For incremental changes this might be your easiest way to go.
Regards
Well, It seems like such a simple solution to the many problems that can arise from insecure services and applications. But I'm not sure if it's possible, or maybe nobody's thought of this idea yet...
Instead of leaving it up to programmers/developers to ensure that their applications use stored procedures/parameterised queries/escape strings etc to help prevent sql injection/other attacks - why don't the people who make the databases just build these security features into the databases so that when an update or insert query is performed on the database, the database secures/sanitizes the string before it is inserted into the database?
The database would not necessarily know the context of what is going on. What is malicious for one application is not malicious for another. Sometimes the intent IS to
drop table users--
It is much better to let the database do what it does best, arranging data. And let the developers worry about the security implementations.
The problem is that the database cannot readily tell whether the command it is requested to execute is legitimate or not - it is syntactically valid and there could be a valid reason for the user to request that it be executed.
There are heuristics that the DBMS could apply. For example, if a single request combined both a SELECT operation and a DELETE operation, it might be possible to infer that this is more likely to be illegitimate than legitimate - and the DBMS could reject that combined operation. But it is hard to deal with a query where the WHERE condition has been weakened to the point that it shows more data than it was supposed to. A UNION query can deliberately select from multiple tables. It is not sufficient to show that there is a weak condition and a strong condition OR'd together - that could be legitimate.
Overall, then, the problem is that the DBMS is supposed to be able to execute a vast range of queries - so it is essentially impossible to be sure that any query it is given to execute is, or is not, legitimate.
The proper way to access the database is with stored procedures. If you were using SQL Server and C#/VB.NET you could use LINQ to SQL, which allows you to build the query in the language witch then gets turned into a parameterized SP. Good stuff.
I am using Microsoft SQL 2005 server. I have created several SP and Functions in TSQL. After more than one year's development, the number of SP/Functions have grown dramatically. Only a few of SP/Functions are used as scheduled jobs or triggers, and most of them are used internally(I mean they are called by SP/Functions in jobs and triggers).
With the number of SP/Functions growing, it is very hard to manage them in our SQL server. It is hard for other developer/DBAs to know their dependency and usages. I would like to keep all the dependent SP/Functions as private ones just like ones in PL/SQL's packages. I could rewrite them in CLR but customized CLR is not allowed in our SQL server.
Any way to limit SP/Function's usage scope, i.e., limit them as private in a "package"? Not sure if SQL 2008 have any features to resolve this issue?
In SQL Server 2005 (onwards), you can use Schemas for this purpose, BUT it does have security chaining implications.
Unfortunately, SQL Server does not yet have a 'package' equivalent, though I believe that this is something which has been requested often.
Isn't it ironic? The very tool that was supposed to help you reduce complexity creates more. I used SP last about 2 years ago when I created the first version of a blogging website that I was making. They caused me a lot of heartache cause I couldn't fit them well in the scheme of things (my C# code).
So finally I got rid of them and took all my SP code back to C#. That made things easier to organizer.
Now I may use Views, but I don't use SPs any more.
Here's an argument for SPs that I haven't heard. Flamers, be gentle with the down tick,
Since there is overhead associated with each trip to the database server, I would suggest that a POSSIBLE reason for placing your SQL in SPs over embedded code is that you are more insulated to change without taking a performance hit.
For example. Let's say you need to perform Query A that returns a scalar integer.
Then, later, the requirements change and you decide that it the results of the scalar is > x that then, and only then, you need to perform another query. If you performed the first query in a SP, you could easily check the result of the first query and conditionally execute the 2nd SQL in the same SP.
How would you do this efficiently in embedded SQL w/o perform a separate query or an unnecessary query?
Here's an example:
--This SP may return 1 or two queries.
SELECT #CustCount = COUNT(*) FROM CUSTOMER
IF #CustCount > 10
SELECT * FROM PRODUCT
Can this/what is the best way to do this in embedded SQL?
A very persuasive article
SQL and stored procedures will be there for the duration of your data.
Client languages come and go, and you'll have to re-implement your embedded SQL every time.
In the example you provide, the time saved is sending a single scalar value and a single follow-up query over the wire. This is insignificant in any reasonable scenario. That's not to say there might not be other valid performance reasons to use SPs; just that this isn't such a reason.
I would generally never put business logic in SP's, I like them to be in my native language of choice outside the database. The only time I agree SPs are better is when there is a lot of data movement that don't need to come out of the db.
So to aswer your question, I'd rather have two queries in my code than embed that in a SP, in my view I am trading a small performance hit for something a lot more clear.
How would you do this efficiently in
embedded SQL w/o perform a separate
query or an unnecessary query?
Depends on the database you are using. In SQL Server, this is a simple CASE statement.
Perhaps include the WHERE clause in that sproc:
WHERE (all your regular conditions)
AND myScalar > myThreshold
Lately I prefer to not use SPs (Except when uber complexity arises where a proc would just be better...or CLR would be better). I have been using the Repository pattern with LINQ to SQL where my query is written in my data layer in a strongly typed LINQ expression. The key here is that the query is strongly typed which means when I refactor I am refactoring properties of a class that is directly generated from the database table (which makes changes from the DB carried all the way forward super easy and accurate). While my SQL is generated for me and sent to the server I still have the option of sticking to DRY principles as the repository pattern allows me to break things down into their smallest component. I do have the issue that I might make a trip to the server and based on the results of query I may find that I need to make another trip to the server. I don't worry about this up front. If I find later that it becomes an issue then I may refactor that code into something more performant. The over all key here is that there is no one magic bullet. I tend to work on greenfield applications which allows this method of development to be most efficient for me.
Benefits of SPs:
Performance (are precompiled)
Easy to change (without compiling the application)
SQL set based features make very easy doing really difficult data tasks
Drawbacks:
Depend heavily on the database engine used
Makes deployment of upgrades a little harder (you have to deploy the App + the scripts)
My 2 cents...
About your example, it can be done like this:
select * from products where (select count(*) from customers>10)
My firm have a talented and smart operations staff who are working very hard. I'd like to give them a SQL-execution tool that helps them avoid common, easily-detected SQL mistakes that are easy to make when they are in a hurry. Can anyone suggest such a tool? Details follow.
Part of the operations team remit is writing very complex ad-hoc SQL queries. Not surprisingly, operators sometimes make mistakes in the queries they write because they are so busy.
Luckily, their queries are all SELECTs not data-changing SQL, and they are running on a copy of the database anyway. Still, we'd like to prevent errors in the SQL they run. For instance, sometimes the mistakes lead to long-running queries that slow down the duplicate system they're using and inconvenience others until we find the culprit query and kill it. Worse, occasionally the mistakes lead to apparently-correct answers that we don't catch until much later, with consequent embarrassment.
Our developers also make mistakes in complex code that they write, but they have Eclipse and various plugins (such as FindBugs) that catch errors as they type. I'd like to give operators something similar - ideally it would see
SELECT U.NAME, C.NAME FROM USER U, COMPANY C WHERE U.NAME = 'ibell';
and before you executed, it would say "Hey, did you realise that's a Cartesian product? Are you sure you want to do that?" It doesn't have to be very smart - finding obviously missing join conditions and similar evident errors would be fine.
It looks like TOAD should do this but I can't seem to find anything about such a feature. Are there other tools like TOAD that can provide this kind of semi-intelligent error correction?
Update: I forgot to mention that we're using MySQL.
If your people are using the mysql(1) program to run queries, you can use the safe-updates option (aka i-am-a-dummy) to get you part of what you need. Its name is somewhat misleading; it not only prevents UPDATE and DELETE without a WHERE (which you're not worried about), but also adds an implicit LIMIT 1000 to SELECT statements, and aborts SELECTs that have joins and are estimated to consider over 1,000,000 tuples --- perfect for discouraging Cartesian joins.
..."writing very complex ad-hoc SQL queries.... they are so busy"
Danger Will Robinson!
Automate Automate Automate.
Ideally, the ops team should not be put into a position where they have to write queries on the fly in a high stress situation – it’s a recipe for disaster! Better for them to build up a library of pre-written scripts that have undergone the appropriate testing to make sure it a) does what you want b) provides an audit trail c) has a possible ‘undo’ type function.
Failing that, giving them a user ID that only has SELECT premissions might help :-)
You might find SQL Prompt from redgate useful. I'm not sure what database engine you're using, as it's only for MSSQL Server
I'm not expecting anything like this to exist. The tool would have to first implement everything that the SQL parser in your database implements, and then it would have to do a data model analysis to predict "bad" queries.
Your best bet might be to write a plugin for a text editor that did some basic checking for suspicious patterns and highlighted them differently than the standard .sql mode. But even that would be quite difficult.
I would be happy with a tool that set off alarm bells whenever I typed in an update statement without a where clause. And perhaps administered a mild electric shock, since it's usually about 1 in the morning after a long day when mistakes like that happen.
It would be pretty easy to build this by setting up a sample database with a extremely small amount of dummy data, which would receive the query first. A couple of things will happen:
You might get a SQL syntax error, which would not load the database much since it's a small database.
You might get back a response which could clearly be shown to contain every row in one or more tables, which is probably not what they want.
Things which pass the above conditions are likely to be okay, so you can run them against the copy of the production database.
Assuming your schema doesn't change much and is not particularly weird, writing the above is likely the quickest solution to your problem.
I'd start with some coding standards - for instance never use the type of join in your example - it often results in bad results (especially in SQL Server if you try to do an outer join that way, you will get bad results). require them to do explicit joins.
If you have complex relationships, you might consider putting them in views and then writing the adhoc queries from the views. Then at least they will never make the mistake of getting the joins wrong.
Can't you just limit the amount of time a query can run for? I'm not sure about MySQL, but for SQL Server, even just the default query analyzer can restrict how long queries will run before they time out. Couple that with limited rights so they can only run SELECT queries, and you should be pretty much covered.