Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
There was this old sql server tool called Lectoneth or something like that, you'd put sql queries in it, and it would rewrite it for you.
I think quest bought them out, but I can't find where to download a free copy of that software.
Really helps when you have no dba, and have lots of sql queries to rewrite.
Thanks
Craig
Doesn't ring a bell, and presumably you've seen, but nothing obvious on Quest's website
Perhaps a tool like Red Gate's SQL Prompt would help - the Pro edition does SQL reformatting.
Edit
Think i've found what you're looking for, mentioned here - LECCO SQL Expert. The link to the Lecco website does indeed direct to quest, but a 404.
LECCO SQL Expert is the only complete
SQL performance tuning and
optimization solution offering
problematic SQL detection and
automatic SQL rewrite. With its
built-in Artificial Intelligence (AI)
based Feedback Searching Engine, LECCO
SQL Expert reduces the effort required
to optimize SQL and makes even the
most junior programmer an expert.
Developers use LECCO SQL Expert to
optimize SQL during application
development. DBAs eliminate
problematic SQL before users
experience application performance
problems by using LECCO SQL Expert in
production systems.
Looks like it's no longer about - all mentions of I could find indicated it supported up to SQL 2000, and stale links - looks like it wasn't a free tool. As said in my comments, I think this kind of thing is a skill well worth possessing and would benefit in the long run to not relay on a tool to try and do it for you.
I wasn't aware of this tool before now, so I have picked up something from this question - got me intrigued!
Final Update:
To confirm, that product has indeed gone as Lecco was acquired some years ago now. Thanks to Brent Ozar for confirmation.
I think you're looking for a product that's been merged into Toad for SQL Server. The commercial version of Toad has a SQL Optimizer feature that tries lots of ways to rewrite your SQL statements, then tests them to find which ways are the fastest.
You can download Toad here:
http://www.toadsoft.com/
But be aware that that feature is a paid-version-only feature.
Well rather than spending your time looking for a magic bullet, why not spend some time learning performance tuning (you will need a book, this is too complex for the Internet generally). Plus it is my belief that if you want ot write decent new code, you need to understand performance in databases. There is no reason to be unable to write code that avvoids the most common problems.
First, rewrite every query to use ANSII syntax anytime you open it up to revise it for any other reason. Code review all SQl changes and do not pass the code review unless explicit joins were used.
Your first step in performance tuning to identify which queries and procs are causing the trouble. You can use tools that will tell you the worst performing queries in terms of overall time, but don't forget to tune the queries that are run frequnetly as well. Cutting seconds off a query that runs thousands of times a day can really speed things up. Also since you are in oprod already, likely your users are complaining about certain areas, those areas should be looked at first.
Things to look for that cause performance problems:
Cursors
Correlated subqueries
Views that call views
Lack of proper indexing
Functions (especially scalar function that make the query run row by row insted of through a set)
Where clauses that aren't sargeable
EAV tables
Returning more data than you need (If you have anything with select * and a join, immediately fix that.)
Reusing sps that act on one record to loop throuhg a large group of records
Badly designed autogenerated complex queries from ORMs
Incorrect data types resulting in the need to be continually be converting data in order to use it.
Since you have the old style syntax it is highly likely you have a lot of accidental cross joins
Use of distinct when it can be replaced with a derived table instead
Use of union when Union all would work
Bad table design that requires difficult construction of queries that can never perform well. If you find yourself frequently joining to the same table multiple times to get the data you need, then look at the design of the tables.
Also since you have used implicit joins you need to be aware that even in SQL Server 2000 the left and right implicit syntax does not work correctly. Sometimes this interprets as a cross join instead of a left join or right join. I would make it a priority to find and fix all of these queries immediately as they may currently be returning an incorrect result set. Bad data results are even worse that slow data returns.
Good luck.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
Introduction:
So, I have an interview tomorrow and I'm trying to review SQL and databases. The job posting says that they want someone with:
Experience with database design and development
Strong knowledge of SQL
Experience with SQL Server and/or Postgres
I've read through Questions every good database SQL developer should be able to answer, and a bunch of questions tagged with SQL and interview-questions. So I realize that I need to know about SELECT, JOIN and WHERE.
Questions:
What are essential SQL, Postgres and database concepts that I need to know in order to do well in the interview?
What do I need to know about transaction and normalization?
What are some general ways to optimize slow queries?
Should I learn about the functions, keywords or both?
It depends on how much of the role is based around database development and design. For your SQL syntax, you should also understand the difference between the types of joins, and be able to use GROUP BY, ORDER BY, HAVING as well as the aggregate functions that can be used in conjunction with them.
In terms of performance monitoring, I would be looking at execeution plans (not sure about the Postgres equivalent) and how they can provide tips on increasing performance, as well as using SQL Profiler to see what instructions the server is executing in real time.
Transactions can be useful for rolling back, well, transactions (stored procs, ad-hoc queries etc.) that require queries to complete in a certain way to maintain data consistency. Some people (myself included) have a practice of placing any statements that make any changes to data into a transaction that automatically rolls back (BEGIN TRAN ... ROLLBACK TRAN) to check that the correct amount of data is manipulated before pushing changes to a live server. Have a look at the ACID model - Atomicity, Consistency, Isolation, Durability.
Normalization is something that can take a little time to go through, but just know and partially understand up to 3rd form normalization and that will get you started.
Optimisation can be a huge topic. Just remember to try and do things like UPDATE using set based queries, rather than row based (updating in a WHILE loop is an example of row based updating, but it CAN have its uses).
I hope this helps a little.
Besides the basics of sql syntax, which you listed, you should know some things about query performance. What are some common causes of slow queries and what are the remedies for those, and how can you evaluate the performance of a query.
I'm on a project where I was asked to take a quick peek at some reporting SQL (in a SQL Server 2K5 environment) and was surprised at what I found: 4 to 5 levels of subquerys, distinct clauses, unions, and NoLock hints (which were needed because the SQL was running so long it was blocking standard processing) - all in the same set!.
Because I (foolishly :) mentioned that I thought the SQL was inefficient I've been labeled the "expert" and have been tasked with creating a test for a couple of interviewees to do that will assess their SQL optimizing abilities. I'm hoping someone can point me to some URLS, or maybe provide a list that I can use to help weed out the good from the bad.
Since you mentioned the SQL Server 2005 environment:
More SQL Server interview questions than you possibly could have imagined:
The classic set. Most interviewees will probably have studied these...maybe a good way to gauge who has prepared.
Another classic
Questions from one of the original Stack Overflow DBAs
Another link for best SQL questions and answers
I would give them a Query plan (via EXPLAIN, or whatever your flavor of SQL uses as a keyword) and see if they can decipher what it means, what the weak points are, and how to improve the query.
Look at MySQL's Explain Documentation for help using MySQL's explain and what it means.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I work in a group of about 25 developers. I'm responsible for coming up with the database design (tables, views, etc) and am called apon for performance tuning when necessary.
There are a couple of different applications that connect. Database access is via JDBC, hibernate, and iBatis SQL maps. Developers with various levels of experience write SQL statements.
What guidelines would you give to developers to write good SQL?
By good I mean: correct, performs well, easy to understand and maintain.
These are just meant to be easy to follow guidelines - I want to get people onto the right track for the majority of situations. We will break these guidelines when it makes sense.
EDIT: We have in place code reviews for all source commits (SQL, java, etc) enforced through a jira workflow.
If you have 25 developers writing SQL queries against your database you are in quite a bit of trouble. Guidelines are not worth much when your junior developers are learning SQL and checking in a mess.
I would like to offer 4 recommendations
Use an ORM of sorts so your all your devs write less SQL.
Invest in training, buy books, send people to courses.
Have all the SQL reviewed by the senior SQL developers, by all, I mean every SQL statement, no exceptions. This way your senior guys will be able to teach the juniors over time.
Have a single person, who lives and breaths Oracle, responsible for the database. By responsible I mean knows every query, understands all the structure and is able to give expert advice.
Here are some additional things you may add to your existing guidelines/checklist.
Have you tested your queries on a large data set? How was performance?
Have you performed a quick index review on the tables that are being accessed? Are all the right indexes in place? Do you recommend and new indexes?
For high volume queries, are any covering indexes required?
Are you using "NOT IN" in cases where a "LEFT JOIN" should be used?
Is your work transactionally sound? Are you missing a transaction somewhere?
Here's what I already have in my guidelines.
Work in sets, not row by row
The best way to make something go quicker is to avoid doing work you do not have to do
Databases love to join
Fully qualify and specify column names (so SQL does not break when additional columns are added)
Select only the data you need (never select *, never more rows than you require, never every column just becaues it's there)
How to use rownum to limit resultsets
Bind Variables vs Literals (use bind variables in all but a few special cases related to skewed data)
Avoid functions or calculations on columns in the WHERE clause (except for a special case of function based index)
Use ORDER BY for all queries returning more than one row (this is mostly for testability)
Each of these points is expanded a bit in the actual guidelines I've written out with an example relevant to our database schema.
Read Tom Kyte's books. He explains how you can write fast code and how you can measure performance and scalability. If you have a problem you can probably find the answer on the "ask tom"-site.
Introduce basic style guide that covers:
naming (of everything - tables, columns, procedures, aliases, ...) .
formatting style
line width
what reserved words require new line (e.g where)
are reserved word capitalized or small caps
indenting
...
Here are some examples:
Oracle PL/SQL Programming, Fourth Edition. There is older, 2nd edition - available online
SQL and PL/SQL Coding Standards
Be very strict about naming, it will be easier for you to read other people's code.
As formatting is concerned, there are tools available that can format automatically, so maybe you don't need very detailed description here.
If you are a database developer, you need to know what an EXECUTION PLAN is. If you don't then go mine coal or something.
Before developing:
first, you think what best EXECUTION PLAN will be,
second you create tables and indexes, and
third you use hints to persuade the optimizer to come out with the plan you made.
You do use hints. Forget automatic optimization, it's a marketing myth. No optimizer knows your data better than you and never will.
There are no "programmers who create queries" and "system administrators who create indexes". Programmers program, system administrators make backups (or whatever they make).
Triggers are evil.
Prefix you columns, tables and views (SELECT prs_name FROM t_person)
Make lines and indent
An hour long presentation on some Oracle fundamentals (eg parsing, SGA vs PGA). "Do this" rules may or may not apply to your situation. Give them an understanding of what the DB side does, and they at least have a basis on which to make a decision.
Plus Code reviews.
Pair-program. Any advantange it provides for agile development in general, at least doubles for SQL development.
Second choice, code reviews for all SQL.
Along with the recommendation to have queries reviewed by senior programmers, if you can get the buy-in, have code reviews which involve as many team members as possible.
I'm by no means a guru but here are my tips:
Don't use ORDER BY unless you really need an ordered list as it incurs a performance hit.
Understand the explain plan and also recognise that the plan on your development environment is often different from your production environment. Don't expect it to accurately reflect real life performance
The pros of using hints is that you get to choose your explain plan, the cons of using hints is that the optimal plan may change over time and you might be choosing a plan that is suboptimal in the long term
Make sure the developers know when to use INNER JOIN, OUTER JOIN, [NOT] IN, [NOT] EXISTS - you can put in place a lot of processes but one or two Cartesian products will bring production performance to its knees
Ensure your developers understand indexes - what they are, when they should be used, when they should be avoided
Have a DBA monitor the most executed queries and the most expensive queries and highlight these as candidates for optimisation
Peer review
Coding standards (especially code comments on particularly long/complex queries)
Unit testing
Don't write SQL if you can help it, use HQL (or JPQL is on Java EE) whenever possible
Don't use SELECT *
Pick your internet sources wisely (e.g. asktom.oracle.com)
Don't use cursors
Don't do string concatenation in SQL
Write queries such that they use indexes (fundamentally this means base WHERE predicates on the indexes that exist)
use MERGE instead of other awkward 'upsert' type logic
When working with dates, make sure you understand how they're stored in Oracle vs. how they are stored in Java, especially when it relates to TimeZone. Depending on the Calendar/Date types, this information can be stripped out, remapped to the TZ of the default locale, etc.
Most importantly: Don't use the excuse of being a developer for not knowing how to write good SQL, and how the database works. You don't have to be a DBA, but you need to invest in your own training to make yourself suitable for the task. By the same token, your company needs to invest in that as well.
I don't mean to say that these "Don'ts" always apply. It's just that, if you're talking about a developer who is not comfortable with Oracle, they need to know what they're doing before they start deciding whether those types of things are necessary and appropriate.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
What is the best practical way of learning index tuning while writing tsql queries? I have VS2008 SQL Express. Could someone please provide me examples, etc? I have already found online articles and they are great in theory, but I still fail to see index tuning in real life action. Are there small easy to create examples out there?
To tune indexes, you tend to need large tables with lots of data, so small simple examples aren't easy to come by.
My experience is with the SQL 2000 tools. Query Analyser, showing the Execution Plan and looking at the types of index and joins used. Very hard to describe it here.
I can recommend a good book on the subject, particularly Chapter 9.
http://www.amazon.com/Professional-Server-Performance-Tuning-Programmer/dp/0470176393
I would discourage you from using the automated Index Tuning tools until you understand how to do it yourself manually. I think it's important when it recommends adding an index that you have the ability to sanity-check the recommendation and decide for yourself whether it is a good option. Often it will recommend you add a "covering" index with many columns in order to speed up a single query you've asked to be analysed, but this may have adverse effects on your database overall when you look at all queries against that table.
Kimberly Trip (SQL Goddess) is an expert and has talked & written lots on the subject:
http://www.sqlskills.com/BLOGS/KIMBERLY/category/Indexes.aspx
Ken England's Microsoft SQL Server 2000 Performance Optimization and Tuning Handbook is a classic. (The Amazon reviews say the 2005 version of this book is not as good)
If you have SQL Developer edition, you want to have a look at the Database engine tuning advisor.
If you use the profiler to capture a standard workload against your database, the DETA can recommend what statistics and indexes you could apply.
Beware though that tuning requires a good deal of thought, as adding new indexes may increase the speed of your workload, but you may increase the speed of an unimportant query at the detriment of a very important one.
There is a good book about TSQL Querying which has some very good advice about how queries work and how you can look at tuning them, which is MSSQL specific.
In SQL Server, if your database schema is a more simple one, you don't really need to tune much, since primary keys automatically result in a clustered index, and foreign key constraints require a unique index.
So your joins are taken care of, usually.
Where is gets tricky is with searches and filters that the queries in the application use. You will have to specifically look at these queries and identify candidates for additional indices.
Another option is the SQL Server Tuning Advisor, but I discourage it's use, because it generates so much noise. You can use it to find the worst-offending queries, though.
For a starting point of identifying your worst performing queries, please see my answer here.
Learning how to tune is something that requires practice. If you want to become skilled at it, start by buying Sajal Dam's book SQL Server Query Performance Tuning Distilled
My firm have a talented and smart operations staff who are working very hard. I'd like to give them a SQL-execution tool that helps them avoid common, easily-detected SQL mistakes that are easy to make when they are in a hurry. Can anyone suggest such a tool? Details follow.
Part of the operations team remit is writing very complex ad-hoc SQL queries. Not surprisingly, operators sometimes make mistakes in the queries they write because they are so busy.
Luckily, their queries are all SELECTs not data-changing SQL, and they are running on a copy of the database anyway. Still, we'd like to prevent errors in the SQL they run. For instance, sometimes the mistakes lead to long-running queries that slow down the duplicate system they're using and inconvenience others until we find the culprit query and kill it. Worse, occasionally the mistakes lead to apparently-correct answers that we don't catch until much later, with consequent embarrassment.
Our developers also make mistakes in complex code that they write, but they have Eclipse and various plugins (such as FindBugs) that catch errors as they type. I'd like to give operators something similar - ideally it would see
SELECT U.NAME, C.NAME FROM USER U, COMPANY C WHERE U.NAME = 'ibell';
and before you executed, it would say "Hey, did you realise that's a Cartesian product? Are you sure you want to do that?" It doesn't have to be very smart - finding obviously missing join conditions and similar evident errors would be fine.
It looks like TOAD should do this but I can't seem to find anything about such a feature. Are there other tools like TOAD that can provide this kind of semi-intelligent error correction?
Update: I forgot to mention that we're using MySQL.
If your people are using the mysql(1) program to run queries, you can use the safe-updates option (aka i-am-a-dummy) to get you part of what you need. Its name is somewhat misleading; it not only prevents UPDATE and DELETE without a WHERE (which you're not worried about), but also adds an implicit LIMIT 1000 to SELECT statements, and aborts SELECTs that have joins and are estimated to consider over 1,000,000 tuples --- perfect for discouraging Cartesian joins.
..."writing very complex ad-hoc SQL queries.... they are so busy"
Danger Will Robinson!
Automate Automate Automate.
Ideally, the ops team should not be put into a position where they have to write queries on the fly in a high stress situation – it’s a recipe for disaster! Better for them to build up a library of pre-written scripts that have undergone the appropriate testing to make sure it a) does what you want b) provides an audit trail c) has a possible ‘undo’ type function.
Failing that, giving them a user ID that only has SELECT premissions might help :-)
You might find SQL Prompt from redgate useful. I'm not sure what database engine you're using, as it's only for MSSQL Server
I'm not expecting anything like this to exist. The tool would have to first implement everything that the SQL parser in your database implements, and then it would have to do a data model analysis to predict "bad" queries.
Your best bet might be to write a plugin for a text editor that did some basic checking for suspicious patterns and highlighted them differently than the standard .sql mode. But even that would be quite difficult.
I would be happy with a tool that set off alarm bells whenever I typed in an update statement without a where clause. And perhaps administered a mild electric shock, since it's usually about 1 in the morning after a long day when mistakes like that happen.
It would be pretty easy to build this by setting up a sample database with a extremely small amount of dummy data, which would receive the query first. A couple of things will happen:
You might get a SQL syntax error, which would not load the database much since it's a small database.
You might get back a response which could clearly be shown to contain every row in one or more tables, which is probably not what they want.
Things which pass the above conditions are likely to be okay, so you can run them against the copy of the production database.
Assuming your schema doesn't change much and is not particularly weird, writing the above is likely the quickest solution to your problem.
I'd start with some coding standards - for instance never use the type of join in your example - it often results in bad results (especially in SQL Server if you try to do an outer join that way, you will get bad results). require them to do explicit joins.
If you have complex relationships, you might consider putting them in views and then writing the adhoc queries from the views. Then at least they will never make the mistake of getting the joins wrong.
Can't you just limit the amount of time a query can run for? I'm not sure about MySQL, but for SQL Server, even just the default query analyzer can restrict how long queries will run before they time out. Couple that with limited rights so they can only run SELECT queries, and you should be pretty much covered.