Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I don't even want to think about how many man hours have been spent writing the same queries to join over the exact same tables at my company.
When I first started at my job I identified this as an inefficiency and started writing views in a separate schema for the sole purpose of developer convenience.
My boss didn't like this very much and recommended I start committing my common queries to source control in a separate folder from the production SQL. This makes sense because some scripts require parameters and not all are read only.
What is a Common Query?
Script used to diagnose certain problems
Script to view relationships between several tables (doing multiple joins)
Script that we don't want in a stored procedure because it is often tweaked to diagnose the issue of the day
Issues I want to Address
Discoverability, queries will be rewritten if nobody can find them
IDE integration, want to be able to easily view queries in IDE. I've tried the SQL Server Solutions but they really suck because they lock you into only working on that set of files.
I was wondering how all the pro's out there share their common SQL queries.
Thanks
Seems like the op wants to know how to get the word out to the team about useful SQL that others can/should use so not to recreate.
How I have done this in the past is through two ways:
Create a team web wiki page that
details the SQL with examples of how
it is used.
Email the team when new SQL is
created that should be shared.
Of course, we always include the SQL code in version control, just the wiki and email are use in the "getting word out there" part.
If it is something that I would call "common" I would probably create a stored procedure that the folks with necessary permissions can run.
If the stored procedure route won't work well for your team, then the other option is to create a view. Creating a view comes with unique challenges though such as ensuring that everyone running the view has select permissions on all of the tables in the view as well.
Outside of storing the scripts in source control of some kind, maybe storing them on a Share Point site or a network file share would work OK for your team. The real challenge that you will have in sharing scripts is that people have different ways to identify what they are looking for. A wiki type of site that allows tagging the different types of things the queries do would be useful.
You create a view.
Lots of ways to do this (including some that you've mentioned already):
Table-Valued User Defined Functions
Stored Procedures
Views
Source Control
Formal, shared Data Access Layer for client code
Views are the right way to handle this sort of thing. Or, in some cases, a stored procedure.
But there's no rule that says you can't also store the DDL for a View or a Stored Procedure in source control.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I read both the Microsoft Docs and this article. I however can't seem to find where I could only use cursors and not something else so I'd appreciate If someone could give a few examples.
I also came across this answer on stackoverflow and Jeffrey Hantin gave me more doubts.
Imagine a command which must get a literal parameter like BCP.
Try to read 100 XML files living somewhere on your disc and you have their path and filenames in a table.
In this case you will create the statements dynamically and use EXEC to run each separately.
This might be any kind of loop, but a CURSOR seems to be the closest.
The general answer is: Avoid loops, most needs can be solved without using a loop. With SQL one should use set-based approaches over procedural approaches.
I find I only ever use cursors for administration tasks. For example I have a script that runs through my high activity large tables. That rebuilds the indexes overnight that query itself is pretty set based. However I have multiple customer databases on the same box with the same structure. I would wrap the maintenance script in a cursor that pulls out the database names and loops through them.
I find this reduces the amount of work I have to do, as the list of databases to work through is generated by the cursor. If I add customers/ delete customers the maintenance is unaffected. If I run up a new box I can add that as part of the set up process so when I get round to setting up a customer system on the new box it is automatically being maintained.
Here is a sample of somebody using the same concept to backup all of their databases. https://www.mssqltips.com/sqlservertip/1070/simple-script-to-backup-all-sql-server-databases/
Same benefits that I've mentioned in that it is defined once and whether you add remove or move databases they are already automatically in the backup plan
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
My question is more for practice than a debug issue.
At work, we use a Java-JEE/Oracle solution and the least I can say is we need to perform SQL query, anticipate SQL performances, handle SQL issues like foreign key or orphan line and so on.
So from my point of view, doing SQL is very important. For a new project, we are looking to implement the solution in Ruby on Rails. But most of the tutorial and code I see, seems to nest every Postgres SQL code under Active Record implementation. I have already experienced some similar issue with the Java Hibernate framework and its "no need SQL code." Some production issues were madness, the generated hidden SQL query were not easy to read and there is no deal with index or foreign key.
Any one can tell me what risk we have to use only Active Record ?
What is the proper process to avoid most common Ruby/SQL interface issues ?
When did you need to open your SQL console et type some SQL query ?
Share a little bit its experience on these points.
If you have any relevant link dealing with this topic.
Thank you very much !
You can still use sql.
Either low level, where you receive an array of arrays of values.
Or a little more high level, so you receive objects, with methods like find_by_sql.
Or by providing only sql-fragments, for example for the where-clause.
How often you need sql depends on your use case.
Ruby is about objects, sql is about tables. ActiveRecord handles objects as rows in a table. That works most of the time quite good. All simple queries are handles automatically. You can describe relations between objects, and even joins to retrieve these relations are handled.
For queries with several joins or group_by, it is sometimes easier to write the sql instead of instructing activerecord to build the sql you have in mind.
Also you need to have an eye on what sql is generated, as it is easy to write code that is inefficient, for example by generating many small sql statements.
The official Rails guides about "models" are the most important resource. From sql perspective you should have a look at "Active Record Query Interface"
http://guides.rubyonrails.org/active_record_querying.html
I also done a presentation about rails database optimisation, but it for rails 3.2 and a little out of date (joins are now better handled)
http://meier-online.com/en/2012/08/presentation-rails-database/
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I have heard some strong developers say that "inline SQL in and of itself is not evil". I do not understand how inline SQL is acceptable. To me its just like hard coding. Many a developer would scoff in my face for putting a connection string in the code vs a config file. So why is it that "SELECT value1,value2 FROM TABLE" is perfectly acceptable in compiled code?
I think there is a lot of coupling between a database query and the code that surrounds it. For example, a query might fetch first name and last name for a particular user from a database, and then create an XML file containing the first name and last name.
If you put the database query somewhere else (e.g. config file) it might seem like you've increased configurability and flexibility. However, if you want to actually change it, say add a field called "age" to the XML file, you can't just change the query alone, you need to change the code writing the XML file. So in fact you've replaced the problem of having to change one piece of code, with the problem of having to change two things (code and configuration), and the danger that they might not be consistent, and you haven't gained any extra flexibility.
(If it seems like one can just write generic code to take all database fields and put them in the XML file, consider the case where age is stored as the birth year in the database, and you have to do computation to create the age.)
When I see the length programmers go to just to avoid simple queries like:
select * from products where productid in (1,2) and productactive = 1
or:
select * from products where not exists (
select * from order_lines where products.id = order_lines.productid)
Then I always wonder what their real goal is. Because it can't be simplicity.
SQL is code, it's just not the same programming language as the rest of your code. If you need to change the SQL, you usally always need to change the code dealing with that SQL, data and database schema as well. The SQL shouldn't vary between installation.
If you add a column, changing the SQL isn't enough, you'll need to change code handling that column as well. Pretty much any change to the database schema and SQL, just changing the SQL is not enough.
A connection string on the other hand is configuration, not code.
(And yes, there's exceptions to all of this, if you're creating a general purpose reporting tool, you'll need to engineer that so the SQL can be "configurable")
SQL queries are code, not configuration.
It is useful in an agile environment to use stored procedures on a system like SQL Server where they are dynamically compiled, as you can often fix or workaround an application problem by patching the stored proc.
But it is still code.
Another distinction (aside from code vs. configuration) is that a connection string is probably used all over your codebase whereas a SQL query is more likely to be coupled to a specific class or method. Therefore, if you subscribe to "Don't Repeat Yourself", you'd definitely benefit from centralizing the connection string but would likely gain nothing from moving the SQL string to a different location.
That being said, some of the most elegant solutions involve avoiding SQL strings entirely by using Hibernate or another ORM, but that's a different topic.
"SELECT value1,value2 FROM TABLE" in compiled code was the essence of so called Object-relational impedance mismatch. Since LINQ was released inline SQL is no longer acceptable in vast majority of cases...
I've made systems where all the SQL is in stored procedures and the names of the stored procedures are themselves not even stored in the program, but in the app.config.
This means then not only can stored procedures be patched, but different customers can use different procedures.
The database services interface provided to the application(s) is thus part of the system design.
A drawback of having any SQL in a client application at all (and this includes table-based ORMs and LINQ) is that you have very little explicit security interface. With stored procedures, there is a single GRANT - EXEC. With tables or even views, the ORM will need SELECT, UPDATE, INSERT, DELETE for whatever SQL is generated by the tool or stored in the application. Taking an inventory of the rights needed for the system to properly run and being able to audit is much more difficult than having a good solid layer of stored procedures (and, to some extent, read-only views) which everything goes through.
So I would say that "SELECT value1, value2 FROM TABLE" is not acceptable in client code, because every user who would potentially execute that statement needs to have SELECT on those two column from a base table, and that is hard to manage, even with roles.
One way to look at it is to think of table names and column names as part of the database's "public" interface. That's how databases implement physical data independence. And updatable views implement logical data independence, to some degree.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
My DBA told me to use a user-defined SQL datatype to represent addresses, and then use a single column of that new type in our users table instead of multiple address columns. I've never done this before and am wondering if this is a common approach.
Also, what's the best place to get information about this - is it product-specific?
As far as I can tell, at least in the SQL Server world, UDT aren't used very much.
Trouble with UDT is the fact you can't easily update them. Once created and used in databases, they're almost like set in stone.
There's no "CREATE OR ALTER (UDT)" command :-( So to change something, you have to do a lot of shuffling around - possibly copying away existing data, then dropping lots of columns from other tables, then dropping your UDT, re-creating it with the new structure and reapplying the data and everything.
That's just too much hassle - and you know : there will be change!
Right now, in SQL Server land, UDT are just a nice idea - but really badly implemented. I wouldn't recommend using them extensively.
Marc
There are a number of other questions on SO about how to represent addresses in a database. AFAICR, none of them suggest a user-defined type for the purpose. I would not regard it as a common approach; that is not to say it is not a reasonable approach. The main difficulties lie in deciding what methods to provide to manipulate the address data - those used for formatting the data to appear on an envelope, or in specific places on a printed form, or to update fields, worrying about the many ramifications of international addresses, and so on.
Defining user-defined types is very product specific. The ways you do it in Informix are different from the ways it is done in DB2 and Oracle, for example.
I would also rather avoid using User defined datatypes as their defination and usability will make your code dependant on a particular database.
Instead if you are using any object oriented language, create a composition relationship to define addresses for an employee (for example) and store the addresses in a separate table.
Eg. Employees table and Employee_Addresses table. One employee can have multiple addresses.
user-defined SQL datatype to represent addresses
User-defined types can be quite useful, but a mailing address doesn't jump out as one of those cases (to me, at least). What is a mailing address to you? Is it something you print on an envelope to mail someone? If so, text is about as good as it's going to get. If you need to know what state someone is in for legal reasons, store that separately and it's not a problem.
Other posts here have criticized UDTs, but I think they do have some amazing uses. PostgreSQL has had full text search as a plugin based on UDTs for a long time before full-text search was actually integrated into the core product. Right now PostGIS is a very successful GIS product that is entirely a plugin based on UDTs (it has GPL license, so will never be integrated into core).
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
Here's my situation. I've noticed that code gets harder to maintain when you keep embedding queries in every function that will use them. Some queries tend to grow very fast and tend to lose readability after concatenating every line. Another issue that comes with all of the concatenation is when you must test a specific query and paste it. You are forced to remove all of the " that held your query string together.
So my question is, what methods are being used to separate queries from the code? I have tried searching but it doesn't look like it's the right thing because i'm not finding anything relevant.
I'd like to note that views and stored procedure are not possible since my queries fetch data from a production database.
Thank you.
If you follow an MVC pattern, then your queries should be all in the model - i.e. the objects representing actual data.
If not, then you could just put all your repetitive queries in script files, including only those needed in each request.
However, concatenating and that kind of stuff is hard to get rid of; that's why programmers exist :)
These two words will be your best friend: Stored Procedure
I avoid this problem by wrapping queries in classes that represent the entities stored in the table. So the accounts table has an Account object. It'll have an insert/update/delete query.
I've seen places where the query is stored in a file and templates are used to replace parts of the query.
Java had something called SQLJ - don't know if it ever took off.
LINQ might provide some way around this as an issue too.
Risking being accused of not answering the question, my suggestion to you would be simply don't. Use an O/RM of your choice and you'll see this problem disappear in no time.
I usually create a data class that represents the data requirements for objects that are represented in the database. So if I have a customer class, then I create a customerData class as well that houses all the data access logic in them. This keeps data logic out of your entity classes. You would add all you CRUD methods here as well as custom data methods.
Youc an also use Stored Proceedures or an ORM tool depending on your language.
The main key is to get your data logic away from your business and entity logic.
I use stored procedures in my production environment, and we process the rows too...