I am creating a new SQL database that needs to connect to several existing DB's, all of which have the same schema.
I would like to create stored procedures that can be used to access any of the existing DB's whilst re-using code as much as possible. My question is, what is the best way to write generic SP's that can be used to access any one of these existing databases?
Here are some options I have considered so far. Please note, these are just simple code snippets to illustrate the question, not real-world code. First I tried switching on the DB name:
IF #db = 'A'
BEGIN
SELECT * FROM A.dbo.SERIES_DATA
END
IF #db = 'B'
BEGIN
SELECT * FROM B.dbo.SERIES_DATA
END
This has the disadvantage that the same SQL statement is repeated several times. So then I thought of using dynamic SQL:
DECLARE #Command varchar(100)
SET #Command = 'select * from ' + #db + '.dbo.SERIES_DATA'
EXEC (#Command)
This solves the problem of duplicate staements, but has the risks of dynamic SQL (e.g. injection attacks etc). So finally I hit on the idea of creating a Table-valued Function:
CREATE FUNCTION [dbo].[ufnSeries_Data] (#db varchar(10))
RETURNS TABLE
AS
RETURN (
SELECT * FROM A.dbo.Series_Data WHERE #db = 'A'
UNION
SELECT * FROM B.dbo.Series_Data WHERE #db = 'B'
)
GO
This gives me the ability to write single, generic code to access any of the DB's:
SELECT * FROM ufnSeries_Data(#db)
The problem with this approach is that I have to give all users of the system read-access to all of the databases in order for this query to work, otherwise I get an access-denied error. I'm also not certain of the performance impact of creating a TVF like this.
Does anyone has any other ideas for how I can structure my code to minimize duplication of statements? Maybe there is a much simpler approach that I'm missing? I would be very grateful for any suggestions.
You can do this by removing the reference to your database and schema
SELECT * FROM SERIES_DATA
It is a question of creating the schema in all your databases, and granting the login access to the databases and schemas (in this case you are using the default dbo schema).
If you are using the management studio, when you create logins, you can handle the rest with user mapping.
If you have the same schema (e.g. dbo), you can change your select to:
SELECT * FROM dbo.SERIES_DATA
You really don't want to include the database name unless you are joining tables across databases.
Related
Consider this:
CREATE PROCEDURE [dbo].[setIdentifier](#oldIdentifierName as varchar(50), #newIdentifierName as varchar(50))
AS
BEGIN
DECLARE #old_id as int;
DECLARE #new_id as int;
SET #old_id = (SELECT value FROM Configuration WHERE id = #oldIdentifierName);
SET #new_id = (SELECT value FROM Configuration WHERE id = #newIdentifierName);
IF #old_id IS NOT NULL AND #new_id IS NOT NULL
BEGIN
UPDATE Customer
SET type = #new_id
WHERE type = #old_id;
END;
END
[...]
EXECUTE dbo.setIdentifier '1', '2';
What this does is create a stored procedure that accepts two parameters which it then uses to update a Customer table.
The problem is that the entire script above runs within a schema other than "dbo". Let's just assume the schema is "company1". And when the stored procedure is called, I get an error from the SELECT statement, which says that the Configuration table cannot be found. I'm guessing this is because MS SQL by default looks for tables within the same schema as the location of the stored procedure, and not within the calling context.
My question is this:
Is there some option or parameter or switch of some kind that will
tell MS SQL to look for tables in the "caller's default schema" and
not within the schema that procedure itself is stored in?
If not,
what would you recommend? I don't really want to prefix the tables
with the schema name, because it would be kind of unflexible to do
that. So I'm thinking about using dynamic sql (and the schema_name()
function which returns the correct value even within the procedure),
but I am just not experienced enough with MS SQL to construct the
proper syntax.
It would be a tad more efficient to explicitly specify the schema name. And generally speaking, schema's are mainly used to divide a database into logical area's. I would not anticipate on tables schema-hopping often.
Regarding your question, you might want to have a look at the 'execute as' documentation on msdn, since it allows to explicitly control your execution context.
I ended up passing the schema name to my script as a property on the command line for the "sqlcmd" command. Like this:
C:/> sqlcmd -vSCHEMANAME=myschema -imysqlfile
In the SQL script I can then access this variable like this:
SELECT * from $(SCHEMANAME).myTable WHERE.... etc
Not quite as flexible as dynamic sql, but "good enough" as it were.
Thanks all for taking time to respond.
I have about half a dozen generic, but fairly complex stored procedures and functions that I would like to use in a more generic fashion.
Ideally I'd like to be able to pass the table name as a parameter to the procedure, as currently it is hard coded.
The research I have done suggests I need to convert all existing SQL within my procedures to use dynamic SQL in order to splice in the dynamic table name from the parameter, however I was wondering if there is a easier way by referencing the table in another way?
For example:
SELECT * FROM #MyTable WHERE...
If so, how do I set the #MyTable variable from the table name?
I am using SQL Server 2005.
Dynamic SQL is the only way to do this, but I'd reconsider the architecture of your application if it requires this. SQL isn't very good at "generalized" code. It works best when it's designed and coded to do individual tasks.
Selecting from TableA is not the same as selecting from TableB, even if the select statements look the same. There may be different indexes, different table sizes, data distribution, etc.
You could generate your individual stored procedures, which is a common approach. Have a code generator that creates the various select stored procedures for the tables that you need. Each table would have its own SP(s), which you could then link into your application.
I've written these kinds of generators in T-SQL, but you could easily do it with most programming languages. It's pretty basic stuff.
Just to add one more thing since Scott E brought up ORMs... you should also be able to use these stored procedures with most sophisticated ORMs.
You'd have to use dynamic sql. But don't do that! You're better off using an ORM.
EXEC(N'SELECT * from ' + #MyTable + N' WHERE ... ')
You can use dynamic Sql, but check that the object exists first unless you can 100% trust the source of that parameter. It's likely that there will be a performance hit as SQL server won't be able to re-use the same execution plan for different parameters.
IF OBJECT_ID(#tablename, N'U') IS NOT NULL
BEGIN
--dynamic sql
END
ALTER procedure [dbo].[test](#table_name varchar(max))
AS
BEGIN
declare #tablename varchar(max)=#table_name;
declare #statement varchar(max);
set #statement = 'Select * from ' + #tablename;
execute (#statement);
END
Using SQL Server 2005, I'd like to run a stored procedure and insert all of the results into a new table.
I'd like the new table to have its columns automatically configured based upon the data returned by the stored procedure.
I am familiar with using the SELECT ... INTO syntax:
SELECT * INTO newtable FROM oldtable
Is this possible?
Edit for clarification: I'm hoping to accomplish something like:
Select * INTO newtable FROM exec My_SP
The only way to do this is w/ OPENROWSET against the local server:
SELECT * INTO #temp
FROM OPENROWSET (
'SQLOLEDB'
, 'Server=(local);TRUSTED_CONNECTION=YES;'
, 'SET FMTONLY OFF EXEC database.schema.procname'
) a
But this is kind of a last-ditch-gotta-do-it-damn-the-consequences kind of method. It requires elevated permissions, won't work for all procedures, and is generally inefficient.
More info and some alternatives here: http://www.sommarskog.se/share_data.html
This seems like a horrible design. You're really going to create a new table to store the results of a stored procedure, every time the stored procedure is called? And you really can't create the table in advance because you have absolutely no idea what kind of output the stored procedure has? What if the stored procedure returns multiple resultsets? What if it has side effects?
Okay, well, if that's what you really want to do...
One way to accomplish this is to use your local server as a linked server and utilize OPENQUERY. First you need to make sure your local server is configured for data access:
EXEC sp_serveroption 'local server name', 'DATA ACCESS', true;
Then you can do something like this:
SELECT * INTO dbo.newtable
FROM OPENQUERY('local server name', 'EXEC yourdb.dbo.yourproc;');
PS How are you going to write code that is going to perform SELECT INTO into a new table name every time (because you can only do SELECT INTO once)? Dynamic SQL? What happens if two users run this code at the same time? Does one of them win, and the other one just gets an error message?
A variation of the same is
create table somename
select * from wherever;
I just ran into a strange thing...there is some code on our site that is taking a giant SQL statement, modifying it in code by doing some search and replace based on some user values, and then passing it on to SQL Server as a query.
I was thinking that this would be cleaner as a parameterized query to a stored proc, with the user values as the parameters, but when I looked more closely I see why they might be doing it...the table that they are selecting from is variably dependant on those user values.
For instance, in one case if the values were ("FOO", "BAR") the query would end up being something like "SELECT * FROM FOO_BAR"
Is there an easy and clear way to do this? Everything I'm trying seems inelegant.
EDIT: I could, of course, dynamically generate the sql in the stored proc, and exec that (bleh), but at that point I'm wondering if I've gained anything.
EDIT2: Refactoring the table names in some intelligent way, say having them all in one table with the different names as a new column would be a nice way to solve all of this, which several people have pointed out directly, or alluded to. Sadly, it is not an option in this case.
First of all, you should NEVER do SQL command compositions on a client app like this, that's what SQL Injection is. (Its OK for an admin tool that has no privs of its own, but not for a shared use application).
Secondly, yes, a parametrized call to a Stored procedure is both cleaner and safer.
However, as you will need to use Dynamic SQL to do this, you still do not want to include the passed string in the text of the executed query. Instead, you want to used the passed string to look up the names of the actual tables that the user should be allowed to query in the way.
Here's a simple naive example:
CREATE PROC spCountAnyTableRows( #PassedTableName as NVarchar(255) ) AS
-- Counts the number of rows from any non-system Table, *SAFELY*
BEGIN
DECLARE #ActualTableName AS NVarchar(255)
SELECT #ActualTableName = QUOTENAME( TABLE_NAME )
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME = #PassedTableName
DECLARE #sql AS NVARCHAR(MAX)
SELECT #sql = 'SELECT COUNT(*) FROM ' + #ActualTableName + ';'
EXEC(#SQL)
END
Some have fairly asked why this is safer. Hopefully, little Bobby Tables can make this clearer:
0
Answers to more questions:
QUOTENAME alone is not guaranteed to be safe. MS encourages us to use it, but they have not given a guarantee that it cannot be out-foxed by hackers. FYI, real Security is all about the guarantees. The table lookup with QUOTENAME, is another story, it's unbreakable.
QUOTENAME is not strictly necessary for this example, the Lookup translation on INFORMATION_SCHEMA alone is normally sufficient. QUOTENAME is in here because it is good form in security to include a complete and correct solution. QUOTENAME in here is actually protecting against a distinct, but similar potential problem know as latent injection.
I should note that you can do the same thing with dynamic Column Names and the INFORMATION_SCHEMA.COLUMNS table.
You can also bypass the need for stored procedures by using a parameterized SQL query instead (see here: https://learn.microsoft.com/en-us/dotnet/api/system.data.sqlclient.sqlcommand.parameters?view=netframework-4.8). But I think that stored procedures provide a more manageable and less error-prone security facility for cases like this.
(Un)fortunately there's no way of doing this - you can't use table name passed as a parameter to stored code other than for dynamic sql generation. When it comes to deciding where to generate sql code, I prefer application code rather that stored code. Application code is usually faster and easier to maintain.
In case you don't like the solution you're working with, I'd suggest a deeper redesign (i.e. change the schema/application logic so you no longer have to pass table name as a parameter anywhere).
I would argue against dynamically generating the SQL in the stored proc; that'll get you into trouble and could cause injection vulnerability.
Instead, I would analyze all of the tables that could be affected by the query and create some sort of enumeration that would determine which table to use for the query.
Sounds like you'd be better off with an ORM solution.
I cringe when I see dynamic sql in a stored procedure.
One thing you can consider is to make a case statement that contains the same SQL command you want, once for each valid table, then pass as a string the table name into this procedure and have the case choose which command to run.
By the way as a security person the suggestion above telling you to select from the system tables in order to make sure you have a valid table seems like a wasted operation to me. If someone can inject passed the QUOTENAME() then then injection would work on the system table just as well as on the underlying table. The only thing this helps with it to ensure it is a valid table name, and I think the suggestion above is a better approach to that since you are not using QUOTENAME() at all.
Depending on whether the set of columns in those tables is the same or different, I'd approach it in two ways in the longer term:
1) if they the same, why not create a new column that would be used as a selector, whose value is derived from the user-supplied parameters ? (is it a performance optimization?)
2) if they are different, chances are that handling of them is also different. As such, it seems like splitting the select/handle code into separate blocks and then calling them separately would be a most modular approach to me. You will repeat the "select * from" part,
but in this scenario the set of tables is hopefully finite.
Allowing the calling code to supply two arbitrary parts of the table name to do a select from feels very dangerous.
I don't know the reason why you have the data spread over several tables, but it sounds like you are breaking one of the fundamentals. The data should be in the tables, not as table names.
If the tables have more or less the same layout, consider if it would be best to put the data in a single table instead. That would solve your problem with the dynamic query, and it would make the database layout more flexible.
Instead of Querying the tables based on user input values, you can pick the procedure instead.
that is to say
1. Create a procedure FOO_BAR_prc and inside that you put the query 'select * from foo_bar' , that way the query will be precompiled by the database.
2. Then based on the user input now execute the correct procedure from your application code.
Since you have around 50 tables, this might not be a feasible solution though as it would require lot of work on your part.
In fact, I wanted to know how to pass table name to create a table in stored procedure. By reading some of the answers and attempting some modification at my end, I finally able to create a table with name passed as parameter. Here is the stored procedure for others to check any error in it.
USE [Database Name]
GO
/****** Object: StoredProcedure [dbo].[sp_CreateDynamicTable] Script Date: 06/20/2015 16:56:25 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROCEDURE [dbo].[sp_CreateDynamicTable]
#tName varchar(255)
AS
BEGIN
SET NOCOUNT ON;
DECLARE #SQL nvarchar(max)
SET #SQL = N'CREATE TABLE [DBO].['+ #tName + '] (DocID nvarchar(10) null);'
EXECUTE sp_executesql #SQL
END
#RBarry Young
You don't need to add the brackets to #ActualTableName in the query string because it is already included in the result from the query in the INFORMATION_SCHEMA.TABLES. Otherwise, there will be error(s) when executed.
CREATE PROC spCountAnyTableRows( #PassedTableName as NVarchar(255) ) AS
-- Counts the number of rows from any non-system Table, SAFELY
BEGIN
DECLARE #ActualTableName AS NVarchar(255)
SELECT #ActualTableName = QUOTENAME( TABLE_NAME )
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME = #PassedTableName
DECLARE #sql AS NVARCHAR(MAX)
--SELECT #sql = 'SELECT COUNT(*) FROM [' + #ActualTableName + '];'
-- changed to this
SELECT #sql = 'SELECT COUNT(*) FROM ' + #ActualTableName + ';'
EXEC(#SQL)
END
I would avoid dynamic SQL at all costs.
Isn't the most elegant solution but does the job perfectly.
PROCEDURE TABLE_AS_PARAMTER (
p_table_name IN VARCHAR2
) AS
BEGIN
CASE p_table_name
WHEN 'TABLE1' THEN
UPDATE TABLE1
SET
COLUMN1 =1
WHERE
ID =1;
WHEN 'TABLE2' THEN
UPDATE TABLE1
SET
COLUMN1 =1
WHERE
ID =2;
END CASE;
COMMIT;
EXCEPTION
WHEN OTHERS THEN
ROLLBACK
END TABLE_AS_PARAMTER;
How would you handle cross database queries in different environments. For example, db1-development and db2-development, db1-production and db2-production.
If I want to do a cross-database query in development from db2 to db1 I could use the fully qualified name, [db1-development].[schema].[table]. But how do I maintain the queries and stored procedures between the different environments? [db1-development].[schema].[table] will not work in production because the database names are different.
I can see search and replace as a possible solution but I am hoping there is a more elegant way to solve this problem. If there are db specific solutions, I am using SQL Server 2005.
Why are the database names different between dev and prod? It'd, obviously, be easiest if they were the same.
If it's a single table shared, then you could create a view over it - which only requires that you change that view when moving to production.
Otherwise, you'll want to create a SYNONYM for the objects, and make sure to always reference that. You'll still need to change the SYNONYM creation scripts, but that can be done in a build script fairly easily, I think.
For this reason, it's not practical to use different names for development and production databases. Using the same db name on development, production, and optionally, acceptance/Q&A environments, makes your SQL code much easier to maintain.
However, if you really have to, you could get creative with views and dynamic SQL. For example, you put the actual data retrieval query inside a view, and then you select like this:
declare #environment varchar(10)
set #environment = 'db-dev' -- input parameter, comes from app layer
declare #sql varchar(8000)
set #sql = 'select * from ' + #environment + '.dbo.view'
execute(#sql)
But it's far from pretty...