I am looking for a way of dealing with the following situation:
We have a database server with multiple databases on it (all have the same schema, different data).
We are looking for a way to query across all the databases (and for it to be easy to configure, as more databases may be added at any time). This data access must be realtime.
Say, as an example, you have an application that inserts orders - each application has its own DB etc. What we are then looking for is an efficient way for a single application to then access the order information in all the other databases in order to query it and subsequently action it.
My searches to date have not revealed very much, however I think I may just be missing the appropriate keywords in order to find the correct info...
You must specify the database name before any database object.
Single database:
SELECT * FROM [dbo].[myTable]
Multiple dabases:
SELECT * FROM [DB01].[dbo].[myTable]
UNION ALL
SELECT * FROM [DB02].[dbo].[myTable]
UNION ALL
SELECT * FROM [DB03].[dbo].[myTable]
It's not going to be the cleanest solution ever, but you could define a view on a "Master database" (if your individual databases are not going to stay constant) that includes the data from the individual databases, and allows you to execute queries on a single source.
For example...
CREATE VIEW vCombinedRecords AS
SELECT * FROM DB1.dbo.MyTable
UNION ALL
SELECT * FROM DB2.dbo.MyTable
Which allows you to do...
SELECT * FROM vCombinedRecords WHERE....
When your databases change, you just update the view definition to include the new tables.
You can build the union dynamically:
select name from sys.databases
and then check if the database has the table:
select name from [dbname_from_above].sys.tables where name = 'YourTable'
That gives you all databases for the union. You can build the query client side or in dynamic SQL.
Note - My answer below received a couple down votes, but only one comment giving any reason why it might be down-voted. The comment was that this answer is very similar to the accepted answer, but even less performant. I disagree with this opinion and I reproduce my response here - in the actual answer - so that anyone else reading my answer might have a better chance at seeing why this is not the same as the accepted answer at all, and in fact better addresses the original question.
My response to the suggestion this is similar to the accepted answer:
on the contrary - the original question notes that new databases are added regularly. The accepted solution will require maintenance each time a new database is added. The solution here will work regardless of whether any new databases are added (in line with the original question that states they all have the same schema). Further, the accepted answer requires you to duplicate the query once per database queried. If the query is complex, that gets ugly fast. The proposal here ensures a single source of truth for the logic being used in the query.
And the answer itself:
Shooting from the hip here.
use master;
go
create table #Temp (sourceDBName varchar(128), colA varchar(128), colB varchar(128));
exec sp_MSforeachDB ' USE [?];
insert into #Temp
SELECT DISTINCT
''?'',
tableA.colA,
tableB.colB
FROM tableA JOIN tableB on some_conditions
WHERE someCol LIKE ''%some_term%''
'
select sourceDBName, colA, colB from #Temp order by 1, 2, 3;
drop table #Temp;
This logic should allow you to apply a single query to all databases. To use it though, you will want to add logic to filter out system databases, or explicitly include only the databases you specify. To achieve that, you might like to put this logic into a stored procedure which then returns a result set, so in the end, your call to this logic is a select statement that returns a rowset you can join, filter, etc.
Check out https://www.mssqltips.com/sqlservertip/2855/sql-server-multi-database-query-with-registered-servers/
SELECT * FROM (
SELECT
##SERVERNAME as [ServerName],
##version [Version],
Format(##CONNECTIONS,'N0') [Conections],
Format(##CPU_BUSY ,'N0')[CPUBusy]
) SQLInfo
LEFT JOIN (
SELECT
##SERVERNAME AS [ServerName],
SERVERPROPERTY('ProductVersion') [Version Build],
SERVERPROPERTY ('Edition') AS [Edition],
SERVERPROPERTY('ProductLevel') AS [Service Pack],
CASE SERVERPROPERTY('IsIntegratedSecurityOnly')
WHEN 0 THEN 'SQL Server and Windows Authentication mode'
WHEN 1 THEN 'Windows Authentication mode'
END AS [Server Authentication],
CASE SERVERPROPERTY('IsClustered')
WHEN 0 THEN 'False'
WHEN 1 THEN 'True'
END AS [Is Clustered?],
SERVERPROPERTY('ComputerNamePhysicalNetBIOS') AS [Current Node Name],
SERVERPROPERTY('Collation') AS [ SQL Collation],
[cpu_count] AS [CPUs],
[physical_memory_kb]*0.00000095367432 AS [RAM (GB)]
FROM [sys].[dm_os_sys_info]
) SQLInfo2 on SQLInfo.[ServerName]=SQLInfo2.[ServerName]
LEFT JOIN (
SELECT
##SERVERNAME as [ServerName],
NodeName,
Status_Description,
is_Current_Owner
FROM [MASTER].[sys].[fn_virtualservernodes]()
)Clusterinfo on SQLInfo.[ServerName]=Clusterinfo.[ServerName]
Related
Is it possible to get table name in select statement?
Example:
SELECT Id, "Users" as TableName
FROM Users
I don't want to write "Users" manually but to determine it based on FROM statement.
I'm doing some migrations from one database to another (with different structure), and I have mapping table that stores old table names so that's why I need this (it's less error prone).
You can use following sub-query to get this information (works on SQL Server 2012+):
SELECT *,
(select object_name(ind.object_id)
from sys.fn_PhysLocCracker(%%physloc%%) plc
INNER JOIN SYS.DM_DB_DATABASE_PAGE_ALLOCATIONS(DB_ID(),null,null,null,null) ind
ON ind.allocated_page_file_id = plc.file_id
AND ind.allocated_page_page_id = plc.page_id) as table_name
FROM [your table]
It is painfully slow, caching on the side page allocation will speed things up.
Alternatively, if you want to do it for all tables, you can use following stored procedure:
USE [DB_NAME]
EXEC sp_msforeachtable 'SELECT Id, ''?'' AS [TableName] FROM ?'
Maybe it will be helpful for somebody :)
I have developed a SQL query in SSMS-2017 like this:
DECLARE #property NVARCHAR(MAX) = #p;
SET #property = REPLACE(#property, '''', '');
DECLARE #propList TABLE (hproperty NUMERIC(18, 0));
IF CHARINDEX('SELECT', #property) > 0 OR CHARINDEX('select', #property) > 0
BEGIN
INSERT INTO #propList
EXECUTE sp_executesql #property;
END;
ELSE
BEGIN
DECLARE #x TABLE (val NUMERIC(18, 0));
INSERT INTO #x
SELECT CONVERT(NUMERIC(18, 0), strval)
FROM dbo.StringSplit(#property, ',');
INSERT INTO #propList
SELECT val
FROM #x;
END;
SELECT ...columns...
FROM ...tables and joins...
WHERE ...filters...
AND HMY IN (SELECT hproperty FROM #propList)
The issue is, it is possible that the value of the parameter #p can be a list of IDs (Example: 1,2,3,4) or a direct select query (Example: Select ID from mytable where code='A123').
The code is working well as shown above. However it causes a problem in our system (as we use Yardi7-Voyager), and we need to leave only the select statement as a query. To manage it, I was planning to create a function and use it in the where clause like:
WHERE HMY IN (SELECT myFunction(#p))
However I could not manage it as I see I cannot execute a dynamic query in an SQL Function. Then I am stacked. Any idea at this point to handle this issue will be so appreciated.
Others have pointed out that the best fix for this would be a design change, and I agree with them. However, I'd also like to treat your question as academic and answer it in case any future readers ever have the same question in a use case where a design change wouldn't be possible/desirable.
I can think of two ways you might be able to do what you're attempting in a single select, as long as there are no other restrictions on what you can do that you haven't mentioned yet. To keep this brief, I'm just going to give you psuedo-code that can be adapted to your situation as well as those of future readers:
OPENQUERY (or OPENROWSET)
You can incorporate your code above into a stored procedure instead of a function, since stored procedures DO allow dynamic sql, unlike functions. Then the SELECT query in your app would be a SELECT from OPENQUERY(Execute Your Stored Prodedure).
UNION ALL possibilities.
I'm about 99% sure no one would ever want to use this, but I'm mentioning it to be as academically complete as I know how to be.
The second possibility would only work if there are a limited, known, number of possible queries that might be supported by your application. For instance, you can only get your Properties from either TableA, filtered by column1, or from TableB, filtered by Column2 and/or Column3.
Could be more than these possibilities, but it has to be a limited, known quantity, and the more possibilities, the more complex and lengthy the code will get.
But if that's the case, you can simply SELECT from a UNION ALL of every possible scenario, and make it so that only one of the SELECTs in the UNION ALL will return results.
For example:
SELECT ... FROM TableA WHERE Column1=fnGetValue(#p, 'Column1')
AND CHARINDEX('SELECT', #property) > 0
AND CHARINDEX('TableA', #property) > 0
AND CHARINDEX('Column1', #property) > 0
AND (Whatever other filters are needed to uniquely identify this case)
UNION ALL
SELECT
...
Note that fnGetValue() isn't a built-in function. You'd have to write it. It would parse the string in #p, find the location of 'Column1=', and return whatever value comes after it.
At the end of your UNION ALL, you'd need to add a last UNION ALL to a query that will handle the case where the user passed a comma-separated string instead of a query, but that's easy, because all the steps in your code where you populated table variables are unnecessary. You can simply do the final query like this:
WHERE NOT CHARINDEX('SELECT', #p) > 0
AND HMY IN (SELECT strval FROM dbo.StringSplit(#p, ','))
I'm pretty sure this possibility is way more work than its worth, but it is an example of how, in general, dynamic SQL can be replaced with regular SQL that simply covers every possible option you wanted the dynamic sql to be able to handle.
I have the following query:
SELECT * FROM MailingList
There are about 20+ columns in the MailingList table, one which is called Address. This column has some fields which contain commas, which I need to take out. So I updated my query to:
SELECT REPLACE(Address, ',', '') AS Address, * FROM MailingList
But now I have two Address columns. Is there a way to only display one Address column while still using the wildcard (*) for all the other columns?
There is not a way to do this, though listing the columns you want explicitly is a good idea anyway.
You can trick as following query:
Get the data into a temp table
Drop the cloumns that are not needed
Get results and drop temp table
SELECT *, REPLACE(Address, ',', '') AS Address2
INTO #TempTable
FROM MailingList
ALTER TABLE #TempTable
DROP COLUMN [Address]
SELECT * FROM #TempTable
DROP TABLE #TempTable
I agree with Shadow - avoid using the * wild card if you can...
I know listing out ALL of the columns in select statement for big tables is a pain so here is a quick short cut you may not be aware of: In SQL Server Management Studio, browse through the object explorer and find the table you want to select from (MailingList). Right-click it to view the context menu and choose "Script Table as" then "SELECT TO" then "New Query Editor Window". This will create a new select statement with each column spelled out. In the future, use this method to create select statements, queries, procedures, etc. rather then the * wildcard. Performance is better and it just looks nicer :-)
Then you can solve your alias issue with the replace function.
Is there any possibility to write delete query inside Where clause.
Example:
Select ID,Name From MyTable Where ID IN(Delete From MyTable)
It may be crazy, but let me explain my situation. In our reporting tool, we are supporting to enter SQL where query.
We will use our own Select and From Clause query and combine the user's where query input.
Example:
Select ID,Name From MyTable Where ("Query typed by user")
Here, user can type any kind of where query filter..
If he types like ID=100 our final query becomes like this
Select ID,Name From MyTable Where (ID=100)
One of our customer asked us what will happen if anyone type the delete query as where query filter. he feels this may be the security hole..so we have tried that kind of possibility in our dev environment. But the sql returns error for the following query.
Select ID,Name From MyTable Where ID IN(Delete From MyTable)
So finally, my question is, is there any other possibility to write Delete Query inside Where clause or Select clause.. If it possible, how can I restrict it?
Yes. They can run a delete. They can type:
1 = 1; DELETE FROM MY_TABLE;
Or even worse in some ways, (since you should have backups):
1 = 0 UNION SELECT SOCIAL_SECURITY_NUMBER, CREDIT_CARD_NUMBER, OTHER_SENSITIVE_DATA FROM MY_SENSITIVE_TABLE;
Now, in your case its hard to validate. Normally if you are just passing a value to filter on you can use parameterised sql to save yourself. You however also need to let the user select a column. In cases like these, usually we use a drop down to allow the user to select a predefined list of columns and then validate the column name server side. We give the user a text box to enter the value to match and then parameterise that.
It's not quite possible. But he can do something like this :
Select ID,Name From MyTable Where (ID=100); (DELETE FROM MyTable Where 1 = 1)
by using ID=100); (DELETE FROM MyTable Where 1 = 1 instead of ID=100
I believe what your customer is talking about is SQL injection, as long as you have taken appropriate methods to block other queries from running after your select statement is done, then you should have no problem in letting them type whatever it is that you want.
From my experience there is no way to delete anything when you are doing a select statement.
Just make sure you have query terminator characters so they don't write something like the following.
select column1,column2, from myTable where ID in (1,2); delete from my table
this would be a valid worry from your customer if you aren't taking proper steps to prevent sql injection from happening.
You could have your SQL reporting tool just not have update, or delete permission and just have it have Read permission. However, it is up to you guys have you handle your sql injection security.
Platform: SQL Server 2008
Language: TSQL
I have a number of queries that currently take the general form of (for simplicity sake)
-- Sample begin results
SELECT * from DB01.dbo.table UNION ALL
SELECT * from DB02.dbo.table UNION ALL --many other databases follow with same syntax
How can I modify these queries such that, when a new database comes into existence (named, say DB39C), I ensure that my queries already includes those new records?
--Sample end results
SELECT * from DB01.dbo.table UNION ALL
SELECT * from DB02.dbo.table UNION ALL
SELECT * from DB39C.dbo.table -- this was created as soon as a new database came into existence
I am looking to make sure programmatically, that this happens without my awareness as new databases are added quite regularly and I need the queries I rely on to keep pace.
You might want to have a look at using something like
SELECT name AS DATABASENAME
FROM master.dbo.sysdatabases
and creating dynamic queries
sys.databases (Transact-SQL)