I have a COTS application database that creates a new table every week that follows a specific naming convention and always contains the same columns. I have written a query to select the tables within a certain time range:
DECLARE #today VARCHAR(8)
SET #today = CONVERT(VARCHAR(8),GETDATE(),112)
DECLARE #monthold VARCHAR(8)
SET #monthold = CONVERT(VARCHAR(8),dateadd(day,-31,#today),112)
SELECT name
FROM sys.tables
WHERE name <= 'Event_Audit'+ #today AND name >= 'Event_Audit'+ #monthold
AND name like 'Event_Audit%'
ORDER BY name DESC
Now I am looking for a way to have a query call each of the tables that is selected to aggregate the data. This will be used for SSRS reporting.
Something like this where table1, table2, etc. through all the included tables will get populated (whether it is 4, 5, or more):
SELECT *
FROM table1
UNION ALL
SELECT *
FROM table2
UNION ALL
...
ORDER BY EVENT_DATE DESC
I can't create views, new tables, or make any changes to existing tables.
Poor design will lead to hard work every time, this one is no exception. I'm guessing by now it's too late to give up the idea of creating the same table over and over again for each month.
You do not need to create a new or temporary table to execute sp_executeSql, you only need to generate the sql script:
DECLARE #today char(8) = CONVERT(char(8),GETDATE(),112),
#monthold char(8) = CONVERT(char(8),dateadd(day,-31,GETDATE()),112),
#sql nvarchar(max) = ''
SELECT #Sql = #Sql + ' UNION ALL SELECT * FROM ' + name
FROM sys.tables
WHERE name <= 'Event_Audit'+ #today AND name >= 'Event_Audit'+ #monthold
AND name like 'Event_Audit%'
ORDER BY name DESC
SET #SQL = STUFF(#SQL, 1, 11, '') + -- remove the first UNION ALL
' ORDER BY EVENT_DATE DESC' -- add the ORDER BY
PRINT #Sql
--EXEC sp_executeSql #Sql
Once you've printed the #Sql and checked it's ok, unremark the EXEC line and run your script.
Related
I have 1000 tables in my SQL Server database. Table names like CIP_01_12_2022_10_15_20 that I have 1000 table. Table name saved as data time(CIP_DD_MM_YYYY_HH_MM_SS).
So I need to delete like between particular dates.
Tables named like below
CIP_01_12_2022_10_15_20
CIP_01_12_2022_10_15_25
CIP_01_12_2022_10_15_35
CIP_01_12_2022_10_15_45
CIP_01_12_2022_10_15_55
CIP_01_12_2022_10_15_58
CIP_01_12_2022_10_15_59
CIP_01_12_2022_10_16_20
CIP_01_12_2022_10_16_25
CIP_02_12_2022_10_15_20
In the above, I have to delete between two dates. For example I have to delete between these dates 01-12-2022 00:00:00 to 01-12-2022 11:59:59 delete all tables except 2nd December 2022 table.
I think you can use a cursor in order to do that:
DECLARE table_cursor CURSOR FOR
SELECT name
FROM sys.tables
WHERE name LIKE 'CIP_%'
Then do loop on it.
EXEC('DROP TABLE ' + #<yourFetchVariable>)
Below is an example that uses a strict pattern matching for the dated tables and extracts the date value from the name for use in the date range criteria (a single date per your sample data).
DECLARE
#StartDate date = '20221201' --ISO 8601 date format to avoid ambiguity
, #EndDate date = '20221201'
, #DropScript nvarchar(MAX);
WITH date_tables AS (
SELECT name AS TableName, TRY_CAST(SUBSTRING(name, 11, 4)+SUBSTRING(name, 8, 2)+SUBSTRING(name, 5, 2) AS date) AS TableDate
FROM sys.tables
WHERE name LIKE N'CIP[_][0-9][0-9][_][0-9][0-9][_][0-9][0-9][0-9][0-9][_][0-9][0-9][_][0-9][0-9][_][0-9][0-9]'
)
SELECT #DropScript = STRING_AGG(N'DROP TABLE ' + TableName, N';') + N';'
FROM date_tables
WHERE TableDate BETWEEN #StartDate AND #EndDate;
EXEC sp_executesql #DropScript;
GO
I've had like lists of 300 tables.
Every table has date column INSERT_DATE (of type Varchar).
And I wanted to check if the table is having the same date.
So that let's say - first table:
INSERT_DATE
20221231
20221231
20221231
Second table:
INSERT_DATE
20221231
20221231
20221231
Third table:
INSERT_DATE
20221230
20221230
20221230
Desired results of my query:
Table Name
Date
FIRST_TABLE
20221231
SECOND_TABLE
20221231
THIRD_TABLE
20221230
I'm confused on how to make it all "automatic" rather than querying one by one.
Like it'd take forever for me to query
SELECT TOP 1 INSERT_DATE
FROM FIRST_TABLE
then
SELECT TOP 1 INSERT_DATE
FROM SECOND_TABLE
then
SELECT TOP 1 INSERT_DATE
FROM THIRD_TABLE
Until it'd get to 300 tables.
Btw , each of the tables would have only and should have only 1 date. Because the procedure was to truncate insert, so if i saw some tables not matching the dates like the others, i knew that table have some problems and i'd have to run it manually.
It sounds like you need to scan the database's INFORMATION_SCHEMA to select all tables of interest and then build and execute dynamic SQL that queries each selected table and UNIONs the result.
It was unclear whether you wanted the top/max insert date or a distinct list of insert dates, so I have included SQL templates for both.
The following will select all tables having an INSERT_DATE column, and will build up and execute the necessary dynamic SQL.
-- Select tables
DECLARE #SelectedTables TABLE(TableName SYSNAME)
INSERT #SelectedTables
SELECT T.TABLE_NAME
FROM INFORMATION_SCHEMA.TABLES T
JOIN INFORMATION_SCHEMA.COLUMNS C
ON C.TABLE_NAME = T.TABLE_NAME
AND C.TABLE_SCHEMA = T.TABLE_SCHEMA
WHERE T.TABLE_TYPE = 'BASE TABLE'
AND C.COLUMN_NAME = 'INSERT_DATE'
-- Define SQL Templates
DECLARE #SqlTemplateMax NVARCHAR(MAX) = 'SELECT TABLE_NAME = <TableNameString>, MAX(INSERT_DATE) AS INSERT_DATE FROM <TableName>
'
DECLARE #SqlTemplateDistinct NVARCHAR(MAX) = 'SELECT DISTINCT TABLE_NAME = <TableNameString>, INSERT_DATE FROM <TableName>
'
DECLARE #UnionAll NVARCHAR(10) = 'UNION ALL
'
DECLARE #OrderBy NVARCHAR(MAX) = 'ORDER BY
TABLE_NAME,
INSERT_DATE DESC
'
-- Build dynamic SQL
DECLARE #Sql NVARCHAR(MAX) = (
SELECT STRING_AGG(
REPLACE(REPLACE(#SqlTemplateDistinct
,'<TableNameString>', QUOTENAME(T.TableName, ''''))
,'<TableName>', QUOTENAME(T.TableName))
, #UnionAll)
FROM #SelectedTables T
)
SET #Sql = #Sql + #OrderBy
-- Print and Execute dynamic SQL
PRINT #Sql
EXEC (#Sql)
This will generate and execure the following dynamic SQL:
SELECT DISTINCT TABLE_NAME = 'FIRST_TABLE', INSERT_DATE FROM [FIRST_TABLE]
UNION ALL
SELECT DISTINCT TABLE_NAME = 'SECOND_TABLE', INSERT_DATE FROM [SECOND_TABLE]
UNION ALL
SELECT DISTINCT TABLE_NAME = 'THIRD_TABLE', INSERT_DATE FROM [THIRD_TABLE]
ORDER BY
TABLE_NAME,
INSERT_DATE DESC
See this db<>fiddle for a working example.
Note the use of the QUOTENAME() function above. Its use is good practice to ensure proper escaping of potential special characters when injecting 'strings' and [identifiers] into dynamic SQL.
The STRING_AGG() function joins the generated SQL fragments together, separated by the second parameter (UNION ALL in this case). If you are using an older version of SQL server that does not support STRING_AGG(), you will need to resort for using a FOR XML hack that was commonly used in the past to concatenate strings. See this db<>fiddle for the FOR XML version of the above.
Dears,
We have a database that creates a new table for each new day. The naming of the tables follows this pattern: History_tbl_[year]_[month]_[day]. A sample name of the tables for the last 5 days for example is:
History_tbl_2021_10_02
History_tbl_2021_10_01
History_tbl_2021_09_30
History_tbl_2021_09_29
History_tbl_2021_09_28
My goal is to be able to query all the tables from a given date range at once. I can manually select the tables with union all, but it takes lots of time especially if I want to do a long date range. Is there a better way to solve this?
Note: Unfortunately, I don't have the privilege to change the structure and make all data being stored in a single table.
One bare-bones method to generate a list of unioned queries and dynamically execute them would be as follows, tweak as necessary:
declare #from date='20211001', #to date='20211004', #sql nvarchar(max)='';
with d as (
select DateAdd(day, number, #from) dt
from master..spt_values
where type = 'P'
and DateAdd(day, number, #from) <= #to
)
select #sql=String_Agg(qry,' ')
from (
select 'select col1, col2, col3 from History_tbl_'
+ Concat(Year(dt),'_',Right(Concat('0',month(dt)),2),'_', Right(Concat('0',Day(dt)),2))
+ Iif( dt=Max(dt) over() ,'', ' union all') qry
from d
)x
exec sp_executesql #sql
Note that the CTE generates the date range on the fly, ideally you would use a permanent calendar table
I'm Trying to create a stored procedure that will allow me to pick a start date and end date to get data from and to have a variable table name to write this data to.
I would like to pass in the two dates and the table name as parameters in the stored procedure. Here is that part I'm stuck on. I took out the stored procedure to try and get this working. this way I can see the lines the error is on.
DECLARE #MinDateWeek DATETIME
SELECT #MinDateWeek= DATEADD(WEEK, DATEDIFF(WEEK,0,GETDATE()), -7)
DECLARE #MaxDateWeek DATETIME
SELECT #MaxDateWeek= DATEADD(WEEK, DATEDIFF(WEEK,0,GETDATE()),0)
DECLARE #SQLCommand NVARCHAR(MAX)
SET #SQLCommand = ' --ERROR ON THIS LINE
-- Getting how much space is used in the present
DECLARE #Present Table (VMName NVARCHAR(50), UseSpace float(24))
INSERT INTO #Present
SELECT VMName
,SUM(CapacityGB-FreeSpaceGB)
FROM VMWareVMGuestDisk
GROUP BY VMName;
-- Getting how much space was used at the reference date
DECLARE #Past Table (VMName NVARCHAR(50), UseSpace float(24))
INSERT INTO #Past
SELECT VMName
,SUM(CapacityGB-FreeSpaceGB)
FROM VMWareVMGuestDisk
WHERE Cast([Date] AS VARCHAR(20))= '''+CAST(#MinDateWeek AS varchar(20))+'''
GROUP BY VMName;
--Inserting the average growth(GB/DAY) between the 2 dates in a Temporary Table
CREATE TABLE #TempWeek (VMName NVARCHAR(50)
, CapacityGB float(24)
, GrowthLastMonthGB float(24)
, FreeSpace FLOAT(24) )
INSERT INTO #TempWeek
SELECT DISTINCT V.VMName
,SUM(V.CapacityGB)
,SUM(((W1.UseSpace-W2.UseSpace)/(DATEDIFF(DAY,'''+CONVERT(VARCHAR(50),#MaxDateWeek)+''','''+CONVERT(VARCHAR (50),#MaxDateWeek)+'''))))
,SUM(V.FreeSpaceGb)
FROM VMWareVMGuestDisk AS V
LEFT JOIN
#Present AS W1
ON
V.VMName=W1.VMName
LEFT JOIN
#Past AS W2
ON
W1.VMName=W2.VMName
WHERE (CONVERT(VARCHAR(15),Date))='''+CONVERT(VARCHAR(50),#MaxDateWeek)+'''
GROUP BY V.VMName;
-- Checking if there is already data in the table
TRUNCATE TABLE SAN_Growth_Weekly;
--insert data in permanent table
INSERT INTO SAN_Growth_Weekly (VMName,Datacenter,Cluster,Company,DaysLeft,Growth, Capacity,FreeSpace,ReportDate)
SELECT DISTINCT
G.VMName
,V.Datacenter
,V.Cluster
,S.Company
, DaysLeft =
CASE
WHEN G.GrowthLastMonthGB IS NULL
THEN ''NO DATA''
WHEN (G.GrowthLastMonthGB)<=0
THEN ''UNKNOWN''
WHEN (G.FreeSpace/G.GrowthLastMonthGB)>0 AND (G.FreeSpace/G.GrowthLastMonthGB) <=30
THEN ''Less then 30 Days''
WHEN (G.FreeSpace/G.GrowthLastMonthGB)>30 AND (G.FreeSpace/G.GrowthLastMonthGB)<=60 THEN ''Less then 60 Days''
WHEN (G.FreeSpace/G.GrowthLastMonthGB)>60 AND (G.FreeSpace/G.GrowthLastMonthGB)<=90
THEN ''Less then 90 Days''
WHEN (G.FreeSpace/G.GrowthLastMonthGB)>90 AND (G.FreeSpace/G.GrowthLastMonthGB)<=180 THEN ''Less then 180 Days''
WHEN (G.FreeSpace/G.GrowthLastMonthGB)>180 AND (G.FreeSpace/G.GrowthLastMonthGB)<=365 THEN ''Less then 1 Year''
ELSE ''Over 1 Year''
END
,G.GrowthLastMonthGB
,G.CapacityGB
,G.FreeSpace
,'''+#MaxDateWeek+'''
FROM #tempWeek AS G
RIGHT JOIN VMWareVMGuestDisk AS V
ON V.VMName = G.VMName COLLATE SQL_Latin1_General_CP1_CI_AS
LEFT JOIN Server_Reference AS S
ON G.VMName COLLATE SQL_Latin1_General_CP1_CI_AS=S.[Asset Name]
WHERE '''+CONVERT(VARCHAR(50),#MaxDateWeek)+'''= CONVERT(VARCHAR(50),V.Date);'
EXEC sp_executesql #SQLCommand;
The error I get is
Conversion failed when converting date and/or time from character
string.
Thanks for the help.
Are you forgetting to enclose your Group By in the dynamic sql?:
ALTER PROCEDURE SAN_DISK_GROWTH
#MaxDateWeek DATETIME ,
#MinDateWeek DATETIME
AS
BEGIN
DECLARE #SQLCommand NVARCHAR(MAX)
SELECT #SQLCommand = '
DECLARE #Present Table (VMName NVARCHAR(50), UseSpace float(24))
INSERT INTO #Present
SELECT VMName
,SUM(CapacityGB - FreeSpaceGB)
FROM VMWareVMGuestDisk
WHERE CONVERT(VARCHAR(15),Date) = '''
+ CONVERT(VARCHAR(50), #MaxDateWeek) + ''' GROUP BY VMName;'
END
Try specifying your date/time values as parameters to the dynamic SQL query. In other words, instead of converting the dates to a varchar, use parameters in the query:
WHERE #MaxDateWeek = V.Date;
And pass the parameters on the call to sp_executesql like so:
EXEC sp_executesql #SQLCommand,
'#MindateWeek datetime, #MaxDateWeek datetime',
#MinDateWeek = #MinDateWeek,
#MaxDateWeek = #MaxDateWeek
Then you won't have to convert your dates to strings.
Note that this does not work for dynamic table names or column names. Those need to be concatenated together as part of the dynamic SQL itself.
For example, if you had a table name variable like this:
declare #TableName sysname
set #TableName = 'MyTable'
And you wanted the dynamic SQL to retrieve data from that table, then you would need to build your FROM statement like this:
set #SQLCommand = N'SELECT ...
FROM ' + #TableName + N' WHERE...
This build the name into the SQL like so:
'SELECT ... FROM MyTable WHERE...'
I need to export data from a non-normalized database where there are multiple columns to a new normalized database.
One example is the Products table, which has 30 boolean columns (ValidSize1, ValidSize2 ecc...) and every record has a foreign key which points to a Sizes table where there are 30 columns with the size codes (XS, S, M etc...). In order to take the valid sizes for a product I have to scan both tables and take the value SizeCodeX from the Sizes table only if ValidSizeX on the product is true. Something like this:
Products Table
--------------
ProductCode <PK>
Description
SizesTableCode <FK>
ValidSize1
ValidSize2
[...]
ValidSize30
Sizes Table
-----------
SizesTableCode <PK>
SizeCode1
SizeCode2
[...]
SizeCode30
For now I am using a "template" query which I repeat for 30 times:
SELECT
Products.Code,
Sizes.SizesTableCode, -- I need this code because different codes can have same size codes
Sizes.Size_1
FROM Products
INNER JOIN Sizes
ON Sizes.SizesTableCode = Products.SizesTableCode
WHERE Sizes.Size_1 IS NOT NULL
AND Products.ValidSize_1 = 1
I am just putting this query inside a loop and I replace the "_1" with the loop index:
SET #counter = 1;
SET #max = 30;
SET #sql = '';
WHILE (#counter <= #max)
BEGIN
SET #sql = #sql + ('[...]'); -- Here goes my query with dynamic indexes
IF #counter < #max
SET #sql = #sql + ' UNION ';
SET #counter = #counter + 1;
END
INSERT INTO DestDb.ProductsSizes EXEC(#sql); -- Insert statement
GO
Is there a better, cleaner or faster method to do this? I am using SQL Server and I can only use SQL/TSQL.
You can prepare a dynamic query using the SYS.Syscolumns table to get all value in row
DECLARE #SqlStmt Varchar(MAX)
SET #SqlStmt=''
SELECT #SqlStmt = #SqlStmt + 'SELECT '''+ name +''' column , UNION ALL '
FROM SYS.Syscolumns WITH (READUNCOMMITTED)
WHERE Object_Id('dbo.Products')=Id AND ([Name] like 'SizeCode%' OR [Name] like 'ProductCode%')
IF REVERSE(#SqlStmt) LIKE REVERSE('UNION ALL ') + '%'
SET #SqlStmt = LEFT(#SqlStmt, LEN(#SqlStmt) - LEN('UNION ALL '))
print ( #SqlStmt )
Well, it seems that a "clean" (and much faster!) solution is the UNPIVOT function.
I found a very good example here:
http://pratchev.blogspot.it/2009/02/unpivoting-multiple-columns.html