I have several tables, each with a different key.
For example: the key for the Customer could be 2 or more columns.
Input - dbo.customer:
Customer e_Date Value
------------------------
1000 2019-05-26 200
1000 2019-05-25 100
2000 2019-04-23 50
2000 2019-04-21 20
Output :
Customer e_date value
----------------------------
1000 2019-05-26 200
2000 2019-04-23 50
The max dates and the values for them was return for each customer (key).
I want to build a function or procedure in SQL where I will enter the name of table and the key and will return me the output. A return table function.
exec procedure get_Last_Row_By_Key (#Table_Name, #Key)
and it will show me the output.
In this example :
exec procedure get_Last_Row_By_Key ('dbo.customer', Customer)
I guess that when the #key will be multiple value I can do concat of the other columns to make them a one column key.
put below query in your function using row_number() window function
select customer,e_date,value from
(select *,row_number()over(partition by customer order by e_date desc) rn
from table ) a where a.rn=1
The function call would need to be something like this:
exec procedure get_Last_Row_By_Key ('dbo.customer', 'Customer', 'e_date')
To support multiple keys, I would use row_number(), even though that adds an extra column to the result set. So the dynamic SQL would look like:
declare #sql nvarchar(max);
set #sql = '
select t.*
from (select t.*,
row_number() over (partition by [key] order by [datecol]) as seqnum
from [table] t
) t
where seqnum = 1
';
set #sql = replace(#sql, '[table]', #table);
set #sql = replace(#sql, '[key]', #key);
set #sql = replace(#sql, '[datecol]', #datecol);
exec sp_executesql #sql;
Note: I am explicitly not using quotename() here, so the code will allow multiple columns for either the key order "datecol".
Also, as an exercise, this might be useful for learning about dynamic SQL and stored procedures. In general though, such attempts at "generic" processing are not as useful as they seem. People who know SQL know how to write the query to do what they want on a table; they do not know custom stored procedures that do the same thing.
Related
I have a table with Table name, Id, Column Name and Value
I need to run these into a Azure SQL Database as updates.
The tables are all different with different columns for the update.
The only way I could think of is RBAR, but there has to be an easier way to process this, either SQL or maybe even data factory?
Example of the data is:
Table Name Id Column Name Value
Table 1 1234 column1 1
Table 1 1234 column2 2022-01-02
Table 2 4321 column6 2144
Table 2 4321 column12 2022-01-02
The column name could be any column in the table with a update value.
As mentioned I have tried Row By Agonising Row but, as you would expect, WAY too painful as I have approx 161K rows that need processing.
You'll need to build an SQL string for each row and run that string as it's own command.
One thing I would do as part of this is pull the table/column information from Information_Schema.Columns. This will give you the same text you already have, but it ensures (in a safe way) that the data in your source table really is a valid object in your database, and not an SQL injection attempt. I would also use sp_executesql for the final query, to include the new value in a safe way.
So the main query would like looks this:
SELECT d.ID, d.Value,
'UPDATE ' + QUOTENAME(c.Table_Name) + ' SET ' + QUOTENAME(c.Column_Name) + ' = #new_value WHERE ID= #RowID' as SQL
FROM [MyData] d
INNER JOIN Information_Schema.Columns c ON c.[Table Name] = d.Table_Name
AND c.[Column Name] = d.Column_Name
And the loop would then take each row and do something like this:
-- #SQL, #ID, and #Value from the row
Execute sp_executeSql #SQL, '#new_value varchar(50), #id int', #new_value = #Value, #RowID = #ID
I skipped over the part where you loop through the rows because you have many options here, and which is best depends on your specific situation: do you want a cursor? Client code? Hard to know.
The trick here is we don't know the data type of the new value... and it certainly looks like you're mixing data types in there. You know that, for example, storing dates in varchar columns is really bad, right?
I have 203 tables in my SQL database, I want to print the latest records for each table. I know the query to get the latest row of one table at one time. How do I query for the latest row of each table in one go?
Here I do not know what are the names of your DB. So I assume that they can be indexed in the way I am about to show:
DECLARE #Counter INT
SET #Counter=1
WHILE ( #Counter <= 203)
BEGIN
EXEC('SELECT TOP(5) * FROM TABLE_'+#Counter+'ORDER BY Date DESC')
SET #Counter = #Counter + 1
END
Here make sure that you have defined everything using dynamic queries. In addition, I did not know in what format you need your pulled results to look.
Use SHOW TABLES and GROUP CONCAT and
SET #Expression = SELECT CONCAT('SELECT...
SELECT GROUP_CONCAT(...
PREPARE myquery FROM #Expression;
EXECUTE myquery;
Is it possible to execute a string in MySQL?
I have the following code in SQL Server that is generating a count of users by Code and by Date:
SELECT
code, CONVERT(DATE, created_at) Date, COUNT(account_id) UserCount
FROM
Code_Table
GROUP BY
code, CONVERT(DATE, created_at)
This just generates a count of users by each unique code and date combination. I'm attempting to use the Pivot function to get:
the list of unique codes in the first column
a column for each unique date, starting in the second column
a count of each user associated with the date-code combinations populating the table.
One issue I'm running into is: I would like to set this query to update automatically daily, which will add an additional column with the most recent date each day. The only pivots functions I've found require a declaration of the number of columns that are being created from rows.
I understand this would be much more easily done in Excel w/ a Pivot, but I don't currently have that option.
Any advice would be much appreciated!
You need a dynamic pivot to accomplish this. You're correct that you need an explicit column list -- so you'll query your table and actually generate SQL syntax as the result, and then execute that syntax with sp_executesql.
You can find that on Stack Overflow:
SQL Server dynamic PIVOT query?
A word of warning: This is usually not best practice. You won't be able to do any filtering or any date-related logic on this result set. Whatever front end reporting software you are using is probably where you want to implement the matrix/crosstab like behavior that you're getting from pivoting.
Try using this store procedure:
CREATE PROCEDURE GetData
-- Add the parameters for the stored procedure here
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
DECLARE #columns varchar(MAX) = '';
SELECT #columns = #columns + '[' + CONVERT(VARCHAR,ct.created_at) + '],'
FROM Code_Table ct
GROUP BY created_at
ORDER BY created_at ASC;
SET #columns = SUBSTRING(#columns,1,LEN(#columns) - 1);
DECLARE #query varchar(MAX)= 'SELECT
*
FROM
(SELECT code,account_id,created_at from Code_Table) AS result
PIVOT
(
COUNT(account_id) FOR created_at IN ('+ #columns +')
) p'
;
EXEC (#query)
END
I built the column's header dynamically depending of existing values of dates.
It looks like #temptables created using dynamic SQL via the EXECUTE string method have a different scope and can't be referenced by "fixed" SQLs in the same stored procedure.
However, I can reference a temp table created by a dynamic SQL statement in a subsequence dynamic SQL but it seems that a stored procedure does not return a query result to a calling client unless the SQL is fixed.
A simple 2 table scenario:
I have 2 tables. Let's call them Orders and Items. Order has a Primary key of OrderId and Items has a Primary Key of ItemId. Items.OrderId is the foreign key to identify the parent Order. An Order can have 1 to n Items.
I want to be able to provide a very flexible "query builder" type interface to the user to allow the user to select what Items he want to see. The filter criteria can be based on fields from the Items table and/or from the parent Order table. If an Item meets the filter condition including and condition on the parent Order if one exists, the Item should be return in the query as well as the parent Order.
Usually, I suppose, most people would construct a join between the Item table and the parent Order tables. I would like to perform 2 separate queries instead. One to return all of the qualifying Items and the other to return all of the distinct parent Orders. The reason is two fold and you may or may not agree.
The first reason is that I need to query all of the columns in the parent Order table and if I did a single query to join the Orders table to the Items table, I would be repoeating the Order information multiple times. Since there are typically a large number of items per Order, I'd like to avoid this because it would result in much more data being transfered to a fat client. Instead, as mentioned, I would like to return the two tables individually in a dataset and use the two tables within to populate a custom Order and child Items client objects. (I don't know enough about LINQ or Entity Framework yet. I build my objects by hand). The second reason I would like to return two tables instead of one is because I already have another procedure that returns all of the Items for a given OrderId along with the parent Order and I would like to use the same 2-table approach so that I could reuse the client code to populate my custom Order and Client objects from the 2 datatables returned.
What I was hoping to do was this:
Construct a dynamic SQL string on the Client which joins the orders table to the Items table and filters appropriate on each table as specified by the custom filter created on the Winform fat-client app. The SQL build on the client would have looked something like this:
TempSQL = "
INSERT INTO #ItemsToQuery
OrderId, ItemsId
FROM
Orders, Items
WHERE
Orders.OrderID = Items.OrderId AND
/* Some unpredictable Order filters go here */
AND
/* Some unpredictable Items filters go here */
"
Then, I would call a stored procedure,
CREATE PROCEDURE GetItemsAndOrders(#tempSql as text)
Execute (#tempSQL) --to create the #ItemsToQuery table
SELECT * FROM Items WHERE Items.ItemId IN (SELECT ItemId FROM #ItemsToQuery)
SELECT * FROM Orders WHERE Orders.OrderId IN (SELECT DISTINCT OrderId FROM #ItemsToQuery)
The problem with this approach is that #ItemsToQuery table, since it was created by dynamic SQL, is inaccessible from the following 2 static SQLs and if I change the static SQLs to dynamic, no results are passed back to the fat client.
3 around come to mind but I'm look for a better one:
1) The first SQL could be performed by executing the dynamically constructed SQL from the client. The results could then be passed as a table to a modified version of the above stored procedure. I am familiar with passing table data as XML. If I did this, the stored proc could then insert the data into a temporary table using a static SQL that, because it was created by dynamic SQL, could then be queried without issue. (I could also investigate into passing the new Table type param instead of XML.) However, I would like to avoid passing up potentially large lists to a stored procedure.
2) I could perform all the queries from the client.
The first would be something like this:
SELECT Items.* FROM Orders, Items WHERE Order.OrderId = Items.OrderId AND (dynamic filter)
SELECT Orders.* FROM Orders, Items WHERE Order.OrderId = Items.OrderId AND (dynamic filter)
This still provides me with the ability to reuse my client sided object-population code because the Orders and Items continue to be returned in two different tables.
I have a feeling to, that I might have some options using a Table data type within my stored proc, but that is also new to me and I would appreciate a little bit of spoon feeding on that one.
If you even scanned this far in what I wrote, I am surprised, but if so, I woul dappreciate any of your thoughts on how to accomplish this best.
You first need to create your table first then it will be available in the dynamic SQL.
This works:
CREATE TABLE #temp3 (id INT)
EXEC ('insert #temp3 values(1)')
SELECT *
FROM #temp3
This will not work:
EXEC (
'create table #temp2 (id int)
insert #temp2 values(1)'
)
SELECT *
FROM #temp2
In other words:
Create temp table
Execute proc
Select from temp table
Here is complete example:
CREATE PROC prTest2 #var VARCHAR(100)
AS
EXEC (#var)
GO
CREATE TABLE #temp (id INT)
EXEC prTest2 'insert #temp values(1)'
SELECT *
FROM #temp
1st Method - Enclose multiple statements in the same Dynamic SQL Call:
DECLARE #DynamicQuery NVARCHAR(MAX)
SET #DynamicQuery = 'Select * into #temp from (select * from tablename) alias
select * from #temp
drop table #temp'
EXEC sp_executesql #DynamicQuery
2nd Method - Use Global Temp Table:
(Careful, you need to take extra care of global variable.)
IF OBJECT_ID('tempdb..##temp2') IS NULL
BEGIN
EXEC (
'create table ##temp2 (id int)
insert ##temp2 values(1)'
)
SELECT *
FROM ##temp2
END
Don't forget to delete ##temp2 object manually once your done with it:
IF (OBJECT_ID('tempdb..##temp2') IS NOT NULL)
BEGIN
DROP Table ##temp2
END
Note: Don't use this method 2 if you don't know the full structure on database.
I had the same issue that #Muflix mentioned. When you don't know the columns being returned, or they are being generated dynamically, what I've done is create a global table with a unique id, then delete it when I'm done with it, this looks something like what's shown below:
DECLARE #DynamicSQL NVARCHAR(MAX)
DECLARE #DynamicTable VARCHAR(255) = 'DynamicTempTable_' + CONVERT(VARCHAR(36), NEWID())
DECLARE #DynamicColumns NVARCHAR(MAX)
--Get "#DynamicColumns", example: SET #DynamicColumns = '[Column1], [Column2]'
SET #DynamicSQL = 'SELECT ' + #DynamicColumns + ' INTO [##' + #DynamicTable + ']' +
' FROM [dbo].[TableXYZ]'
EXEC sp_executesql #DynamicSQL
SET #DynamicSQL = 'IF OBJECT_ID(''tempdb..##' + #DynamicTable + ''' , ''U'') IS NOT NULL ' +
' BEGIN DROP TABLE [##' + #DynamicTable + '] END'
EXEC sp_executesql #DynamicSQL
Certainly not the best solution, but this seems to work for me.
I would strongly suggest you have a read through http://www.sommarskog.se/arrays-in-sql-2005.html
Personally I like the approach of passing a comma delimited text list, then parsing it with text to table function and joining to it. The temp table approach can work if you create it first in the connection. But it feel a bit messier.
Result sets from dynamic SQL are returned to the client. I have done this quite a lot.
You're right about issues with sharing data through temp tables and variables and things like that between the SQL and the dynamic SQL it generates.
I think in trying to get your temp table working, you have probably got some things confused, because you can definitely get data from a SP which executes dynamic SQL:
USE SandBox
GO
CREATE PROCEDURE usp_DynTest(#table_type AS VARCHAR(255))
AS
BEGIN
DECLARE #sql AS VARCHAR(MAX) = 'SELECT * FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = ''' + #table_type + ''''
EXEC (#sql)
END
GO
EXEC usp_DynTest 'BASE TABLE'
GO
EXEC usp_DynTest 'VIEW'
GO
DROP PROCEDURE usp_DynTest
GO
Also:
USE SandBox
GO
CREATE PROCEDURE usp_DynTest(#table_type AS VARCHAR(255))
AS
BEGIN
DECLARE #sql AS VARCHAR(MAX) = 'SELECT * INTO #temp FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = ''' + #table_type + '''; SELECT * FROM #temp;'
EXEC (#sql)
END
GO
EXEC usp_DynTest 'BASE TABLE'
GO
EXEC usp_DynTest 'VIEW'
GO
DROP PROCEDURE usp_DynTest
GO
How can I get the number of columns returned by an SQL query using SQL Server?
For example, if I have a query like following:
SELECT *
FROM A1, A2
It should return the total number of columns in table A1 + total number of columns in table A2. But the query might be more complicated.
Here is one method:
select top 0
into _MYLOCALTEMPTABLE
from (your query here) t
select count(*)
from Information_Schema.Columns c
where table_name = '_MYLOCALTEMPTABLE'
You can do something similar by creating a view.
You didn't specify your SQL Server version but I'm assuming it's not 2012. However, future readers of this question might be on 2012+ so I'm posting this answer for them.
SQL Server 2012 provides a set of procedures to provide more meta-data about queries and parameters. In this case, the stored procedure sp_describe_first_result_set will provide a handy tabular form.
There is also a DMO function, sys.dm_exec_describe_first_result_set, to provide similar content which is what you'd want to use in your example
DECLARE
-- Your query goes here
#query nvarchar(4000) = N'SELECT * FROM mdm.tblStgBatch AS TSB';
-- Tabular results
EXECUTE sys.sp_describe_first_result_set #tsql = #query;
-- Simple column count
SELECT
COUNT(1) AS column_count
FROM
sys.dm_exec_describe_first_result_set(#query, NULL, 0);
The new metadata discovery options are replacing FMTONLY which is how one would solve this problem prior to 2012. My TSQL chops are apparently not strong enough to do anything useful with it and instead I'd have to bail out to a .NET language to work with the output of FMTONLY.
SET FMTONLY ON;
SELECT *
FROM A1, A2;
SET FMTONLY OFF;
Try this;
--Insert into a temp table (this could be any query)
SELECT *
INTO #temp
FROM [yourTable]
--Select from temp table
SELECT * FROM #temp
--List of columns
SELECT COUNT(name) NumOfColumns FROM tempdb.sys.columns WHERE object_id =
object_id('tempdb..#temp');
--drop temp table
DROP TABLE #temp
Ugly I know:
SELECT COUNT(*) +
(
SELECT COUNT(*)
FROM information_schema.columns
WHERE table_name = 'A1'
)
FROM information_schema.columns
WHERE table_name = 'A2'