I have a SSIS package that syncs data from source to destination Tables. However I am looking for a solution where whenever the package runs it automatically selects table from previous month i.e. (TableName-1). for example <102019> at the end of tablename.
I have used something as below in the SSIS package via VS Data tool's source but did not worked. I got error any further steps I have to do?
declare #year varchar(200) = 'TABLE1'
set #year = #year + CONCAT(DATEPART(mm,getdate())-1,DATEPART(yyyy,getdate()))
select #year as result
Chaitanya, Your query running properly and getting matched output that you want.
Are you looking for any other options for that??
You could also attack your problem from the information_schema side.
declare #LastMo date = dateadd(month,-1,getdate())
declare #LastMoText varchar(6) = right('0' + cast(month(#LastMo) as varchar(2)) , 2)
+ cast(Year(#LastMo) as varchar(4))
select table_name
from INFORMATION_SCHEMA.tables
where left(TABLE_NAME,6) = 'table1'
and right(table_name,6) = #LastMoText
This would let you know if the table exists. And therefore, your package would not fail if you put that into a foreach loop.
Exec SQL --> above SQL and stored result in ADO.Object variable.
Foreach ADO object
Store each loop in to variable called #table
Set #SQL = "SELECT * FROM " + #table
Add Dataflow and source from #SQL
Related
I have a procedure in which there are calculations being done and the final result is inserted into a permanent table. I want to remove the permanent table and I cannot use Temp table as well. So i want to use a dynamic table name, which is stored in a variable:
Current scenario:
Insert into xyz_table
Select col1,col2,sum(col3)
from BaseTable
(In reality, there are lot of columns and a lot of calculations)
What I want:
Select col1,col2,sum(col3) into #DynamicTableName
from BaseTable
where the name of the table would be dynamic in nature i.e.,
#DynamicTableName = 'xyz ' + cast(convert(date,getdate()) as nvarchar)+' '+convert(nvarchar(5),getdate(),108)
It will have date and time in its name every time the procedure is run.
I want to use this name in the "Select * into statement"
How can I achieve this?
i tried it with the some short code. But since my procedure has a lot of calculations and UNIONS , I cannot use that code for this. Any help would be appreciated.
declare #tablename nvarchar(30)= 'xyz ' + cast(convert(date,getdate()) as nvarchar)+' '+convert(nvarchar(5),getdate(),108)
declare #SQL_Statement nvarchar(100)
declare #SQL_Statement2 nvarchar(100)
declare #dropstatement nvarchar(100)
SET #SQL_Statement = N'SELECT * Into ' +'['+#tablename +'] '+'FROM '+ 'dimBranch'
print #SQL_Statement
EXECUTE sp_executesql #SQL_Statement
SET #SQL_Statement= N'select * from ' + '['+#tablename + '] '
print #SQL_Statement
EXECUTE sp_executesql #SQL_Statement
set #dropstatement = 'DROP TABLE' + '['+#tablename + '] '
PRINT #dropstatement
exec sp_executesql #dropstatement
Reason why I want this is because I use this procedure in ETL job as well as in SSRS report. And if someone runs the package and the SSRS report at the same time, the incorrect or weird data gets stored in the table. Therefore I need a dynamic name of the table with date and time.
You can't parameterize an identifier in SQL, only a value
--yes
select * from table where column = #value
--no
select * from #tablename where #columnname = #value
The only thin you can do to make these things dynamic is to build an sql string and execute it dynamically, but your code is already doing this with sp_executesql
More telling is your complaint at the bottom of your question, that if the procedure is invoked simultaneously it gives problems. Perhaps you should consider using local table variables for temporary data storage that the report is using rather than pushing data back into the db
DECLARE #temp TABLE(id INT, name varchar100);
INSERT INTO #temp SELECT personid, firstname FROM person;
-- work with temp data
select count(*) from #temp;
--when #temp goes out of scope it is lost,
--no other procedure invoked simultaneously can access this procedure'a #temp
Consider a local temp table, which is automatically session scoped without the need for dynamic SQL. For example:
SELECT *
INTO #YourTempTable
FROM dimBranch;
The local temp table will automatically be dropped when the proc completes so there is no need for an explict drop in the proc code.
I need to transfer data from a linked server to our main SQL server. The issue is, that the table name changes everyday.
I have looked around this site to find out if it is even possible to have a variable database name, which it is, but also to see if it is possible to have variables in a OPENQUERY, which it also is.
But i am struggling to combine those needs, so i have a variable table name in a OPENQUERY.
I need something like this:
Declare #LinkedServer as varchar(max) = 'LinkedServer'
Declare #TName as varchar(max) = 'TName'+substring(cast(cast(getdate() as date) as
varchar(50)),1,4)+substring(cast(cast(getdate() as date) as
varchar(50)),6,2)+substring(cast(cast(getdate() as date) as
varchar(50)),9,2)
SELECT * FROM OPENQUERY(#LinkedServer, 'SELECT * FROM dbo.#TName')
Is there any way i can make a variable table name in a OPENQUERY ?
Thank you for your help.
/Mikkel
I'd write a synonym which gets updated every day before you kick off your data extraction job. Then you don't need to be updating (potentially a tonne of) references.
CREATE SYNONYM LinkedTableA
FOR
ServerName.DBName.dbo.TName20170331
SELECT * FROM LinkedTableA
The answer i have, is this:
USE [DataBase]
GO
DROP SYNONYM [dbo].[eCallByCallStat]
GO
declare #tablename varchar(50)
set #tablename = 'Server1..dbo.eCallByCallStat'+substring(cast(cast(getdate()-1 as date) as varchar(50)),1,4)+substring(cast(cast(getdate()-1 as date) as varchar(50)),6,2)+substring(cast(cast(getdate()-1 as date) as varchar(50)),9,2)
declare #sql varchar(500)
set #sql = 'CREATE SYNONYM [dbo].[eCallByCallStat] FOR ' + #tablename
exec (#sql)
This will run everymorning updating the table name in the synonym, and then we will insert that data into a prober table so we have all the data.
I am attempting to create a stored procedure to insert records from a table in one DB to another DB, using fully qualified names for each table, however, since the tables can be in different DB's, the DB name needs to be able to change based on where a field is located. I attempted to set a variable and use the variable as the DB name, however, SQL does not allow this method.
Please supply me with any suggestions.
Below is a the query that I have to this point:
DECLARE #FromPractice varchar(5) = 'MCE'
DECLARE #ToPractice varchar(5) = 'CRRLL'
DECLARE #FromEnvironment varchar(5)
DECLARE #ToEnvironment varchar(5)
SET #FromEnvironment = ( select environment from practice where practice = #FromPractice)
SET #ToEnvironment = ( select environment from practice where practice = #ToPractice)
PRINT #FromEnvironment
Print #ToEnvironment
INSERT INTO [#ToEnvironment].dbo.Patinfo
(
---Fields
)
SELECT
---Values
FROM [#FromEnvironment].dbo.PatInfo
where practice = #FromPractice
and pat_num = 25970
Here is a rough example of how you could do this in dynamic sql. Please note that this is vulnerable to sql injection. You need to add some logic in your process prior to this to make sure that you are safe. The easiest way given the nature of this is to check sys.databases for your environment variables. You are likely going to need those variable to hold more than 5 characters unless your database names are awfully short.
declare #SQL nvarchar(max)
set #SQL = 'INSERT INTO [' + #ToEnvironment + '].dbo.Patinfo
(
--Fields
)
SELECT
--Values
FROM [' + #FromEnvironment + '].dbo.PatInfo
where practice = ''' + #FromPractice + '''
and pat_num = 25970'
--look before you execute
select #SQL
--When satisfied uncomment this
--exec sp_executesql #SQL
Query should output a certain list of Items, along with info like store information and manager info. Uses a Cursor to flip through list of various different levels of management, selects relevant information, then emails that person what the query returned for their district/region/store.
My issue is with the SSIS leg of the journey. Although the code acts like it runs, if I run Itemdata.dtsx separately (so as to see errors), it throws me the error:
"Warning: The external columns for component "Sources-ItemData"(1) are out of sync with the data source columns. The external column "RM_Email"(49) needs to be removed from the external columns. The external column "SM_Email"(46) needs to be removed from the external columns. The external column "DM_Email"(43) needs to be removed from the external columns."
This results in SQL Server Mngt Studio saying it ran, but the emails' contents are nothing but the table headers; no data, and the table headers don't change regardless of what I do.
I have eliminated these columns from any mention in my source code (posted below) and in the Table it uses. What am I missing?
BEGIN
SET NOCOUNT ON;
DECLARE #ProcedureName varchar(255)
DECLARE #ExportFolder varchar(255)
DECLARE #TempFolder varchar(255)
DECLARE #WarningLevel varchar(255) = 'log'
DECLARE #MsgDescription varchar(2000) = ''
DECLARE #RecordCount int = 0
DECLARE #ReportDate date = getdate()
DECLARE #Begdate date = convert(date,(dateadd(month,-1,getdate())))
DECLARE #Enddate date = convert(date,(dateadd(day,-1,getdate())))
DECLARE #Siteid int
DECLARE #Subject varchar(75) = ''
DECLARE #Body varchar(50) = ''
DECLARE #RMLastName varchar(25)
DECLARE #RMFirstName varchar(25)
DECLARE #RegionKey int
DECLARE #DistrictKey int
DECLARE #Email varchar(50)
BEGIN TRY
--Table used as data source for each pass
Truncate Table Example.dbo.itemdata
--Send reports to Regional Managers by building Cursor RMListCursor,
--then running SELECT statement against each name (using #RMLastName and #RMFirstName to discern),
--then emails results of SELECT statement to that Regional Manager.
--Goes through CursorList, then ends.
BEGIN
--Set cursor for RM Email; returns all regional managers.
DECLARE RMListCursor CURSOR FOR
SELECT distinct t.lastname, t.firstname, t.Email
FROM Example.[dbo].[tblUser] t
JOIN example.dbo.vStoreDistrictRegionActive vs
ON t.LastName = vs.RMLastName and t.FirstName = vs.RMFirstName
ORDER BY LastName
OPEN RMListCursor
FETCH NEXT FROM RMListCursor
INTO #RMLastName
, #RMFirstName
, #Email
WHILE ##FETCH_STATUS = 0--(#SetInt < 6)
BEGIN
Truncate table Example.dbo.itemdata
--Builds data, then inserts into Table built for this query. Note that there is no mention of DM_EMAIL, RM_EMAIL, or SM_EMail anywhere.
INSERT INTO Example.dbo.itemdata
SELECT InvoiceNumber,
shipFROMid,
ad.SiteId,
MfrCode,
PartCode,
UnitNetCore,
ad.QuantityShipped,
ShipDate,
--First/Last Name of this item's store's District Manager.
rtrim(substring((SELECT ISNULL(DMfirstName,'') FROM example.dbo.vSiteRegionDistrictActiveV2 dm WHERE ad.siteid = dm.SiteNumber),1,30)) + ' ' +
substring((SELECT ISNULL(DMLastName,'') FROM example.dbo.vSiteRegionDistrictActiveV2 dm WHERE ad.siteid = dm.SiteNumber),1,30) DM
--This is where DM_EMAIL, RM_EMAIL, and SM_EMail originally were before they were removed from both here and .ItemData.
FROM example.dbo.vInvoiceHeaderDetail_Adis ad
join example.dbo.Site ss on ad.SiteId=ss.siteid
join example.dbo.vStoreDistrictRegionActive vs on ad.SiteId = vs.SiteId
WHERE ad.siteid is not null and UnitNetCore>=250 and SUBSTRING(InvoiceNumber,2,1)='D' and QuantityShipped>0
and isactive=1 and isowned=1
and ShipDate between #Begdate and #Enddate
and vs.RMFirstName = #RMFirstName
and vs.RMLastname = #RMLastName
ORDER BY ad.SiteId,ShipFROMID,shipdate
-- Execute SSIS package which downloads table to d: for email.
set #RecordCount=##ROWCOUNT
--Quick check so that if the results were blank, don't bother sending a blank email.
IF #RecordCount<>0
BEGIN
set #Subject = 'Cores billed from PWI >= $250 ' + cast(CONVERT(date,GETDATE()) as varchar(12))
set #Body = 'Run date/time- ' + cast(GETDATE() as CHAR(20))
EXEC xp_cmdshell 'd:\"Program Files (x86)"\"Microsoft SQL Server"\100\DTS\Binn\DTexec.exe /f "D:\etl\bulk\ssis\Misc\ItemInfo.dtsx"'
EXEC msdb.dbo.sp_send_dbmail
#profile_name ='SQL Mail',
#recipients ='test', --#email
#subject = #Subject,
#body = #Body,
#body_format = 'HTML',
#File_attachments = 'D:\export\temp\ItemInfo.xls',
#attach_query_result_as_file =0,
#query_attachment_filename='\ItemInfo.xls',
#query_result_width = 500
END
--Brings CURSOR back up to next name on List, repeats process.
FETCH NEXT FROM RMListCursor
INTO #RMLastName
, #RMFirstName
, #Email
END
END
CLOSE RMListCursor
DEALLOCATE RMListCursor
END TRY
BEGIN CATCH
SET #WarningLevel = 'error'
SET #MsgDescription = 'SQL Err= [' + CAST(ERROR_MESSAGE() as varchar(200)) + ' (' + CAST(ERROR_LINE() as varchar) + ')]'
EXEC example.dbo.spAddSysMessage 'Store Aging', #WarningLevel , #ProcedureName , '', 'EOM Store Aging Report', #RecordCount
END CATCH
END
Recheck column assignment between source and destination and in-between component as well.
Give it a try by setting data flow component's Properties, ValidateExternalMetadata to False.
Please have a look of this as well.
It seems like you were changing the OLEDB Source or other type of source recently.
Thus, you must delete OLEDB Source and create a new one again. Then also delete the mapping of particular field, save, go back to mapping again and map it back. Than it should work fine.
Rather than deleting and recreating:
Copy the existing Data Source.
Paste it back into the Data Flow Task. The pasted Data Source has a refreshed schema. You've preserved any downstream mappings that are possible to keep. You now face the least possible maintenance.
Delete the connector from the old Data Source.
Substitute the connector from the new Data Source.
Maintain mappings, if necessary, in the Data Destination.
I had also same issue where all my column names were same but still getting the "The external columns for 'excelName' are out of synchronization with the data source columns. The column "mycolname" needs to be added to the external columns.
The column "F62" needs to be added to the external columns."
I have validated many time as well but no luck,even I matched my column header with destination table as well in excel formula and all seems to correct.So manually one by one column I checked and finally found that the file I have used for my connection initially had first letter in capital "Mycolname" , where as the file I was using (I used to overwrite the source file ) I just pasted data from some other file due which my header got changed. I changed the header again my input file and it worked.
Note : Be careful while using SSIS...really dangerous.
I have been researching this for a couple of days and feel like I am going around in circles. I have basic knowledge of SQL but there are many areas I do not understand.
I have a table that stores the names and fields of all the other tables in my database.
tblFields
===================================================
TableName FieldName BookmarkName
---------------------------------------------------
Customer FirstName CustomerFirstName
Customer LastName CustomerLastName
Customer DOB CustomerDOB
I want to write a SELECT statement like the following but i am unable to get it work:
SELECT (SELECT [FieldName] FROM [TableName]) FROM tblFields
Is this possible? The application I have developed requires this for user customization of reports.
If i understand what you are trying to do, i think this will help you. It is not pretty and it works for SQL Server 2005 and above, but maybe this is what you are looking for:
declare #tableName nvarchar(100)
declare #sqlQuery nvarchar(max)
declare #fields varchar(500)
set #tableName = 'YourTableName'
set #fields = ''
select #fields = #fields + QUOTENAME(t.fieldname) + ',' from (
select distinct fieldname from tblfields where tablename = #tableName)t
set #sqlQuery = 'select ' + left(#fields, LEN(#fields)-1) + ' from ' + QUOTENAME(#tableName)
execute sp_executesql #sqlQuery
Edit: As Martin suggested, i edited so that the columns and tablename are using QUOTENAME
If I understand correctly what you are trying to do, you are probably better off doing this as two separate queries from your program. One which gets the fields you want to select which you then use in your program to build up the second query which actually gets the data.
If it must be done entirely in SQL, then you will need to tell us what database you are using. If it is SQL Server, you might be able to user a cursor over the first query to build up the second query which you then execute with the sp_executesql stored procedure. But doing doing it outside of SQL would be recommended.