SQL Server prompt user for input and pass to variable - sql

I am trying to write code for my client that they can use in the future without my help. They will install SQLServer Express on one of their desktops (like I have been using) so that the designated person can run the pre-written queries on locally stored data for specific information they need on a daily basis. I don't want to get into why we are using SQL Server Express and managing our data in this manner, but trust that we've looked into alternatives and this is the most feasible option.
I would like to make executing the import query as simple as possible for them, as they are not familiar with SQL or coding in general. At a high level, I have about 15 tab-delimited files automatically loaded into a folder on my local drive every day with the filename consistent except for the pre-fix, which is the date that the file was loaded. For example, 20180912-xxx, 20180912-yyy, etc. for all of the files loaded on Sept 12.
I run a bulk import that creates a Database for Sept 12 ("sep12"), and then loads the tables into the database; then I use the #date variable in the import statements to define the file location. For example:
CREATE DATABASE aug29
USE aug29
DECLARE #date VARCHAR(15)
SET #date = '20180829'
#import = 'BULK INSERT dbo.Table FROM ''\\Drive\Documents\' + #Date + '-xxx.txt''
WITH (FIRSTROW = 2, FIELDTERMINATOR = ''\t'', ROWTERMINATOR = ''\n'')'
EXEC(#import)
As you can see, there is manual edit needed for the 1st, 2nd, and 4th lines. I've simplified the code to only require 1 manual edit, by defining 1 variable as the date (ex. #dateinput = '9/12/2018'), and then the other variables define themselves accordingly (ex. #DBName = 'sep12', #FilePrefix = '20180912', etc.).
My question is this:
Can I prompt the user to enter this date before running the code? Something like
Click Execute > message prompt pops up > User enters a date > click OK > code runs with the inputted value stored as the variable

Create a Stored Procedure?
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author:
-- Create date:
-- Description:
-- =============================================
CREATE PROCEDURE [dbo].[ImportData]
#date varchar(15)
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
-- Insert statements for procedure here
DECLARE #query AS varchar(MAX)
SET #query = 'CREATE DATABASE aug29
USE aug29
BULK INSERT dbo.Table FROM ''\\Drive\Documents\''' + #date + '''-xxx.txt''
WITH (FIRSTROW = 2, FIELDTERMINATOR = ''\t'', ROWTERMINATOR = ''\n'')'
EXEC(#query)
END
Then on SSMS open your database > Programmability > Stored Procedures > Right Click on the newly created Stored Procedure (ImportData or whichever you name it) > Execute Stored Procedure.
Users can then input the value for the #date parameter.

You could make your script completely independent of user input by using some of SQL Server's built in functions like I am doing below:
--Variable to use for dynamic sql
DECLARE #sqlStatement varchar(MAX) = '';
--Returns the month as its full name like June, or July
DECLARE #fullMonthValue varchar(100) = DATENAME(month, DATEADD(month, MONTH(GETDATE()) -1, CAST('2008-01-01' AS datetime)));
--Get the database name how you gave in your example, for example today is August 30th, this will result in aug30
DECLARE #databaseName varchar(100) = LOWER(SUBSTRING(#fullMonthName, 1, 3)) + CAST(DAY(GETDATE()) AS varchar(3));
--Now get the current date as string for use in your bulk insert
DECLARE #today = CAST(GETDATE() AS Date);
--cast the current date to varchar (string) and remove the hyphens
DECLARE #stringDate = REPLACE(CAST(#today AS varchar(100)), '-', ''); --Need to remove the hyphens
--Set the sql statement for creating the database
SET #sqlStatment = 'Create DataBase ' + #databaseName;
--Execute the sqlStatement to create the database
EXEC(#sqlStatement);
--At this point #stringDate is already the format you want for your example variable of #date
--Just put your USE statement into your dynamic sql string
#import = 'USE ' + #databaseName + 'BULK INSERT dbo.Table FROM ''\\Drive\Documents\'' + #stringDate + '-xxx.txt''
WITH (FIRSTROW = 2, FIELDTERMINATOR = ''\t'', ROWTERMINATOR = ''\n'')';
EXEC(#import);

Related

Dynamically select table names from VS Data tool

I have a SSIS package that syncs data from source to destination Tables. However I am looking for a solution where whenever the package runs it automatically selects table from previous month i.e. (TableName-1). for example <102019> at the end of tablename.
I have used something as below in the SSIS package via VS Data tool's source but did not worked. I got error any further steps I have to do?
declare #year varchar(200) = 'TABLE1'
set #year = #year + CONCAT(DATEPART(mm,getdate())-1,DATEPART(yyyy,getdate()))
select #year as result
Chaitanya, Your query running properly and getting matched output that you want.
Are you looking for any other options for that??
You could also attack your problem from the information_schema side.
declare #LastMo date = dateadd(month,-1,getdate())
declare #LastMoText varchar(6) = right('0' + cast(month(#LastMo) as varchar(2)) , 2)
+ cast(Year(#LastMo) as varchar(4))
select table_name
from INFORMATION_SCHEMA.tables
where left(TABLE_NAME,6) = 'table1'
and right(table_name,6) = #LastMoText
This would let you know if the table exists. And therefore, your package would not fail if you put that into a foreach loop.
Exec SQL --> above SQL and stored result in ADO.Object variable.
Foreach ADO object
Store each loop in to variable called #table
Set #SQL = "SELECT * FROM " + #table
Add Dataflow and source from #SQL

Problem executing a stored procedure inside another stored procedure from a linked server

I'm having a problem and I don't know how to solve it, I have searched the web and found good advice but I can't work it out.
This is the problem: I have a SQL Server instance running on my PC, and I linked one of the main servers SRVOLD\SQLDESA to it. I want to execute main server's stored procedures from my PC's SQL Server instance and insert the results into a new table. I found the perfect way to do it using the following:
SELECT *
INTO Bank
FROM OPENQUERY([SRVOLD\SQLDESA],
'EXEC Bank_Database.Bank.usp_GetTDcodes 1, 5')
GO
There is important information about this server, it's SQL Server version is 2008. Keep this in mind for later.
Ok so I managed to executed this Stored Procedure but I found out something, turns out that inside this Stored Procedure there's an execution of another stored procedure, check this out:
1st stored procedure:
CREATE PROCEDURE Bank.usp_GetTDcodes
(#code TINYINT = NULL, #qty TINYINT = NULL)
WITH ENCRYPTION
AS
DECLARE ##msg VARCHAR(100)
DECLARE ##OK INT
DECLARE ##today CHAR(30)
SELECT ##today = CONVERT(VARCHAR(30), GETDATE(), 112) + ' ' +
CONVERT(VARCHAR(30), GETDATE(), 8)
SELECT bnk_code, bnk_descr
FROM CODBNK
WHERE bnk_code < 50
EXECUTE ##OK = Bank.usp_WriteEvent #qty, #code, ##today, 500
IF ##OK <> 0
RETURN ##OK
RETURN 0
GO
Now let's look inside the 2nd stored procedure:
CREATE PROCEDURE Bank.usp_WriteEvent
(#code TINYINT = NULL,
#qty TINYINT = NULL,
#date DATETIME = NULL,
#number SMALLINT = NULL,
#ideve INT = 0 OUTPUT)
WITH ENCRYPTION
AS
DECLARE ##sdate VARCHAR(30)
DECLARE ##ret SMALLINT
INSERT INTO Event (eve_code, eve_qty, eve_date, eve_number)
VALUES (#code, #qty, #date, #number)
SET ##ret = ##error
IF ##ret = 0
BEGIN
SELECT #ideve = ##IDENTITY
SELECT ##sdate = CONVERT(VARCHAR(30), #date, 112) + ' ' +
VARCHAR(30), #date, 8)
END
ELSE
RETURN ##ret
GO
When I executed the 1st stored procedure, I was able to insert it's result into a new table, but I was hoping to find a new row inserted in the table Event, because that is the expected result when executing 2nd stored procedure.
So I started to search online and managed to achieve this by doing the following:
SELECT *
INTO Bank
FROM OPENQUERY([SRVTEST\SQLDESA],
'SET FMTONLY OFF;SET NOCOUNT ON;EXEC Bank_Database.Bank.usp_GetTDcodes 1, 5')
GO
So, the SET FMTONLY OFF;SET NOCOUNT ON worked and I was happy. But something happened...
I needed to execute the same stored procedure, but this time adding a new linked server SRVNEW\SQLDESA. This server's version is 2012, so the new solution didn't work. I kept trying and trying different ways, there's just one way to make it work and is the following:
EXEC [SRVNEW\SQLDESA].[Bank_Database].Bank.usp_GetTDcodes 1,5
But it doesn't work for me because I need the 1st stored procedure result into a new table. And I don't know its schema that's why SELECT INTO works best for me.
I don't know what else I can do, maybe is the OPENQUERY that doesn't work? Do I need to change something else?
PD: I also tried using OPENROWSET didn't work either.
Thanks in advance, and have a nice day!
Peace!
Some references: http://www.sommarskog.se/share_data.html#OPENQUERY

bulk insert skip last row

I am doing a bulk insert that I have to skip the last row. Otherwise , I got an error saying "Bulk Insert: Unexpected end-of-file (EOF) encountered in data file."
If I set ROWTERMINATOR='\r\n', then I got 0 rows imported.
I wonder if there is any code that can help me skip the lastrow of the txt file? (The last row is dynamic.) My company currently doesn't have SSIS installed.
My code for bulk insert is
Declare #SQL1 varchar(150), #path varchar(100),
#pathtable varchar(100), #date datetime
set #date = getdate()
-- set path for files
set #path= 'C:\imp\'
set #pathtable = #path + 'importfile.txt'
delete from IDX
-- set sql
set #SQL1 = "BULK INSERT dbo.table FROM '" + #pathtable
+ "' WITH (FIRSTROW = 2, MAXERRORS = 0)"
-- Bulk insert
exec(#sql1)
The issue is that the last row contains a row count from the export process. If you're able to modify the export process, make sure you use the SQL command:
SET NOCOUNT ON;
If you're using a GUI to export the data there should be a place to modify the T-SQL used or an option to set nocount on.
This will prevent the last row from writing out to your file.
If you cannot modify the export process... You can get crazy and right either a console application to read the data and remove the last line or a CLR that does basically that very task.. Open the file, remove the last line, save the file then call your stored procedure above to bulk insert your data.
You need to use single quotes ' multiple times, You have used double quotes " which are treated as identifiers in sql server.
Your query should look like this...
Declare #SQL1 varchar(150)
, #path varchar(100)
, #pathtable varchar(100)
, #date datetime
SET #date = getdate();
SET #path= 'C:\imp\'
SET #pathtable = #path + 'importfile.txt'
SET #SQL1 = 'BULK INSERT dbo.table
FROM ''' + #pathtable + '''
WITH (
FIRSTROW = 2
, MAXERRORS = 0
)';
Now if you print this SQL statement it would look like this...
PRINT #SQL1
RESULT:
BULK INSERT dbo.table
FROM 'C:\imp\importfile.txt'
WITH (
FIRSTROW = 2
, MAXERRORS = 0
)

SSIS Error: External Column for Source out of sync with Data Source columns; How do I remove External Columns?

Query should output a certain list of Items, along with info like store information and manager info. Uses a Cursor to flip through list of various different levels of management, selects relevant information, then emails that person what the query returned for their district/region/store.
My issue is with the SSIS leg of the journey. Although the code acts like it runs, if I run Itemdata.dtsx separately (so as to see errors), it throws me the error:
"Warning: The external columns for component "Sources-ItemData"(1) are out of sync with the data source columns. The external column "RM_Email"(49) needs to be removed from the external columns. The external column "SM_Email"(46) needs to be removed from the external columns. The external column "DM_Email"(43) needs to be removed from the external columns."
This results in SQL Server Mngt Studio saying it ran, but the emails' contents are nothing but the table headers; no data, and the table headers don't change regardless of what I do.
I have eliminated these columns from any mention in my source code (posted below) and in the Table it uses. What am I missing?
BEGIN
SET NOCOUNT ON;
DECLARE #ProcedureName varchar(255)
DECLARE #ExportFolder varchar(255)
DECLARE #TempFolder varchar(255)
DECLARE #WarningLevel varchar(255) = 'log'
DECLARE #MsgDescription varchar(2000) = ''
DECLARE #RecordCount int = 0
DECLARE #ReportDate date = getdate()
DECLARE #Begdate date = convert(date,(dateadd(month,-1,getdate())))
DECLARE #Enddate date = convert(date,(dateadd(day,-1,getdate())))
DECLARE #Siteid int
DECLARE #Subject varchar(75) = ''
DECLARE #Body varchar(50) = ''
DECLARE #RMLastName varchar(25)
DECLARE #RMFirstName varchar(25)
DECLARE #RegionKey int
DECLARE #DistrictKey int
DECLARE #Email varchar(50)
BEGIN TRY
--Table used as data source for each pass
Truncate Table Example.dbo.itemdata
--Send reports to Regional Managers by building Cursor RMListCursor,
--then running SELECT statement against each name (using #RMLastName and #RMFirstName to discern),
--then emails results of SELECT statement to that Regional Manager.
--Goes through CursorList, then ends.
BEGIN
--Set cursor for RM Email; returns all regional managers.
DECLARE RMListCursor CURSOR FOR
SELECT distinct t.lastname, t.firstname, t.Email
FROM Example.[dbo].[tblUser] t
JOIN example.dbo.vStoreDistrictRegionActive vs
ON t.LastName = vs.RMLastName and t.FirstName = vs.RMFirstName
ORDER BY LastName
OPEN RMListCursor
FETCH NEXT FROM RMListCursor
INTO #RMLastName
, #RMFirstName
, #Email
WHILE ##FETCH_STATUS = 0--(#SetInt < 6)
BEGIN
Truncate table Example.dbo.itemdata
--Builds data, then inserts into Table built for this query. Note that there is no mention of DM_EMAIL, RM_EMAIL, or SM_EMail anywhere.
INSERT INTO Example.dbo.itemdata
SELECT InvoiceNumber,
shipFROMid,
ad.SiteId,
MfrCode,
PartCode,
UnitNetCore,
ad.QuantityShipped,
ShipDate,
--First/Last Name of this item's store's District Manager.
rtrim(substring((SELECT ISNULL(DMfirstName,'') FROM example.dbo.vSiteRegionDistrictActiveV2 dm WHERE ad.siteid = dm.SiteNumber),1,30)) + ' ' +
substring((SELECT ISNULL(DMLastName,'') FROM example.dbo.vSiteRegionDistrictActiveV2 dm WHERE ad.siteid = dm.SiteNumber),1,30) DM
--This is where DM_EMAIL, RM_EMAIL, and SM_EMail originally were before they were removed from both here and .ItemData.
FROM example.dbo.vInvoiceHeaderDetail_Adis ad
join example.dbo.Site ss on ad.SiteId=ss.siteid
join example.dbo.vStoreDistrictRegionActive vs on ad.SiteId = vs.SiteId
WHERE ad.siteid is not null and UnitNetCore>=250 and SUBSTRING(InvoiceNumber,2,1)='D' and QuantityShipped>0
and isactive=1 and isowned=1
and ShipDate between #Begdate and #Enddate
and vs.RMFirstName = #RMFirstName
and vs.RMLastname = #RMLastName
ORDER BY ad.SiteId,ShipFROMID,shipdate
-- Execute SSIS package which downloads table to d: for email.
set #RecordCount=##ROWCOUNT
--Quick check so that if the results were blank, don't bother sending a blank email.
IF #RecordCount<>0
BEGIN
set #Subject = 'Cores billed from PWI >= $250 ' + cast(CONVERT(date,GETDATE()) as varchar(12))
set #Body = 'Run date/time- ' + cast(GETDATE() as CHAR(20))
EXEC xp_cmdshell 'd:\"Program Files (x86)"\"Microsoft SQL Server"\100\DTS\Binn\DTexec.exe /f "D:\etl\bulk\ssis\Misc\ItemInfo.dtsx"'
EXEC msdb.dbo.sp_send_dbmail
#profile_name ='SQL Mail',
#recipients ='test', --#email
#subject = #Subject,
#body = #Body,
#body_format = 'HTML',
#File_attachments = 'D:\export\temp\ItemInfo.xls',
#attach_query_result_as_file =0,
#query_attachment_filename='\ItemInfo.xls',
#query_result_width = 500
END
--Brings CURSOR back up to next name on List, repeats process.
FETCH NEXT FROM RMListCursor
INTO #RMLastName
, #RMFirstName
, #Email
END
END
CLOSE RMListCursor
DEALLOCATE RMListCursor
END TRY
BEGIN CATCH
SET #WarningLevel = 'error'
SET #MsgDescription = 'SQL Err= [' + CAST(ERROR_MESSAGE() as varchar(200)) + ' (' + CAST(ERROR_LINE() as varchar) + ')]'
EXEC example.dbo.spAddSysMessage 'Store Aging', #WarningLevel , #ProcedureName , '', 'EOM Store Aging Report', #RecordCount
END CATCH
END
Recheck column assignment between source and destination and in-between component as well.
Give it a try by setting data flow component's Properties, ValidateExternalMetadata to False.
Please have a look of this as well.
It seems like you were changing the OLEDB Source or other type of source recently.
Thus, you must delete OLEDB Source and create a new one again. Then also delete the mapping of particular field, save, go back to mapping again and map it back. Than it should work fine.
Rather than deleting and recreating:
Copy the existing Data Source.
Paste it back into the Data Flow Task. The pasted Data Source has a refreshed schema. You've preserved any downstream mappings that are possible to keep. You now face the least possible maintenance.
Delete the connector from the old Data Source.
Substitute the connector from the new Data Source.
Maintain mappings, if necessary, in the Data Destination.
I had also same issue where all my column names were same but still getting the "The external columns for 'excelName' are out of synchronization with the data source columns. The column "mycolname" needs to be added to the external columns.
The column "F62" needs to be added to the external columns."
I have validated many time as well but no luck,even I matched my column header with destination table as well in excel formula and all seems to correct.So manually one by one column I checked and finally found that the file I have used for my connection initially had first letter in capital "Mycolname" , where as the file I was using (I used to overwrite the source file ) I just pasted data from some other file due which my header got changed. I changed the header again my input file and it worked.
Note : Be careful while using SSIS...really dangerous.

Invalid column name: SQL Server 2008 R2

I have a stored procedure I use to extract binary data from SQL tables to create physical files. I have used it a couple of times on tables with the same structure restored from SQL backups, and it works OK.
Today I wanted to extract some binary data from a new table inside a restored backup. I opened the stored procedure I have been using and set about altering the code. Once I was happy with the changes I tried to execute the 'ALTER' statement. Unfortunetly, both of the column names I have used are 'invalid' despite existing on the the 'Document' table.
I have read a number of other threads regarding 'invalid column name' errors, but the majority of these seem to be typing errors. I've checked my column names numurous times (intelli sense even lets me put in 'Document.Document_ID' and 'Document.Document_Filename' but they still fail).
Any ideas where I am going wrong?
Source:
USE [Example Live]
GO
/****** Object: StoredProcedure [dbo].[FileExport] Script Date: 10/18/2012 11:42:14 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author: Chris Murray
-- Create date: 18/10/2012
-- Description: Exports binary file data
-- =============================================
ALTER PROCEDURE [dbo].[FileExport]
-- Add the parameters for the stored procedure here
#OutputFilePath VARCHAR(500) = 'C:\Conv\Example\In\Afiles\'
AS
BEGIN
DECLARE #totrow int
DECLARE #currow int
DECLARE #result int
DECLARE #nsql nvarchar(4000)
DECLARE #sqlStatements table (ID int IDENTITY(1, 1), SqlStatement varchar(max))
INSERT
INTO #sqlStatements
SELECT 'BCP "SELECT Document_Data FROM [Example Live].[dbo].[Document] WHERE Document_ID = '''
+ CAST(Document_ID AS VARCHAR(500)) + '''" queryout ' + #OutputFilePath
+ CAST(Document_Filename AS VARCHAR(500)) + ' -S localhost\SQLEXPRESS2008 -T -f C:\Conv\Example\In\AFiles\Images.fmt'
FROM dbo.Photograph
SET #totrow = ##ROWCOUNT
SET #currow = 1
WHILE #totrow > 0 and #currow <= #totrow
BEGIN
SELECT #nsql = SqlStatement
FROM #sqlStatements
WHERE ID = #currow
EXEC #result = xp_cmdshell #nsql
SET #currow = #currow + 1
END
END
Thanks for the ideas everyone.
The problem was of my own making, and a rookie mistake. I neglected to point the SELECT statement at the correct table..