I have created s Stored Procedure in SQL which imports data from a flat file, updates the data and imports the updated data within a table.
After some research, I found that the only way to export this (64bit) table would be to create a SSIS package and use a SQL Job to execute the package.
I have done all of this and managed to get the table data exported but the problem is that it does not clear the data before the import. I have then created the following
When dropping the excel table, I have the following SQL Statement : DROP TABLE [Sheet1$]
When creating the table, I have the following SQL Statement :
CREATE TABLE 'Sheet1$'
(
BRANCH NVARCHAR(10) ,
SRCBRANCH NVARCHAR(10) ,
DEPARTMENT NVARCHAR(10) ,
GLCODE NVARCHAR(10) ,
DOCDATE NVARCHAR(10) ,
VALUE NVARCHAR(50) ,
ITEMREFERENCE NVARCHAR(100) ,
MISCREFERENCE NVARCHAR(100) ,
SUFFIX NVARCHAR(10) ,
NARRATIVE [NVARCHAR(100)
)
GO
After the table has been dropped, it clears all the data together with the header and then fails on the second SQL Task (Create Excel Table) with the following error message:
[Execute SQL Task] Error: Executing the query "CREATE TABLE 'Sheet1$' (
BRANCH NVARCHAR(10) ,
S..." failed with the following error: "Syntax error in CREATE TABLE statement.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
After fail, I cant execute the package due to a validation error. This is because my headers in the Excel sheet has been deleted.
Could someone please point me in the right direction. I have exhausted all options.
Regards
CREATE TABLE 'Sheet1$'
(
BRANCH NVARCHAR(10) ,
SRCBRANCH NVARCHAR(10) ,
DEPARTMENT NVARCHAR(10) ,
GLCODE NVARCHAR(10) ,
DOCDATE NVARCHAR(10) ,
VALUE NVARCHAR(50) ,
ITEMREFERENCE NVARCHAR(100) ,
MISCREFERENCE NVARCHAR(100) ,
SUFFIX NVARCHAR(10) ,
NARRATIVE **[**NVARCHAR(100)
)
GO
***This is your syntax error. This was asked quite a while ago so I would think you have rectified this by now but I wanted to post for others to know what the issue was.
Related
I have a SSIS job with multiple steps, each steps is from different package or project.
When I execute the job , I write log in db with a guid, but each guid is created in its package or project.
I need to update the same value for all the steps , kind of link all the steps together with one value/ID.
Any suggestions how to do this?
SQL Agent jobs do not allow for values to be passed in to job steps.
But, you can rethink how your current invocation of SSIS packages works to meet your goals.
What if you added a precursor step to your SQL Agent job? Type of SQL Task and use that to generate the GUID you'd like for your packages to share. Store it into either a 1 row table or create a key/value style historical table.
CREATE TABLE dbo.CorrelateAgentToSSIS
(
jobid uniqueidentifier NOT NULL
, runid uniqueidentifier NOT NULL
, insert_date datetime NOT NULL CONSTRAINT DF__CorrelateAgentToSSIS__insert_date DEFAULT (GETDATE())
);
4 columns there. The first will be the guid an instance of SQL Server Agent generates. The second column is your tracking guid.
Step 0 would look something like
declare #jobid uniqueidentifier = CONVERT(uniqueidentifier, $(ESCAPE_NONE(JOBID)))
-- Populate this however it needs to be done
, #myguid uniqueidentifier = newid()
Your job steps for SSIS will change a bit. Instead of using the native SSIS jobstep type, you're going to use the TSQL type and do something like this.
DECLARE #execution_id bigint
, #jobid uniqueidentifier = CONVERT(uniqueidentifier, $(ESCAPE_NONE(JOBID)));
DECLARE #runid uniqueidentifier = (SELECT TOP 1 runid FROM dbo.CorrelateAgentToSSIS AS CATS WHERE CATS.jobid = #jobid);
EXEC SSISDB.catalog.create_execution
#package_name = N'SomePackage.dtsx'
, #execution_id = #execution_id OUTPUT
, #folder_name = N'MyFolder'
, #project_name = N'MyProject'
, #use32bitruntime = False
, #reference_id = NULL;
-- ddl left as exercise to the reader
INSERT INTO dbo.RunToSSIS
SELECT
#run
, #execution_id;
DECLARE #var0 smallint = 1;
EXEC SSISDB.catalog.set_execution_parameter_value
#execution_id
, #object_type = 50
, #parameter_name = N'LOGGING_LEVEL'
, #parameter_value = #var0;
-- This assumes you have a parameter defined in SSIS packages to receive the
-- runid guid
EXEC SSISDB.catalog.set_execution_parameter_value
#execution_id
, #object_type = 50
, #parameter_name = N'RUN_ID'
, #parameter_value = #runid;
EXEC SSISDB.catalog.start_execution
#execution_id;
GO
Finally, while you're collecting metrics, you might also want to think about linking a job run to the data the packages collect in the SSISDB. You can bridge that gap by recording the jobid/runid to a bigint of execution_id. If you're running packages from the SSISDB, you can plumb in System variable ServerExecutionID. I do this in the first step of every package with an Execute SQL Task. In packages run from VS, the value is 0. Otherwise, it's the value you see in SSISDB.catalog.operations Knowing those three things will allow you to see how the Agent job did, correlate it to your custom guid and whatever metrics you collect and you can pull apart performance data from the SSIS catalog.
https://dba.stackexchange.com/questions/13347/get-job-id-or-job-name-from-within-executing-job
https://dba.stackexchange.com/questions/38808/relating-executioninstanceguid-to-the-ssisdb
I'm running into a very bizarre error in SQL that I would like some help with. The code below creates the two stored procedures I need for this report:
USE [ONDTTEST]
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROCEDURE ONDT_OPS_OTD_STEP1
AS
BEGIN
SET NOCOUNT ON;
/** Create Table **/
CREATE TABLE ONDT_OPS_OTD_SMARTVIEW
(
ID varchar(15) NOT NULL,
STATUS char(1) NOT NULL
);
/** Insert data into table **/
INSERT INTO [ONDT_OPS_OTD_SMARTVIEW]
SELECT [ID]
,[STATUS]
FROM [CUSTOMER_ORDER]
WHERE [STATUS] = 'H';
/** Update customer order table from H to R **/
UPDATE [CUSTOMER_ORDER]
SET [STATUS] = 'R'
WHERE [STATUS] = 'H';
END
GO
CREATE PROCEDURE ONDT_OPS_OTD_STEP2
AS
BEGIN
SET NOCOUNT ON;
/** Update records back to H **/
UPDATE [CUSTOMER_ORDER]
SET [STATUS] = 'H'
WHERE [ID] IN (SELECT [ID] FROM [ONDT_OPS_OTD_SMARTVIEW]);
/** Drop Table **/
DROP TABLE ONDT_OPS_OTD_SMARTVIEW;
END
GO
When I execute both stored procedures it runs succesfully, however when I call the stored procedure from a scheduled job it yields the following error:
Executed as user: admin. String or binary data would be truncated. [SQLSTATE 22001] (Error 8152) The statement has been terminated. [SQLSTATE 01000] (Error 3621). The step failed.
I ran profiler and saw that it fails at the update step to the CUSTOMER_ORDER table on both procedures. The Status column is set as char(1) and the stored procedure works correctly when called from a query but it is failing when being called from a job.
Thanks
After a lot of struggle I found the culprit. It turns out that SQL does not like the fact that the user name running the job is over 50 characters long. The query works fine when I added:
EXEC AS USER = 'sysadm'
Which is my SQL admin account (same rights as the domain one). Hopefully this will save someone from the headache it caused me.
Thanks to everyone who pitched in to help, you guys are great.
Regards,
EDIT: The things I've tried below came directly from the alleged duplicate. The solutions actually do work fine with a user defined sp (and probably most system sp's), but for whatever reason it doesn't work with this one.
I can run exec sp_showpendingchanges on the distribution publication database without any issues. However I want to capture the results in a table
I've tried:
SELECT * INTO #tmpTable
FROM OPENROWSET('SQLNCLI', 'Server=SERVER; Trusted_Connection=yes;',
'EXEC sp_showpendingchanges')
and:
SELECT * INTO #tmpTable
FROM OPENQUERY(SERVER, 'exec sp_showpendingchanges')
Both of these statements return an error that says: Invalid object name 'sysmergepublications'.
I tried to specify the initial catalog in the connection string and even tried adding a USE statement in the last parameter of each statement (i.e. I used an embedded EXEC statement with double-single quotes and all that). But I still end up with the same error.
So how can I get the results from exec sp_showpendingchanges into a temporary table, and preferably without having to define the table myself? If all else fails I will make a program in C#, but really hoping there's a simpler way to just do this with just SQL.
Here is a working example
You create a table
DECLARE #result_table TABLE
(
destination_server SYSNAME ,
pub_name SYSNAME ,
destination_db_name SYSNAME ,
is_dest_subscriber BIT ,
article_name SYSNAME ,
pending_deletes INT ,
pending_ins_and_upd INT
)
execute the script
INSERT INTO #result_table
EXEC sp_showpendingchanges
view the results
SELECT * FROM #result_table
I read your question but definetly cannot understand what the problem to create temp table. Anyway, if you can execute SP but get an error when you do it through linkedserver or openrowset - than problem is in permissions.
Check permissions on sysmergepublications table. If user, which you use for linked server or openrowset, has no grant on do select this table you need to add this permission to user.
I hope it will help you.
I have created ssis package to generate excel file dynamically from sql table.
But when I try to check whether that excel connection temp table if present or not using below query in sql task it gets failed syntax error
IF object_id(MyExcel) is not null
CREATE TABLE `MyExcel` (
`CUSIP` varchar(50),
`FaceAmount` decimal(18,4),
`Portfolio` varchar(50),
`PositionDate` DateTime,
`PositionCost` decimal(18,6),
`CurrentPrice` decimal(18,6)
)
else drop table MyExcel
ERROR :
[Execute SQL Task] Error: Executing the query "IF object_id(MyExcel) is not null
CREATE TABLE `..." failed with the following error: "Invalid SQL statement; expected 'DELETE', 'INSERT', 'PROCEDURE', 'SELECT', or 'UPDATE'.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
Please advise?
I have tried with answer
IF OBJECT_ID(N'MyExcel') IS NOT NULL
BEGIN
DROP TABLE MyExcel;
END;
CREATE TABLE [MyExcel]
(
[CUSIP] VARCHAR(50),
[FaceAmount] DECIMAL(18,4),
[Portfolio] VARCHAR(50),
[PositionDate] DATETIME,
[PositionCost] DECIMAL(18,6),
[CurrentPrice] DECIMAL(18,6)
);
But still getting same error for statements
IF OBJECT_ID(N'MyExcel') IS NOT NULL
BEGIN
DROP TABLE MyExcel;
END;
I'm using this query inside SQL TASK
Connection type is EXCEL
This appears to be a combination of SQL-Server syntax (OBJECT_ID('ObjectName')) and MySQL syntax (back ticks for object names). I am assuming you are connecting to a SQL-Server database so you should qualify your object names with []. e.g.
IF OBJECT_ID(N'MyExcel') IS NOT NULL
BEGIN
CREATE TABLE [MyExcel]
(
[CUSIP] VARCHAR(50),
[FaceAmount] DECIMAL(18,4),
[Portfolio] VARCHAR(50),
[PositionDate] DATETIME,
[PositionCost] DECIMAL(18,6),
[CurrentPrice] DECIMAL(18,6)
);
END;
ELSE
BEGIN
DROP TABLE MyExcel;
END;
However, I believe your logic is flawed, your statement is saying "If the table exists, create it, if not drop it", so if it does already exist you will get an error saying the table already exists, if it doesn't then you will get an error saying you can't drop it because it doesn't exist. What you would really want is:
IF OBJECT_ID(N'MyExcel') IS NULL
BEGIN
CREATE TABLE [MyExcel]
....
However, this still presents you with a problem, since if the table exists before the task is run, it won't after, if it doesn't exist before then it will be created, which means whether or not the table exists after the task completes is dependent on whether or not the table exists before. I would imagine you want to do something like:
IF OBJECT_ID(N'MyExcel') IS NULL
BEGIN
CREATE TABLE [MyExcel]
(
[CUSIP] VARCHAR(50),
[FaceAmount] DECIMAL(18,4),
[Portfolio] VARCHAR(50),
[PositionDate] DATETIME,
[PositionCost] DECIMAL(18,6),
[CurrentPrice] DECIMAL(18,6)
);
END;
ELSE
BEGIN
TRUNCATE TABLE [MyExcel];
-- If you don't want to truncate the table and want it with
-- it's previous data in just remove the entire `else` clause
END;
Or
IF OBJECT_ID(N'MyExcel') IS NOT NULL
BEGIN
DROP TABLE MyExcel;
END;
CREATE TABLE [MyExcel]
(
[CUSIP] VARCHAR(50),
[FaceAmount] DECIMAL(18,4),
[Portfolio] VARCHAR(50),
[PositionDate] DATETIME,
[PositionCost] DECIMAL(18,6),
[CurrentPrice] DECIMAL(18,6)
);
i.e. after the task is run you will always have an table called MyExcel in the database, so you know it will be there when you get to the next step in your SSIS package.
ADDENDUM
As far as I know, you cannot use IF with an excel connection. There is an article here on querying the meta data from an excel workbook so you can check if a table exists. This is probably the technically correct way of doing it.
I was able to create a work around though, by having an Execute SQL Task with the following SQL:
DELETE
FROM MyExcel;
Then adding another Execute SQL Task to the On Error event handler to run:
CREATE TABLE MyExcel
(
CUSIP VARCHAR(50),
FaceAmount DECIMAL(18,4),
Portfolio VARCHAR(50),
PositionDate DATETIME,
PositionCost DECIMAL(18,6),
CurrentPrice DECIMAL(18,6)
);
So if the table does not exists, the delete statement will throw an error, which will trigger the create table statement. Thus ensuring after the task has run the table MyExcel definitely exists.
These tasks could be reversed, the result would be the same
You don't create an excel file with CREATE TABLE. If you come to think of excel file as database, the tables would be.. worksheets :).
To create an excel file, you'd Excel Connection Manager. You need to point it once to a template file, and after you set up excel destination with correct expressions, the file will be created. If it already exists, you just need to redirect the error.
And then you use Execute Sql to create the table (i.e. worksheet), of your desired structure.
You can add a Execute SQL Task to the end of the control flow to Drop the Excel table(s) in the package. This will leave the environment ready for the package to be processed again.
We have an C# application which posts to a database which is replicated to another database (using merge-replication) and has one custom resolver which is a stored procedure.
This was working fine under SQL Server 2000 , but when testing under SQL Server 2005 the custom resolver is attempting to change any empty varchar columns to be nulls (and failing cos this particular column does not allow nulls).
Note that these varchar fields are not the ones which cause the conflict as they are current empty on both databases and are not being changed and the stored procedure does not change them (all it is doing is attempting to set the value of another money column).
Has anyone come across this problem, or has example of a stored procedure which will leave empty strings as they are?
The actual stored procedure is fairly simply and and re-calculates the customer balance in the event of a conflict.
ALTER procedure [dbo].[ReCalculateCustomerBalance]
#tableowner sysname,
#tablename sysname,
#rowguid varchar(36),
#subscriber sysname,
#subscriber_db sysname,
#log_conflict INT OUTPUT,
#conflict_message nvarchar(512) OUTPUT
AS
set nocount on
DECLARE
#CustomerID bigint,
#SysBalance money,
#CurBalance money,
#SQL_TEXT nvarchar(2000)
Select #CustomerID = customer.id from customer where rowguid= #rowguid
Select #SysBalance = Sum(SystemTotal), #CurBalance = Sum(CurrencyTotal) From CustomerTransaction Where CustomerTransaction.CustomerID = #CustomerID
Update Customer Set SystemBalance = IsNull(#SysBalance, 0), CurrencyBalance = IsNull(#CurBalance, 0) Where id = #CustomerID
Select * From Customer Where rowguid= #rowguid
Select #log_conflict =0
Select #conflict_message ='successful'
Return(0)
You have a few options here, each are a bit of a workaround from what my research seems to show is an issue with SQL Server.
1- Alter this statement: Select * From Customer Where rowguid= #rowguid to explicitly mention each of the columns, and use an "isNull" for the offending fields
2- Alter the column in the table to add a default constraint for ''. What this will do, is if you attempt to insert a 'null', it will replace it with the empty string
3- Add a 'before insert' trigger which will alter the data before the insert, to not contain a 'null' anymore
PS: Are you positive that the replication system has that column marked as "required"? I think if it is not required, it will insert 'null' if no data exists.