How to Pass API Response to Azure SQL Table via Stored Procedure - azure-sql-database

I have a Azure Data Factory pipeline that uploads data to Salesforce and then gets the response back to Azure Data Factory. I am trying to get the response from the failed job and store in my failed records table in Azure SQL. I am using Stored Procedure activity to pass on the values to the table. The issue I am having is that the stored Procedure activity is getting the correct input for response but when it passes the value to SQL it just insert ".
I previously had the failed_records_details column set as Varchar(max) and then tried to change it to NText and see if SQL would accept the string value passes on from the API response. But I haven't had any luck so far. Any help would be really appreciated.

Update
OP changed his table structure to varchar(max) and it worked.
After many tests, I found the answer. I used OPENJSON to process in stored procedure.
This is my table:
CREATE TABLE [dbo].[product](
[PRODUCT_ID] [int] NULL,
[PRODUCT_TYPE] [int] NULL,
[PRODUCT_NAME] [varchar](50) NULL,
[PRODUCT_TITLE] [varchar](255) NULL,
[PRODUCT_PIC] [varchar](255) NULL,
[CREATE_TIME] [datetime] NULL,
[UPDATE_TIME] [datetime] NULL,
[PRODUCT_INTRO] [text] NULL,
[PRODUCT_FEATURE_ID] [varchar](255) NULL,
[PRODUCT_PARAM] [varchar](255) NULL
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO
This is my API response:
{
"productId": 2,
"productTypeId": 2,
"productName": "AD2",
"productTitle": "TopAir AD215-1000A",
"productPic": "/img/productPic/1593684705859user.jpg",
"createTime": "2020-06-05 09:17:31",
"updateTime": "2020-06-05 09:17:31",
"productFeatureId": "",
"productParam": "/img/productPic/param/1593685548627a32415ca9a21f6f9a1d99b2731f224b5d319c424.jpg",
"productIntro": "",
"productType": null,
...
This is my stored procedure, the api request as a string type input parameter.
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROCEDURE [dbo].[uspProduct] #product NVARCHAR(MAX)
AS
BEGIN TRY
INSERT INTO dbo.product(PRODUCT_ID,PRODUCT_TYPE,PRODUCT_NAME,PRODUCT_TITLE,PRODUCT_PIC,CREATE_TIME,UPDATE_TIME,PRODUCT_INTRO,PRODUCT_FEATURE_ID,PRODUCT_PARAM)
SELECT * FROM OPENJSON(#product)
WITH(
PRODUCT_ID int '$.productId',
PRODUCT_TYPE int '$.productTypeId',
PRODUCT_NAME varchar(50) '$.productName',
PRODUCT_TITLE varchar(255) '$.productTitle',
PRODUCT_PIC varchar(255) '$.productPic',
CREATE_TIME datetime '$.createTime',
UPDATE_TIME datetime '$.updateTime',
PRODUCT_INTRO varchar(255) '$.productIntro',
PRODUCT_FEATURE_ID varchar(255) '$.productFeatureId',
PRODUCT_PARAM varchar(255) '$.productParam'
)
END TRY
BEGIN CATCH
PRINT ERROR_MESSAGE ( )
END CATCH
;
Add dynamic content #string(activity('Web1').output). After repeated testing, here we must convert API response from object type to string type.
This is the input of the Stored procedure1 activity.
The debugging results are as follows:
I can see the API response was inserted into the table.

Related

Need to have a Trigger on Table with Encrypted Column in SQL Server that inserts a new record for each update into Archive Table

I have a table similar to following schema in SQL Server 2017:
Table Sample in the main database where TaxID column is encrypted using SQL Server "Always Encrypted" feature:
CREATE TABLE [dbo].[Sample]
(
[CreatedDt] [smalldatetime] NOT NULL,
[LastModDt] [smalldatetime] NOT NULL,
[CompanyID] [int] IDENTITY(1,1) NOT NULL,
[CompanyName] [varchar](250) NOT NULL,
[CompanyTaxName] [varchar](250) NULL,
[TaxID] [varchar](15) COLLATE Latin1_General_BIN2 ENCRYPTED WITH (COLUMN_ENCRYPTION_KEY =
[CEK_Auto1], ENCRYPTION_TYPE = Deterministic, ALGORITHM = 'AEAD_AES_256_CBC_HMAC_SHA_256') NOT NULL,
[Active] [bit] NOT NULL
)
Then we have another table with same schema in archive database for history purposes with TaxID encrypted.
This is the table Sample in the Main_Archive database:
CREATE TABLE [dbo].[Sample]
(
[CreatedDt] [smalldatetime] NOT NULL,
[LastModDt] [smalldatetime] NOT NULL,
[CompanyArchiveID] [int] IDENTITY(1,1) NOT NULL,
[CompanyID] [int] IDENTITY(1,1) NOT NULL,
[CompanyName] [varchar](250) NOT NULL,
[CompanyTaxName] [varchar](250) NULL,
[TaxID] [varchar](15) COLLATE Latin1_General_BIN2 ENCRYPTED WITH (COLUMN_ENCRYPTION_KEY =
[CEK_Auto1], ENCRYPTION_TYPE = Deterministic, ALGORITHM = 'AEAD_AES_256_CBC_HMAC_SHA_256') NOT NULL,
[Active] [bit] NOT NULL
)
Now, we want to have a trigger on the main Sample table that inserts a new record into the archive Sample table for every update.
The trigger for the Sample table in the main database is as follows:
CREATE TRIGGER [dbo].[tr_iud_Sample]
ON [dbo].[Sample]
FOR INSERT, UPDATE, DELETE
AS
BEGIN
SET NOCOUNT ON
DECLARE #CurrDt AS SMALLDATETIME
SELECT #CurrDt = GETDATE()
DECLARE #CurrYear AS INT
SELECT #CurrYear = YEAR(#CurrDt)
UPDATE Sample
SET LastModDt = #CurrDt,
CreatedDt = CASE WHEN d.CompanyID IS NULL THEN #CurrDt ELSE Sample.CreatedDt END
FROM inserted i WITH (NOLOCK)
LEFT JOIN deleted d WITH (NOLOCK) ON d.CompanyID= i.CompanyID
WHERE Sample.CompanyID = i.CompanyID
INSERT INTO [Main_Archive].[dbo].Sample
SELECT CreatedDt, LastModDt, CompanyID, CompanyName, CompanyTaxName, TaxID, Active
FROM deleted
END
ALTER TABLE [dbo].[Sample] ENABLE TRIGGER [tr_iud_Sample]
GO
ALTER TABLE [dbo].[Vendor] DISABLE TRIGGER [tr_iud_Sample]
GO
But this fails and I get this error:
Msg 4920, Level 16, State 0, Line 50
Operand type clash: varchar(15) encrypted with (encryption_type = 'DETERMINISTIC', encryption_algorithm_name = 'AEAD_AES_256_CBC_HMAC_SHA_256', column_encryption_key_name = 'CEK_Auto1', column_encryption_key_database_name = 'NCI_COMMON') collation_name = 'Latin1_General_BIN2' is incompatible with varchar
Is there a way to have a trigger on encrypted table and if so, how to achieve the
desired functionality?
Also, if SQL Server currently does not support that, is there any work around to achieve that?
Thank you in advance
As you are using Always Encrypted your SQL Server version System-Versioned Temporal Tables.
You can make your table system-versioned and leave the work of maintaining the history to the SQL Server Engine (also, when you are changing your table design, the engine will mitigate the changes to the history table).
Temporal tables can be queried using special clauses and bring to you new ways for analyzing historical data.
One disadvantage I have faced is that the history table columns must match the target table ones - so, if you need to have a ModifiedBy column in the history, you must change your application to populate such value in the original table.

SQL Server get table name in database trigger

I want to use the database level DDL trigger for logging actions in my DB.
I need to get table name and action (insert, update, delete) and write it in table with logs.
Can I get table name in database level trigger and use it to insert table name?
Or need to put triggers on all the tables?
Here's a setup that I use on many of my databases. It demonstrates most of the things that you asked about:
SET ANSI_NULLS ON
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [Meta].[DdlEvents](
[ID] [int] IDENTITY(1,1) NOT NULL,
[CreatedOn] [datetime] NULL,
[CreatedBy] [sysname] NULL,
[CreateBy2] [sysname] NULL,
[SchemaName] [sysname] NULL,
[ObjectName] [sysname] NULL,
[HostName] [sysname] NULL,
[ProgramName] [sysname] NULL,
[SqlCommand] [nvarchar](max) NULL,
[XmlData] [xml] NULL
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO
CREATE TRIGGER [DDLTrigger_LogDDL]
ON DATABASE
FOR DDL_DATABASE_LEVEL_EVENTS
AS
BEGIN
SET NOCOUNT ON;
DECLARE
#EventData XML = EVENTDATA();
INSERT INTO Meta.DdlEvents(
SqlCommand,
SchemaName,
ObjectName,
HostName,
ProgramName,
XmlData
)
VALUES (
#EventData.value('(/EVENT_INSTANCE/TSQLCommand)[1]', 'NVARCHAR(MAX)'),
#EventData.value('(/EVENT_INSTANCE/SchemaName)[1]', 'NVARCHAR(255)'),
#EventData.value('(/EVENT_INSTANCE/ObjectName)[1]', 'NVARCHAR(255)'),
HOST_NAME(),
PROGRAM_NAME(),
#EventData
);
END
GO
ENABLE TRIGGER [DDLTrigger_LogDDL] ON DATABASE
GO
Specifically, you use the EVENTDATA() function within the Database DDL trigger to get the Event XML, then you can extract the object(table's) object and schema at the /EVENT_INSTANCE/SchemaName and /EVENT_INSTANCE/ObjectName nodes.
Follow-up on Martin's comment below for the OP, a DDL triger only fires on DDL events, that is commands like CREATE,ALTER and DROP. It does not fire for DML events like INSERT, UPDATE and DELETE. So, if that is what you need, then this answer would not work, and yes, you would need a trigger on every table.

empty string changing to 0 for int

Very unique..... and not authorized to change collation
i have created a table
CREATE TABLE BT_INPUT_CHK
(
[ID] [int] NOT NULL,
[RELID] [int] NULL,
[OPNO] [varchar](35) NULL
)
and when i am inserting data
INSERT ZBT_INPUT_CHK ([ID],[RELID],[OPNO])
VALUES('1000002','','')
the problem starts here.
for varchar empty string inserted as same, but for int empty string is changing to 0.
collation: Latin1_General_CI_AS
can anyone please help me on this.
Thanks
When empty sting is implicitly converted to INT it converts to ZERO. if you need to set it to NULL then use NULLIF function.
DECLARE #RELID VARCHAR(10) = ''
INSERT ZBT_INPUT_CHK ([ID],[RELID],[OPNO])
VALUES('1000002',NULLIF(#RELID,''),'')
Check This..You will get clear idea on Inserting Empty String to Numeric Column
INSERT into BT_INPUT_CHK ([ID],[RELID],[OPNO]) VALUES('1000007',NULL,'');
try this,
CREATE TABLE #BT_INPUT_CHK
(
[ID] [int] NOT NULL,
[RELID] [int] NULL,
[OPNO] [varchar](35) NULL
)
INSERT #BT_INPUT_CHK ([ID],[RELID],[OPNO])
VALUES('1000002',(case when cast('' as varchar(10))='' then 'Error' else '' end),'')

how to find what values were passed onto a store procedure at the last run

My web application is calling a store procedure but not returning the desired results in the ASP.net page. When i execute the store procedure in sql server by providing the same values, i get the desired result. Is there any way i could tell what values were passed onto the store procedure when it was called by the web page?
note: I am using sql server 2012 express.
Create PROCEDURE [dbo].[spvolview]
#Gender nvarchar(50),
#Age int,
#Country nvarchar(100)
AS
select
clinical_study.BRIEF_TITLE, clinical_study.OVERALL_STATUS, clinical_study.PHASE, clinical_study.STUDY_TYPE, clinical_study.GENDER, location_countries.COUNTRY, facilities.CITY
from clinical_study
inner join location_countries
ON clinical_study.NCT_ID=location_countries.NCT_ID
inner join facilities
on clinical_study.NCT_ID = facilities.NCT_ID
where (clinical_study.GENDER LIKE '%'+#Gender+'%' or clinical_study.GENDER like 'Both')
and clinical_study.HEALTHY_VOLUNTEERS <> 'No'
and clinical_study.OVERALL_STATUS like 'Rec%'
and #Age>(case when isnumeric(minimum_age)=1 then left(minimum_age,2) end)
and #Age<(case when isnumeric(MAXIMUM_AGE)=1 then left(MAXIMUM_AGE,2) end)
and location_countries.COUNTRY = #Country
If you want to "see what values were supplied to the store procedure in the database itself" as you wrote in your comment you can create a simple table and make your store procedure write the parameters' values in it:
create table where you will store values (for example use following code in a "New query" tab):
CREATE TABLE [dbo].[TMP_LOG](
[id] [bigint] IDENTITY(1,1) NOT NULL,
[Gender] [nvarchar](50) NULL,
[Age] [int] NULL,
[Country] [nvarchar](100) NULL)
edit your stored procedure adding an insert statement at the beginning of your procedure (right before the select statement):
insert into [dbo].[TMP_LOG] select #Gender, #Age, #Country

How to use user defined table types in multiple Stored-Procedure?

I have created one User Defined Table Type in SQL Server and accessing in two stored procedures.
CREATE TYPE [dbo].[UDType_Sample] AS TABLE(
[Id] [bigint] NULL,
[Id2] [bigint] NOT NULL,
[Id3] [bigint] NOT NULL
)
We are using the type in our first StoredProcedure and executed successfully.And if we are using in the second procedure then it taking long time to execute the procedure and it causes some problem in my application regarding this.
Please suggest me about this issue.
Thanks,
Kanna