Related
I'm trying to create an external table in SQL Server 2019 on premise ( Polybase has been installed and all services are up and running [Instance and 2 services for polybase] , Hadoop configuration = 7).
I want to feed the external table with multiple ".csv" or ".txt" files , for this example, Ill use 1 ".csv" file.
I've the lattest
OBCD Driver for "Driver=Microsoft Access Text Driver (*.txt, *.csv)"
"SQL Server® 2019 for Microsoft® Windows Latest Cumulative Update"
My first step was creating the EXTERNAL DATA SOURCE in my Database where Polybase is emabled using the code below:
CREATE EXTERNAL DATA SOURCE MyODBC
WITH
(
LOCATION = 'odbc://localhost',
CONNECTION_OPTIONS = 'Driver=Microsoft Access Text Driver (*.txt, *.csv);Dbq=C:\Historical\Event\',
PUSHDOWN = OFF
);
Second, I've created the script to CREATE the EXTERNAL TABLE using dynamic SQL based on variables ( #TableName,#FolderPath variable values can change at any time).
Please see my script below:
DECLARE
#TableName VARCHAR(15) = 'BRAO',
#FolderPath VARCHAR(500) = 'BRAO\BRAO_EVEN0000_20220101.csv',
#DataSourceName VARCHAR(20) ='MyODBC',
#SQLScript NVARCHAR(MAX) =''
SET #SQLScript =N'
IF EXISTS
(
SELECT
*
FROM sys.objects
WHERE
object_id = OBJECT_ID(N''[dbo].[Gold' + #TableName + ']'')
AND type in (N''U'')
)
DROP EXTERNAL TABLE [dbo].[Gold' + #TableName + ']
CREATE EXTERNAL TABLE [dbo].[Gold' + #TableName + ']
(
[Column1] [varchar](255) NULL,
[Column2] [varchar](255) NULL,
[Column3] [varchar](255) NULL
)
WITH (DATA_SOURCE =' + #DataSourceName + ',LOCATION = ''' + #FolderPath + ''')'
EXEC sp_executesql #SQLScript
When I run my script above, I get the following error:
Msg 105121, Level 16, State 1, Line 1
105121;The specified LOCATION string 'BRAO\BRAO_EVEN0000_20220101.csv' could not be parsed. A 1-part identifier was found. Expected 2.
Have you looked at the following page? It loads a .csv file but the location is only a single folder: https://learn.microsoft.com/en-us/answers/questions/118102/polybase-load-csv-file-that-contains-text-column-w.html
I created few user defined types in my database as below
CREATE TYPE [dbo].[StringID] FROM [nvarchar](20) NOT NULL
and assigned them to various tables. The tables in my database are in various schemas (not only dbo)
But I realized I need bigger field, and I need to alter, e.g increase from [nvarchar](20) to [nvarchar](50), but there is no ALTER TYPE statement.
I need a script that uses a temp table/cursor whatever and saves all the tables and fields where my type is used. Then change existing fields to base type - e.g. from CustID [StringID] to CustID [nvarchar(20)].
Drop the user type and recreate it with new type - e.g. nvarchar(50)
and finally set back fields to user type
I do not have rules defined on types, so don't have to drop rules and re-add them.
Any help is appreciated.
This is what I normally use, albeit a bit manual:
/* Add a 'temporary' UDDT with the new definition */
exec sp_addtype t_myudt_tmp, 'numeric(18,5)', NULL
/* Build a command to alter all the existing columns - cut and
** paste the output, then run it */
select 'alter table dbo.' + TABLE_NAME +
' alter column ' + COLUMN_NAME + ' t_myudt_tmp'
from INFORMATION_SCHEMA.COLUMNS
where DOMAIN_NAME = 't_myudt'
/* Remove the old UDDT */
exec sp_droptype t_mydut
/* Rename the 'temporary' UDDT to the correct name */
exec sp_rename 't_myudt_tmp', 't_myudt', 'USERDATATYPE'
We are using the following procedure, it allows us to re-create a type from scratch, which is "a start". It renames the existing type, creates the type, recompiles stored procs and then drops the old type. This takes care of scenarios where simply dropping the old type-definition fails due to references to that type.
Usage Example:
exec RECREATE_TYPE #schema='dbo', #typ_nme='typ_foo', #sql='AS TABLE([bar] varchar(10) NOT NULL)'
Code:
CREATE PROCEDURE [dbo].[RECREATE_TYPE]
#schema VARCHAR(100), -- the schema name for the existing type
#typ_nme VARCHAR(128), -- the type-name (without schema name)
#sql VARCHAR(MAX) -- the SQL to create a type WITHOUT the "CREATE TYPE schema.typename" part
AS DECLARE
#scid BIGINT,
#typ_id BIGINT,
#temp_nme VARCHAR(1000),
#msg VARCHAR(200)
BEGIN
-- find the existing type by schema and name
SELECT #scid = [SCHEMA_ID] FROM sys.schemas WHERE UPPER(name) = UPPER(#schema);
IF (#scid IS NULL) BEGIN
SET #msg = 'Schema ''' + #schema + ''' not found.';
RAISERROR (#msg, 1, 0);
END;
SELECT #typ_id = system_type_id FROM sys.types WHERE UPPER(name) = UPPER(#typ_nme);
SET #temp_nme = #typ_nme + '_rcrt'; -- temporary name for the existing type
-- if the type-to-be-recreated actually exists, then rename it (give it a temporary name)
-- if it doesn't exist, then that's OK, too.
IF (#typ_id IS NOT NULL) BEGIN
exec sp_rename #objname=#typ_nme, #newname= #temp_nme, #objtype='USERDATATYPE'
END;
-- now create the new type
SET #sql = 'CREATE TYPE ' + #schema + '.' + #typ_nme + ' ' + #sql;
exec sp_sqlexec #sql;
-- if we are RE-creating a type (as opposed to just creating a brand-spanking-new type)...
IF (#typ_id IS NOT NULL) BEGIN
exec recompile_prog; -- then recompile all stored procs (that may have used the type)
exec sp_droptype #typename=#temp_nme; -- and drop the temporary type which is now no longer referenced
END;
END
GO
CREATE PROCEDURE [dbo].[recompile_prog]
AS
BEGIN
SET NOCOUNT ON;
DECLARE #v TABLE (RecID INT IDENTITY(1,1), spname sysname)
-- retrieve the list of stored procedures
INSERT INTO
#v(spname)
SELECT
'[' + s.[name] + '].[' + items.name + ']'
FROM
(SELECT sp.name, sp.schema_id, sp.is_ms_shipped FROM sys.procedures sp UNION SELECT so.name, so.SCHEMA_ID, so.is_ms_shipped FROM sys.objects so WHERE so.type_desc LIKE '%FUNCTION%') items
INNER JOIN sys.schemas s ON s.schema_id = items.schema_id
WHERE is_ms_shipped = 0;
-- counter variables
DECLARE #cnt INT, #Tot INT;
SELECT #cnt = 1;
SELECT #Tot = COUNT(*) FROM #v;
DECLARE #spname sysname
-- start the loop
WHILE #Cnt <= #Tot BEGIN
SELECT #spname = spname
FROM #v
WHERE RecID = #Cnt;
--PRINT 'refreshing...' + #spname
BEGIN TRY -- refresh the stored procedure
EXEC sp_refreshsqlmodule #spname
END TRY
BEGIN CATCH
PRINT 'Validation failed for : ' + #spname + ', Error:' + ERROR_MESSAGE();
END CATCH
SET #Cnt = #cnt + 1;
END;
END
there's a good example of a more comprehensive script here
It's worth noting that this script will include views if you have any. I ran it and instead of exec'ing inline generated a script as the output which I then tweaked and ran.
Also, if you have functions/sprocs using the user defeined types you'll need to drop those before running your script.
Lesson Learned: in future, don't bother with UDTs they're more hassle than they're worth.
SET NOCOUNT ON
DECLARE #udt VARCHAR(150)
DECLARE #udtschema VARCHAR(150)
DECLARE #newudtschema VARCHAR(150)
DECLARE #newudtDataType VARCHAR(150)
DECLARE #newudtDataSize smallint
DECLARE #OtherParameter VARCHAR(50)
SET #udt = 'Name' -- Existing UDDT
SET #udtschema = 'dbo' -- Schema of the UDDT
SET #newudtDataType = 'varchar' -- Data type for te new UDDT
SET #newudtDataSize = 500 -- Lenght of the new UDDT
SET #newudtschema = 'dbo' -- Schema of the new UDDT
SET #OtherParameter = ' NULL' -- Other parameters like NULL , NOT NULL
DECLARE #Datatype VARCHAR(50),
#Datasize SMALLINT
DECLARE #varcharDataType VARCHAR(50)
DECLARE #Schemaname VARCHAR(50),
#TableName VARCHAR(50),
#FiledName VARCHAR(50)
CREATE TABLE #udtflds
(
Schemaname VARCHAR(50),
TableName VARCHAR(50),
FiledName VARCHAR(50)
)
SELECT TOP 1
#Datatype = Data_type,
#Datasize = character_maximum_length
FROM INFORMATION_SCHEMA.COLUMNS
WHERE Domain_name = #udt
AND Domain_schema = #udtschema
SET #varcharDataType = #Datatype
IF #DataType Like '%char%'
AND #Datasize IS NOT NULL
AND ( #newudtDataType <> 'varchar(max)'
OR #newudtDataType <> 'nvarchar(max)'
)
BEGIN
SET #varcharDataType = #varcharDataType + '('
+ CAST(#Datasize AS VARCHAR(50)) + ')'
END
INSERT INTO #udtflds
SELECT TABLE_SCHEMA,
TABLE_NAME,
Column_Name
FROM INFORMATION_SCHEMA.COLUMNS
WHERE Domain_name = #udt
AND Domain_schema = #udtschema
DECLARE #exec VARCHAR(500)
DECLARE alter_cursor CURSOR
FOR SELECT Schemaname,
TableName,
FiledName
FROM #udtflds
OPEN alter_cursor
FETCH NEXT FROM alter_cursor INTO #Schemaname, #TableName, #FiledName
WHILE ##FETCH_STATUS = 0
BEGIN
SET #exec = 'Alter Table ' + #Schemaname + '.' + #TableName
+ ' ALTER COLUMN ' + #FiledName + ' ' + #varcharDataType
EXECUTE ( #exec
)
FETCH NEXT FROM alter_cursor INTO #Schemaname, #TableName, #FiledName
END
CLOSE alter_cursor
SET #exec = 'DROP TYPE [' + #udtschema + '].[' + #udt + ']'
EXEC ( #exec
)
SET #varcharDataType = #newudtDataType
IF #newudtDataType Like '%char%'
AND #newudtDataSize IS NOT NULL
AND ( #newudtDataType <> 'varchar(max)'
OR #newudtDataType <> 'nvarchar(max)'
)
BEGIN
SET #varcharDataType = #varcharDataType + '('
+ CAST(#newudtDataSize AS VARCHAR(50)) + ')'
END
SET #exec = 'CREATE TYPE [' + #newudtschema + '].[' + #udt + '] FROM '
+ #varcharDataType + ' ' + #OtherParameter
EXEC ( #exec
)
OPEN alter_cursor
FETCH NEXT FROM alter_cursor INTO #Schemaname, #TableName, #FiledName
WHILE ##FETCH_STATUS = 0
BEGIN
SET #exec = 'Alter Table ' + #Schemaname + '.' + #TableName
+ ' ALTER COLUMN ' + #FiledName + ' ' + '[' + #newudtschema
+ '].[' + #udt + ']'
EXECUTE ( #exec
)
FETCH NEXT FROM alter_cursor INTO #Schemaname, #TableName, #FiledName
END
CLOSE alter_cursor
DEALLOCATE alter_cursor
SELECT *
FROM #udtflds
DROP TABLE #udtflds
1: http://www.sql-server-performance.com/2008/how-to-alter-a-uddt/ has replaced http://www.sql-server-performance.com/faq/How_to_alter_a%20_UDDT_p1.aspx
The simplest way to do this is through Visual Studio's object explorer, which is also supported in the Community edition.
Once you have made a connection to SQL server, browse to the type, right click and select View Code, make your changes to the schema of the user defined type and click update. Visual Studio should show you all of the dependencies for that object and generate scripts to update the type and recompile dependencies.
As devio says there is no way to simply edit a UDT if it's in use.
A work-round through SMS that worked for me was to generate a create script and make the appropriate changes; rename the existing UDT; run the create script; recompile the related sprocs and drop the renamed version.
The solutions provided here can only be applied if the user defined types are used in table definitions only, and if the UDT columns are not indexed.
Some developers also have SP's and functions using UDT parameters, which is not covered either. (see comments on Robin's link and in the Connect entry)
The Connect entry from 2007 has finally been closed after 3 years:
Thank you for submitting this
suggestion, but given its priority
relative to the many other items in
our queue, it is unlikely that we will
actually complete it. As such, we are
closing this suggestion as “won’t
fix”.
I tried to solve a similiar problem ALTERing XML SCHEMA COLLECTIONS, and the steps seem to mostly apply to ALTER TYPE, too:
To drop a UDT, the following steps are necessary:
If a table column references the UDT, it has to be converted to the underlying type
If the table column has a default constraint, drop the default constraint
If a procedure or function has UDT parameters, the procedure or function has to be dropped
If there is an index on a UDT column, the index has to be dropped
If the index is a primary key, all foreign keys have to be dropped
If there are computed columns based on a UDT column, the computed columns have to be dropped
If there are indexes on these computed columns, the indexes have to be dropped
If there are schema-bound views, functions, or procedures based on tables containing UDT columns, these objects have to be dropped
I ran into this issue with custom types in stored procedures, and solved it with the script below. I didn't fully understand the scripts above, and I follow the rule of "if you don't know what it does, don't do it".
In a nutshell, I rename the old type, and create a new one with the original type name. Then, I tell SQL Server to refresh its details about each stored procedure using the custom type. You have to do this, as everything is still "compiled" with reference to the old type, even with the rename. In this case, the type I needed to change was "PrizeType". I hope this helps. I'm looking for feedback, too, so I learn :)
Note that you may need to go to Programmability > Types > [Appropriate User Type] and delete the object. I found that DROP TYPE doesn't appear to always drop the type even after using the statement.
/* Rename the UDDT you want to replace to another name */
exec sp_rename 'PrizeType', 'PrizeTypeOld', 'USERDATATYPE';
/* Add the updated UDDT with the new definition */
CREATE TYPE [dbo].[PrizeType] AS TABLE(
[Type] [nvarchar](50) NOT NULL,
[Description] [nvarchar](max) NOT NULL,
[ImageUrl] [varchar](max) NULL
);
/* We need to force stored procedures to refresh with the new type... let's take care of that. */
/* Get a cursor over a list of all the stored procedures that may use this and refresh them */
declare sprocs cursor
local static read_only forward_only
for
select specific_name from information_schema.routines where routine_type = 'PROCEDURE'
declare #sprocName varchar(max)
open sprocs
fetch next from sprocs into #sprocName
while ##fetch_status = 0
begin
print 'Updating ' + #sprocName;
exec sp_refreshsqlmodule #sprocName
fetch next from sprocs into #sprocName
end
close sprocs
deallocate sprocs
/* Drop the old type, now that everything's been re-assigned; must do this last */
drop type PrizeTypeOld;
New answer to an old question:
Visual Studio Database Projects handle the drop and recreate process when you deploy changes. It will drop stored procs that use UDDTs and then recreate them after dropping and recreating the data type.
1.Rename the old UDT,
2.Execute query ,
3.Drop the old UDT.
Simple DROP TYPE first then CREATE TYPE again with corrections/alterations?
There is a simple test to see if it is defined before you drop it ... much like a table, proc or function -- if I wasn't at work I would look what that is?
(I only skimmed above too ... if I read it wrong I apologise in advance! ;)
I created procedure which count not null rows in the column, but query throws errors: #tableName is not declared and invalid object name tempTable. I don't know why code throws that errors, because all variables are declared.
Msg 1087, Level 16, State 1, Procedure getLenCol, Line 7 [Batch Start Line 0]
Must declare the table variable "#tableName".
Msg 208, Level 16, State 1, Line 11
Invalid object name 'tempTable'.
CREATE OR ALTER PROC getLenCol
#tableName varchar(255),
#colName varchar(255)
as
DECLARE #tempTable Table(smth varchar(255));
DECLARE #query varchar(255)
insert into #tempTable(smth) select #colName from #tableName where #colName is not null
exec (#query)
select ##ROWCOUNT
GO
exec getLenCol 'users','name'
Also when I make that program in another way, that code throw
Msg 1087, Level 15, State 2, Line 11
error.
Must declare the table variable "#tempTable".
CREATE OR ALTER PROC getLenCol
#tableName varchar(255),
#colName varchar(255)
as
DECLARE #tempTable Table(smth varchar(255))
DECLARE #query varchar(255)
SET #query = concat('insert into #tempTable(smth) select ',#colName,' from ',#tableName,' where ',#colName,' is not null');/*#colName from #tableName where #colName is not NULL*/
exec (#query)
select ##ROWCOUNT
GO
exec getLenCol 'users','name'
Is it a way to fix that error?
Obviously, your code is subject to SQL injection attacks -- as the comments on the question have explained.
But your issue is the scoping rules around your table variable. You can fix that by using:
set #query = concat('select ', #colName, ' from ', #tableName, ' where ', #colName,' is not null');
insert into #tempTable (smth)
exec(#query);
I don't think there is any way around the SQL injection vulnerabilities for the logic you have suggested. However, your code is so non-sensical that I doubt that it is really representative of your actual code.
As it seems that many are not aware of the dangers of SQL Injection, including Gordon, I wanted to expand on that first. Let's, take the accepted answer (at time of writing), which gives the following:
CREATE OR ALTER PROC getLenCol
#tableName varchar(255),
#colName varchar(255)
as
DECLARE #query varchar(255)
DECLARE #tempTable Table(smth varchar(255))
set #query = concat('select ', #colName, ' from ', #tableName, ' where ', #colName,' is not null');
insert into #tempTable (smth)
exec(#query);
GO
Now, let's be someone malicious:
EXEC dbo.getLenCol #colName = N'1; CREATE LOGIN NewLogin WITH PASSWORD = ''1'', CHECK_POLICY = OFF;/*',
#tableName =N'*/ ALTER SERVER ROLE sysadmin ADD MEMBER NewLogin;--';
So, what does the above, in the dynamic SQL run? Let's find out by adding PRINT #query; to the SP's definition:
select 1; CREATE LOGIN NewLogin WITH PASSWORD = '1', CHECK_POLICY = OFF;/* from */ ALTER SERVER ROLE sysadmin ADD MEMBER NewLogin;-- where 1; CREATE LOGIN NewLogin WITH PASSWORD = '1', CHECK_POLICY = OFF;/* is not null
And, with a little formatting for ease of reading:
select 1;
CREATE LOGIN NewLogin WITH PASSWORD = '1', CHECK_POLICY = OFF;
/* from */
ALTER SERVER ROLE sysadmin ADD MEMBER NewLogin;
-- where 1; CREATE LOGIN NewLogin WITH PASSWORD = '1', CHECK_POLICY = OFF;/* is not null
OH. OHHHHHHHHHHH. Congratulations you are the new proud owner of a SQL Server that has a new sysadmin LOGIN!
NEVER, inject unsanitised string into a string in SQL. NEVER.
Rather than repeating myself, I'm going to link to my article Dos and Don'ts of Dynamic SQL, however, you can easily make the above query secure with a few of uses of QUOTENAME:
CREATE OR ALTER PROC getLenCol
#schemaName sysname = N'dbo', --You should define the schema too
#tableName sysname, --An object can't be longer than 128 characters, so sysname is best
#colName sysname
AS
BEGIN
DECLARE #query nvarchar(MAX);
DECLARE #tempTable Table(smth varchar(255));
SET #QUERY = CONCAT(N'SELECT ', QUOTENAME(#colName),N' FROM ', QUOTENAME(#schemaName), N'.', QUOTENAME(#tableName), N' WHERE ', QUOTENAME(#colName), N' IS NOT NULL;');
PRINT #query;
INSERT INTO #tempTable (smth)
EXEC sys.sp_executesql #query;
END;
GO
And what happens if we run the above EXEC statement before? Well you get the statement below (with added formatting):
SELECT [1; CREATE LOGIN NewLogin WITH PASSWORD = '1', CHECK_POLICY = OFF;/*]
FROM [dbo].[*/ ALTER SERVER ROLE sysadmin ADD MEMBER NewLogin;--]
WHERE [1; CREATE LOGIN NewLogin WITH PASSWORD = '1', CHECK_POLICY = OFF;/*] IS NOT NULL;
And no surprised, that generated the error
Invalid object name 'dbo.*/ ALTER SERVER ROLE sysadmin ADD MEMBER NewLogin;--'.
Now your dynamic statement is safe from injection.
I would highly recommend against this approach, firstly calling this procedure is as much, if not more typing that just doing a count. Compare the two
EXECUTE dbo.getLenCol #tableName = 'dbo.SomeTable', #colName = 'ID';
SELECT COUNT(ID) FROM dbo.SomeTable;
Even with the shortened exec, and not using named parameters it is longer:
EXEC dbo.getLenCol dbo.SomeTable', 'ID';
It is very, very rare that a catch all query like this, with object names being passed as parameters is going to be the correct approach. There are some maintenance queries where it is useful, but these are the exception, not the rule.
If you must do this though, you should do a little bit of validation first, and check that both the table name and column name are valid before executing any dynamic SQL by using COL_LENGTH(#tableName, #ColName). e.g
CREATE OR ALTER PROC getLenCol #tableName SYSNAME, #colName SYSNAME
AS
BEGIN
IF COL_LENGTH(#tableName, #ColName) IS NOT NULL
BEGIN
DECLARE #SQL NVARCHAR(MAX) = CONCAT('SELECT COUNT(', #colName, ') FROM ', #tableName, ';');
EXECUTE sp_executesql #SQL;
RETURN;
END
-- TABLE OR COLUMN WAS NOT VALID RETURN -1 TO INDICATE THAT
SELECT -1;
END
I want to use a stored procedure to copy a table from my test database to a linked server with the same ID's / Identity but I can't get it to work..
I've set the IDENTITY_INSERT to ON but it still complains about the ID column.
Here's my procedure:
CREATE PROCEDURE [dbo].[TEST2PROD_CopyUIDataSServer]
AS Begin
declare #sql nvarchar(max)
-- First truncate target table
set #sql = 'EXEC [LINKEDSERVER].tempdb.sys.sp_sqlexec' + char(39)+ 'TRUNCATE Table [ProductManager].dbo.[UIData]' + char(39)+ ';'
---- SET IDENTITY_INSERT ON
set #sql = #sql + 'EXEC [LINKEDSERVER].tempdb.sys.sp_sqlexec' + char(39)+ 'SET IDENTITY_INSERT [ProductManager].[dbo].[UIData] ON' + char(39)+ ';'
---- INSERT UIDATA records from DB1 into linked server DB2
set #sql = #sql + 'WITH TestData as (SELECT * from ProductManager.dbo.UIData UID)' + NCHAR(13)+ 'INSERT INTO [LINKEDSERVER].[ProductManager].[dbo].[UIData]' + NCHAR(13) + 'select * from TestData;'
print #sql
exec (#sql)
end
But when I execute the SP it gives me the following error:
The OLE DB provider "SQLNCLI10" for linked server .... could not INSERT INTO table "[LINKEDSERVER].[ProductManager].[dbo].[UIData]" because of column "Id". The user did not have permission to write to the column.
Linked server properties RPC and RPC out are set to true. I hope someboy can help me out here?
UPDATE: I decided to pull things apart, first I copy the data from the local server to the linked server in a TEMP_TABLE where I don't have to deal with IDENTITY issues.
Then I wrote a stored procedure on the linked / remote server, since I'm not using SELECT * but specify the column list. Chances are this will work from the local server in an SP too but I don't have the time or interest to check it out yet..
USE [ProductManager]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROCEDURE [dbo].[TEST2PROD_CopyBaseTables]
AS BEGIN
DECLARE #DestTable VARCHAR(50)
DECLARE #DestPath VARCHAR(50)
DECLARE #SrceTable VARCHAR(255)
declare #sql nvarchar(max)
DECLARE #columnList varchar(max)
DECLARE #err int
Begin TRY
declare #comma_delimited_list varchar(4000)
--- FIRST TRY WITH ONE TABLE, EXTENDABLE...
set #comma_delimited_list = 'UIData'
declare #cursor cursor
set #cursor = cursor static for
select * from dbo.Split(#comma_delimited_list,',') a
declare #naam varchar(50)
open #cursor
while 1=1 begin
fetch next from #cursor into #DestTable
if ##fetch_status <> 0 break
--Create tablenames
SET #SrceTable = '[ProductManager].[dbo].TEMP_' + #DestTable
SET #DestPath = '[ProductManager].[dbo].'+ #DestTable
print #srceTable;
print #DestTable;
--Truncate target table
set #sql ='TRUNCATE TABLE '+ #DestPath + ';'
--Insert statement needs column names
set #columnList =''
SELECT #columnList = coalesce(#columnList + '[' + name + '],','') FROM sys.columns Where OBJECT_NAME(OBJECT_ID) = #DestTable
if RIGHT(RTRIM(#columnList),1) = ','
begin
SET #columnList = LEFT(#columnList, LEN(#columnList) - 1)
end
--Transfer data from source table 2 destination
set #sql = #sql + ' SET IDENTITY_INSERT ' + #DestPath + ' ON;' + ' INSERT INTO ' + #DestPath + '(' + #columnList + ') SELECT ' + #columnList + ' FROM ' + #SrceTable
print #sql;
exec (#sql)
end
-- not strictly necessary w/ cursor variables since the will go out of scope like a normal var
close #cursor
deallocate #cursor
End Try
Begin Catch
declare #ErrorMsg nvarchar(MAX);
select #ErrorMsg = ERROR_MESSAGE();
SELECT #err = ##error IF #err <> 0 Return #err
end Catch
END
IDENTITY_INSERT doesn't work with linked servers AFAIK, unless you execute dynamic SQL that includes the SET IDENTITY_INSERT in the batch or have some code (Stored Proc for instance) on the remote server which does that for you.
The IDENTITY_INSERT is per-session (see MSDN) and when you use the remote server this will probably be in a different session from your statement executed via [LINKEDSERVER].tempdb.sys.sp_sqlexec, which causes it to fail as you see it happening.
You can insert an identity value into a table with an identity column on a linked server with the "SWITCH TO" trick.
If you haven't used the "SWITCH TO" trick to add and remove identity on a column, it's very quick, even on large tables!
Conceptually you simply create a new SCHEMA exactly like the table you are wanting to INSERT to without the identity defined. Then switch the table to that SCHEMA and do your INSERT. Then switch back to the SCHEMA with the identity defined.
The sample below has been tested on a linked server in AZURE.
All the caveats of using "SWITCH TO" apply (indexes must be the same, drop and recreate foreign keys, etc)
To test, you can run the full script below on an Linked Azure SQL Server database. You'll need to do a find/replace with [LINKED_SERVER_NAME] and [DATABASE_NAME], replacing with your values. On a non-Azure DB you may need to add "ON PRIMARY" to the table creations.
--Let's setup the example by creating a table with an IDENTITY column on the Linked Server
EXEC('
CREATE TABLE [DATABASE_NAME].[dbo].[Example_Table](
[ID] [int] IDENTITY(1,1) NOT NULL,
[Name] [nchar](10) NULL
)
'
) AT [LINKED_SERVER_NAME]
--INSERT some data into the table
INSERT INTO [LINKED_SERVER_NAME].[DATABASE_NAME].[dbo].[Example_Table] ([Name]) VALUES ('Travis')
INSERT INTO [LINKED_SERVER_NAME].[DATABASE_NAME].[dbo].[Example_Table] ([Name]) VALUES ('Mike')
-- Looks good
SELECT * FROM [LINKED_SERVER_NAME].[DATABASE_NAME].[dbo].[Example_Table]
GO
-- Create a TABLE with an identical schema, without the identity defined
EXEC('
CREATE TABLE [DATABASE_NAME].[dbo].[Example_Table_temp](
[ID] [int] NOT NULL,
[Name] [nchar](10) NULL
)
'
) AT [LINKED_SERVER_NAME]
--Now Use the "SWITCH TO" to move the data to the new table
EXEC('
ALTER TABLE [DATABASE_NAME].[dbo].[Example_Table] SWITCH TO [DATABASE_NAME].[dbo].[Example_Table_temp]
'
) AT [LINKED_SERVER_NAME]
--Drop the old table (It should now be empty, but you may want to verify that if you are unsure here)
EXEC('
DROP TABLE [DATABASE_NAME].[dbo].[Example_Table]
'
) AT [LINKED_SERVER_NAME]
--Rename the new table back to the old table name
-- NOTE the lack of database and owner identifiers in the new name
-- NOTE the use of double single qoutes (ESCAPED single quotes)
EXEC('USE [DATABASE_NAME];
EXEC sp_rename ''[DATABASE_NAME].[dbo].Example_Table_temp'',''Example_Table''
'
) AT [LINKED_SERVER_NAME]
-- Now do your IDENTITY INSERTs !!!!
INSERT INTO [LINKED_SERVER_NAME].[DATABASE_NAME].[dbo].[Example_Table] (ID,[Name]) VALUES (888,'Travis')
INSERT INTO [LINKED_SERVER_NAME].[DATABASE_NAME].[dbo].[Example_Table] (ID,[Name]) VALUES (999,'Mike')
--Verify they got put in
SELECT * FROM [LINKED_SERVER_NAME].[DATABASE_NAME].[dbo].[Example_Table]
--Now let's switch it back to our SCHEMA with an IDENTITY
EXEC('
CREATE TABLE [DATABASE_NAME].[dbo].[Example_Table_temp](
[ID] [int] IDENTITY(1,1) NOT NULL,
[Name] [nchar](10) NULL
)
ALTER TABLE [DATABASE_NAME].[dbo].[Example_Table] SWITCH TO [DATABASE_NAME].[dbo].[Example_Table_temp]
DROP TABLE [DATABASE_NAME].[dbo].[Example_Table]
EXEC sp_rename ''[DATABASE_NAME].[dbo].Example_Table_temp'',''Example_Table''
'
) AT [LINKED_SERVER_NAME]
--Data is still there
SELECT * FROM [LINKED_SERVER_NAME].[DATABASE_NAME].[dbo].[Example_Table]
GO
-- And note you can no longer INSERT the IDENTITY
INSERT INTO [LINKED_SERVER_NAME].[DATABASE_NAME].[dbo].[Example_Table] (ID,[Name]) VALUES (45,'Travis')
GO
You need to execute dynamic query Example :
exec [LINKSERVERNAME].[DATABASENAME].[SCHEMANAME].sp_executesql N'Your Query'
If any column set identity the you need to set SET IDENTITY_INSERT TargetTable ON and need to specified the column name. Example:
SET IDENTITY_INSERT TargetTable ON;
INSERT INTO TargetTable(Col1, Col2, Col3)
SELECT Col1, Col2, Col3 FROM SourceTable;
SET IDENTITY_INSERT TargetTable OFF;
I created few user defined types in my database as below
CREATE TYPE [dbo].[StringID] FROM [nvarchar](20) NOT NULL
and assigned them to various tables. The tables in my database are in various schemas (not only dbo)
But I realized I need bigger field, and I need to alter, e.g increase from [nvarchar](20) to [nvarchar](50), but there is no ALTER TYPE statement.
I need a script that uses a temp table/cursor whatever and saves all the tables and fields where my type is used. Then change existing fields to base type - e.g. from CustID [StringID] to CustID [nvarchar(20)].
Drop the user type and recreate it with new type - e.g. nvarchar(50)
and finally set back fields to user type
I do not have rules defined on types, so don't have to drop rules and re-add them.
Any help is appreciated.
This is what I normally use, albeit a bit manual:
/* Add a 'temporary' UDDT with the new definition */
exec sp_addtype t_myudt_tmp, 'numeric(18,5)', NULL
/* Build a command to alter all the existing columns - cut and
** paste the output, then run it */
select 'alter table dbo.' + TABLE_NAME +
' alter column ' + COLUMN_NAME + ' t_myudt_tmp'
from INFORMATION_SCHEMA.COLUMNS
where DOMAIN_NAME = 't_myudt'
/* Remove the old UDDT */
exec sp_droptype t_mydut
/* Rename the 'temporary' UDDT to the correct name */
exec sp_rename 't_myudt_tmp', 't_myudt', 'USERDATATYPE'
We are using the following procedure, it allows us to re-create a type from scratch, which is "a start". It renames the existing type, creates the type, recompiles stored procs and then drops the old type. This takes care of scenarios where simply dropping the old type-definition fails due to references to that type.
Usage Example:
exec RECREATE_TYPE #schema='dbo', #typ_nme='typ_foo', #sql='AS TABLE([bar] varchar(10) NOT NULL)'
Code:
CREATE PROCEDURE [dbo].[RECREATE_TYPE]
#schema VARCHAR(100), -- the schema name for the existing type
#typ_nme VARCHAR(128), -- the type-name (without schema name)
#sql VARCHAR(MAX) -- the SQL to create a type WITHOUT the "CREATE TYPE schema.typename" part
AS DECLARE
#scid BIGINT,
#typ_id BIGINT,
#temp_nme VARCHAR(1000),
#msg VARCHAR(200)
BEGIN
-- find the existing type by schema and name
SELECT #scid = [SCHEMA_ID] FROM sys.schemas WHERE UPPER(name) = UPPER(#schema);
IF (#scid IS NULL) BEGIN
SET #msg = 'Schema ''' + #schema + ''' not found.';
RAISERROR (#msg, 1, 0);
END;
SELECT #typ_id = system_type_id FROM sys.types WHERE UPPER(name) = UPPER(#typ_nme);
SET #temp_nme = #typ_nme + '_rcrt'; -- temporary name for the existing type
-- if the type-to-be-recreated actually exists, then rename it (give it a temporary name)
-- if it doesn't exist, then that's OK, too.
IF (#typ_id IS NOT NULL) BEGIN
exec sp_rename #objname=#typ_nme, #newname= #temp_nme, #objtype='USERDATATYPE'
END;
-- now create the new type
SET #sql = 'CREATE TYPE ' + #schema + '.' + #typ_nme + ' ' + #sql;
exec sp_sqlexec #sql;
-- if we are RE-creating a type (as opposed to just creating a brand-spanking-new type)...
IF (#typ_id IS NOT NULL) BEGIN
exec recompile_prog; -- then recompile all stored procs (that may have used the type)
exec sp_droptype #typename=#temp_nme; -- and drop the temporary type which is now no longer referenced
END;
END
GO
CREATE PROCEDURE [dbo].[recompile_prog]
AS
BEGIN
SET NOCOUNT ON;
DECLARE #v TABLE (RecID INT IDENTITY(1,1), spname sysname)
-- retrieve the list of stored procedures
INSERT INTO
#v(spname)
SELECT
'[' + s.[name] + '].[' + items.name + ']'
FROM
(SELECT sp.name, sp.schema_id, sp.is_ms_shipped FROM sys.procedures sp UNION SELECT so.name, so.SCHEMA_ID, so.is_ms_shipped FROM sys.objects so WHERE so.type_desc LIKE '%FUNCTION%') items
INNER JOIN sys.schemas s ON s.schema_id = items.schema_id
WHERE is_ms_shipped = 0;
-- counter variables
DECLARE #cnt INT, #Tot INT;
SELECT #cnt = 1;
SELECT #Tot = COUNT(*) FROM #v;
DECLARE #spname sysname
-- start the loop
WHILE #Cnt <= #Tot BEGIN
SELECT #spname = spname
FROM #v
WHERE RecID = #Cnt;
--PRINT 'refreshing...' + #spname
BEGIN TRY -- refresh the stored procedure
EXEC sp_refreshsqlmodule #spname
END TRY
BEGIN CATCH
PRINT 'Validation failed for : ' + #spname + ', Error:' + ERROR_MESSAGE();
END CATCH
SET #Cnt = #cnt + 1;
END;
END
there's a good example of a more comprehensive script here
It's worth noting that this script will include views if you have any. I ran it and instead of exec'ing inline generated a script as the output which I then tweaked and ran.
Also, if you have functions/sprocs using the user defeined types you'll need to drop those before running your script.
Lesson Learned: in future, don't bother with UDTs they're more hassle than they're worth.
SET NOCOUNT ON
DECLARE #udt VARCHAR(150)
DECLARE #udtschema VARCHAR(150)
DECLARE #newudtschema VARCHAR(150)
DECLARE #newudtDataType VARCHAR(150)
DECLARE #newudtDataSize smallint
DECLARE #OtherParameter VARCHAR(50)
SET #udt = 'Name' -- Existing UDDT
SET #udtschema = 'dbo' -- Schema of the UDDT
SET #newudtDataType = 'varchar' -- Data type for te new UDDT
SET #newudtDataSize = 500 -- Lenght of the new UDDT
SET #newudtschema = 'dbo' -- Schema of the new UDDT
SET #OtherParameter = ' NULL' -- Other parameters like NULL , NOT NULL
DECLARE #Datatype VARCHAR(50),
#Datasize SMALLINT
DECLARE #varcharDataType VARCHAR(50)
DECLARE #Schemaname VARCHAR(50),
#TableName VARCHAR(50),
#FiledName VARCHAR(50)
CREATE TABLE #udtflds
(
Schemaname VARCHAR(50),
TableName VARCHAR(50),
FiledName VARCHAR(50)
)
SELECT TOP 1
#Datatype = Data_type,
#Datasize = character_maximum_length
FROM INFORMATION_SCHEMA.COLUMNS
WHERE Domain_name = #udt
AND Domain_schema = #udtschema
SET #varcharDataType = #Datatype
IF #DataType Like '%char%'
AND #Datasize IS NOT NULL
AND ( #newudtDataType <> 'varchar(max)'
OR #newudtDataType <> 'nvarchar(max)'
)
BEGIN
SET #varcharDataType = #varcharDataType + '('
+ CAST(#Datasize AS VARCHAR(50)) + ')'
END
INSERT INTO #udtflds
SELECT TABLE_SCHEMA,
TABLE_NAME,
Column_Name
FROM INFORMATION_SCHEMA.COLUMNS
WHERE Domain_name = #udt
AND Domain_schema = #udtschema
DECLARE #exec VARCHAR(500)
DECLARE alter_cursor CURSOR
FOR SELECT Schemaname,
TableName,
FiledName
FROM #udtflds
OPEN alter_cursor
FETCH NEXT FROM alter_cursor INTO #Schemaname, #TableName, #FiledName
WHILE ##FETCH_STATUS = 0
BEGIN
SET #exec = 'Alter Table ' + #Schemaname + '.' + #TableName
+ ' ALTER COLUMN ' + #FiledName + ' ' + #varcharDataType
EXECUTE ( #exec
)
FETCH NEXT FROM alter_cursor INTO #Schemaname, #TableName, #FiledName
END
CLOSE alter_cursor
SET #exec = 'DROP TYPE [' + #udtschema + '].[' + #udt + ']'
EXEC ( #exec
)
SET #varcharDataType = #newudtDataType
IF #newudtDataType Like '%char%'
AND #newudtDataSize IS NOT NULL
AND ( #newudtDataType <> 'varchar(max)'
OR #newudtDataType <> 'nvarchar(max)'
)
BEGIN
SET #varcharDataType = #varcharDataType + '('
+ CAST(#newudtDataSize AS VARCHAR(50)) + ')'
END
SET #exec = 'CREATE TYPE [' + #newudtschema + '].[' + #udt + '] FROM '
+ #varcharDataType + ' ' + #OtherParameter
EXEC ( #exec
)
OPEN alter_cursor
FETCH NEXT FROM alter_cursor INTO #Schemaname, #TableName, #FiledName
WHILE ##FETCH_STATUS = 0
BEGIN
SET #exec = 'Alter Table ' + #Schemaname + '.' + #TableName
+ ' ALTER COLUMN ' + #FiledName + ' ' + '[' + #newudtschema
+ '].[' + #udt + ']'
EXECUTE ( #exec
)
FETCH NEXT FROM alter_cursor INTO #Schemaname, #TableName, #FiledName
END
CLOSE alter_cursor
DEALLOCATE alter_cursor
SELECT *
FROM #udtflds
DROP TABLE #udtflds
1: http://www.sql-server-performance.com/2008/how-to-alter-a-uddt/ has replaced http://www.sql-server-performance.com/faq/How_to_alter_a%20_UDDT_p1.aspx
The simplest way to do this is through Visual Studio's object explorer, which is also supported in the Community edition.
Once you have made a connection to SQL server, browse to the type, right click and select View Code, make your changes to the schema of the user defined type and click update. Visual Studio should show you all of the dependencies for that object and generate scripts to update the type and recompile dependencies.
As devio says there is no way to simply edit a UDT if it's in use.
A work-round through SMS that worked for me was to generate a create script and make the appropriate changes; rename the existing UDT; run the create script; recompile the related sprocs and drop the renamed version.
The solutions provided here can only be applied if the user defined types are used in table definitions only, and if the UDT columns are not indexed.
Some developers also have SP's and functions using UDT parameters, which is not covered either. (see comments on Robin's link and in the Connect entry)
The Connect entry from 2007 has finally been closed after 3 years:
Thank you for submitting this
suggestion, but given its priority
relative to the many other items in
our queue, it is unlikely that we will
actually complete it. As such, we are
closing this suggestion as “won’t
fix”.
I tried to solve a similiar problem ALTERing XML SCHEMA COLLECTIONS, and the steps seem to mostly apply to ALTER TYPE, too:
To drop a UDT, the following steps are necessary:
If a table column references the UDT, it has to be converted to the underlying type
If the table column has a default constraint, drop the default constraint
If a procedure or function has UDT parameters, the procedure or function has to be dropped
If there is an index on a UDT column, the index has to be dropped
If the index is a primary key, all foreign keys have to be dropped
If there are computed columns based on a UDT column, the computed columns have to be dropped
If there are indexes on these computed columns, the indexes have to be dropped
If there are schema-bound views, functions, or procedures based on tables containing UDT columns, these objects have to be dropped
I ran into this issue with custom types in stored procedures, and solved it with the script below. I didn't fully understand the scripts above, and I follow the rule of "if you don't know what it does, don't do it".
In a nutshell, I rename the old type, and create a new one with the original type name. Then, I tell SQL Server to refresh its details about each stored procedure using the custom type. You have to do this, as everything is still "compiled" with reference to the old type, even with the rename. In this case, the type I needed to change was "PrizeType". I hope this helps. I'm looking for feedback, too, so I learn :)
Note that you may need to go to Programmability > Types > [Appropriate User Type] and delete the object. I found that DROP TYPE doesn't appear to always drop the type even after using the statement.
/* Rename the UDDT you want to replace to another name */
exec sp_rename 'PrizeType', 'PrizeTypeOld', 'USERDATATYPE';
/* Add the updated UDDT with the new definition */
CREATE TYPE [dbo].[PrizeType] AS TABLE(
[Type] [nvarchar](50) NOT NULL,
[Description] [nvarchar](max) NOT NULL,
[ImageUrl] [varchar](max) NULL
);
/* We need to force stored procedures to refresh with the new type... let's take care of that. */
/* Get a cursor over a list of all the stored procedures that may use this and refresh them */
declare sprocs cursor
local static read_only forward_only
for
select specific_name from information_schema.routines where routine_type = 'PROCEDURE'
declare #sprocName varchar(max)
open sprocs
fetch next from sprocs into #sprocName
while ##fetch_status = 0
begin
print 'Updating ' + #sprocName;
exec sp_refreshsqlmodule #sprocName
fetch next from sprocs into #sprocName
end
close sprocs
deallocate sprocs
/* Drop the old type, now that everything's been re-assigned; must do this last */
drop type PrizeTypeOld;
New answer to an old question:
Visual Studio Database Projects handle the drop and recreate process when you deploy changes. It will drop stored procs that use UDDTs and then recreate them after dropping and recreating the data type.
1.Rename the old UDT,
2.Execute query ,
3.Drop the old UDT.
Simple DROP TYPE first then CREATE TYPE again with corrections/alterations?
There is a simple test to see if it is defined before you drop it ... much like a table, proc or function -- if I wasn't at work I would look what that is?
(I only skimmed above too ... if I read it wrong I apologise in advance! ;)