I need to create non-temporary tables in a MariaDB 10.3 database using Node. I therefore need a way of generating a table name that is guaranteed to be unique.
The Node function cannot access information regarding any unique feature about what or when the tables are made, so I cannot build the name from a timestamp or connection ID. I can only verify the name's uniqueness using the current database.
This question had a PostgreSQL answer suggesting the following:
SET #name = GetBigRandomNumber();
WHILE TableExists(#name)
BEGIN
SET #name = GetBigRandomNumber();
END
I attempted a MariaDB implementation using #name = CONCAT(MD5(RAND()),MD5(RAND())) to generate a random 64 character string, and (COUNT(*) FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME LIKE #name) >0 to check if it was a unique name:
SET #name = CONCAT(MD5(RAND()),MD5(RAND()));
WHILE ((COUNT(*) FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME LIKE #name) >0) DO
SET #name = CONCAT(MD5(RAND()),MD5(RAND()));
END WHILE;
CREATE TABLE #name ( ... );
However I get a syntax error when I try to run the above query. My SQL knowledge isn't that great so I'm at a loss as to what the problem might be.
Furthermore, is this approach efficient? The randomly generated name is long enough that it is very unlikely to have any clashes with any current table in the database, so the WHILE loop will very rarely need to run, but is there some sort of built in function to auto increment table names, or something similar?
SET #name := UUID();
If the dashes in that cause trouble, then
SET #name := REPLACE(UUID(), '-', '');
It will be safer (toward uniqueness) than RAND(). And, in theory, there will be no need to verify its uniqueness. After all, that's the purpose of UUIDs.
Related
I am struggling with this. I have looked at Table Level Variables but I am thinking this is way beyond my simple understanding at this stage of SQL.
The issue I have created is I have an array of ID values I am generating inside MS Access as a result of some other tasks in there. I am wanting to send these over to SQL Server to grab the jobs with the ID number that matches.
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [dbo].[get_Job]
#jobID VARCHAR,
#JobIDs id_List READONLY
AS
BEGIN
SELECT #JobID AS JobID;
SELECT *
FROM Job
END;
Is my current stored procedure, however whilst I have been able to get it to return the JobID variable any list I added generates an error. If I insert only 1 ID into JobIDs, this doesn't generate a result either.
As I said I think I am punching well above my weight and am getting a bit lost in all this. Perhaps I can be directed to a better training resource or a site that explains this in baby steps or a book I can purchase to help me understand this? I would appreciate help with fixing the errors above but a fish teaching is probably better.
Thanks in advance
The issue comes down to much is how long is the list of ID's you going to pass to t-sql is the issue?
You could take the passed list (assume it is a string), say like this from Access at a PT query
exec GetHotels '1,2,3,4,5,6,7,10,20,30'
So, the above is the PT query you can/could send to sql server from Access.
So, in above, we want to return records based on above?
The T-SQL would thus become:
CREATE PROCEDURE GetHotels
#IdList nvarchar(max)
AS
BEGIN
SET NOCOUNT ON;
DECLARE #MySQL nvarchar(max)
set #MySQL = 'select * from tblHotels where ID in (' + #IdList + ')'
EXECUTE sp_executesql #mysql
END
GO
Now, in Access, say you have that array of "ID" ? You code will look like this:
Sub MyListQuery(MyList() As String)
' above assumes a array of id
' take array - convert to a string list
Dim strMyList As String
strMyList = "'" & Join(MyList, ",") & "'"
Dim rst As DAO.Recordset
With CurrentDb.QueryDefs("qryPassR")
.SQL = "GetHotels " & strMyList
Set rst = .OpenRecordset
End With
rst.MoveLast
Debug.Print rst.RecordCount
End Sub
Unfortunately, creating t-sql on the fly is a "less" then ideal approach. In most cases, because the table is not known at runtime, you have to specific add EXEC permissions to the user.
eg:
GRANT EXECUTE ON dbo.GetHotels TO USERTEST3
You find that such users can execute + run + use "most" store procedures, but in this case, you have to add specific rights with above grant due to the "table" not being known or resolved until runtime.
So, the above is a way to send a "given" array that you have, but from a general permissions point of view, and that of creating t-sql on the fly - I can't recommend this approach unless you are stuck, and have no other choice.
Edit
Here is a solution that works the same as above, but we don't have to create a SQL statement as a string.
CREATE PROCEDURE [dbo].[GetHotels2]
#IdList nvarchar(max)
AS
BEGIN
SET NOCOUNT ON;
-- create a table from the passed list
declare #List table (ID int)
while charindex(',',#IdList) > 0
begin
insert into #List (ID) values(left(#IDList,charindex(',',#IdList)-1))
set #Idlist = right(#IdList,len(#IdList)-charindex(',',#IdList))
end
insert into #List (ID) values(#IdList)
select * from tblHotels where ID in (select ID from #list)
END
You didn't show us what that table-valued parameter looks like - but assuming id_List contains a column called Id, then you need to join this TVP to your base table something like this:
ALTER PROCEDURE [dbo].[get_Job]
#jobID VARCHAR,
#JobIDs id_List READONLY
AS
BEGIN
SELECT (list of columns)
FROM Job j
INNER JOIN id_List l ON j.JobId = l.Id;
END;
Seems pretty easy to me - and not really all that difficult to handle! Agree?
Also, check out Bad habits to kick : declaring VARCHAR without (length) - you should always provide a length for any varchar variables and parameters that you use. Otherwise, as in your case - that #jobID VARCHAR parameter will be exactly ONE character long - and this is typically not what you expect / want ....
Let's say I have a table Managers with fields Id and Name. There is another table Accounts that has fields Id and Name. These two tables have their relationships defined in a many to many table ManagedAccounts which has a composite key of ManagerId and AccountId. So you can have multiple managers on a certain account, but there can't be the same manager on the account multiple times.
Now, I have a stored procedure called MergeAccounts that takes in a Manager Id and a list of Manager Ids in the form of a comma delimited varchar. It currently looks a lot like this:
create procedure MergeAccounts #managerId nvarchar(12), #mergedManagers nvarchar(max) as declare #reassignment nvarchar(max)
set #reassignment='update ManagedAccounts set ManagerId='+#managerId+' where ManagerId in ('+#mergedManagers+')'
exec sp_executesql #reassignment
Since two managers could be on the same account, it'll give me an error saying that I've violated the compound key I have on that table. How do I need to structure my code to simply delete any redundant rows without regards to order?
Change your dynamic SQL to delete any potential collisions first. Then do your update. Wrap it all in a transaction.
(BTW, I would avoid using dynamic SQL altogether by creating a table-valued function that returns a table from a comma-separated list... this is very useful and you can probably find a function like that already written if you google it)
set #reassignment='
BEGIN TRAN;
BEGIN TRY
DELETE m1
FROM ManagedAccounts m1
JOIN ManagedAccounts m2 ON m1.AccountId = m2.AccountId
WHERE m2.ManagerId = ' + #managerId + '
AND m1.ManagerId IN (' + #mergedAccounts + ')
UPDATE ManagedAccounts SET ManagerId=' + #managerId + ' WHERE ManagerId IN (' + #mergedManagers + ')
COMMIT;
END TRY
BEGIN CATCH
ROLLBACK;
END CATCH;';
I'm new to T-SQL; all my experience is in a completely different database environment (Openedge). I've learned enough to write the procedure below -- but also enough to know that I don't know enough!
This routine will have to go into a live environment soon, and it works, but I'm quite certain there are a number of c**k-ups and gotchas in it that I know nothing about.
The routine copies data from table A to table B, replacing the data in table B. The tables could be in any database. I plan to call this routine multiple times from another stored procedure. Permissions aren't a problem: the routine will be run by the dba as a timed job.
Could I have your suggestions as to how to make it fit best-practice? To bullet-proof it?
ALTER PROCEDURE [dbo].[copyTable2Table]
#sdb varchar(30),
#stable varchar(30),
#tdb varchar(30),
#ttable varchar(30),
#raiseerror bit = 1,
#debug bit = 0
as
begin
set nocount on
declare #source varchar(65)
declare #target varchar(65)
declare #dropstmt varchar(100)
declare #insstmt varchar(100)
declare #ErrMsg nvarchar(4000)
declare #ErrSeverity int
set #source = '[' + #sdb + '].[dbo].[' + #stable + ']'
set #target = '[' + #tdb + '].[dbo].[' + #ttable + ']'
set #dropStmt = 'drop table ' + #target
set #insStmt = 'select * into ' + #target + ' from ' + #source
set #errMsg = ''
set #errSeverity = 0
if #debug = 1
print('Drop:' + #dropStmt + ' Insert:' + #insStmt)
-- drop the target table, copy the source table to the target
begin try
begin transaction
exec(#dropStmt)
exec(#insStmt)
commit
end try
begin catch
if ##trancount > 0
rollback
select #errMsg = error_message(),
#errSeverity = error_severity()
end catch
-- update the log table
insert into HHG_system.dbo.copyaudit
(copytime, copyuser, source, target, errmsg, errseverity)
values( getdate(), user_name(user_id()), #source, #target, #errMsg, #errSeverity)
if #debug = 1
print ( 'Message:' + #errMsg + ' Severity:' + convert(Char, #errSeverity) )
-- handle errors, return value
if #errMsg <> ''
begin
if #raiseError = 1
raiserror(#errMsg, #errSeverity, 1)
return 1
end
return 0
END
Thanks!
I'm speaking from a Sybase perspective here (I'm not sure if you're using SQLServer or Sybase) but I suspect you'll find the same issues in either environment, so here goes...
Firstly, I'd echo the comments made in earlier answers about the assumed dbo ownership of the tables.
Then I'd check with your DBAs that this stored proc will be granted permissions to drop tables in any database other than tempdb. In my experience, DBAs hate this and rarely provide it as an option due to the potential for disaster.
DDL operations like drop table are only allowed in a transaction if the database has been configured with the option sp_dboption my_database, "ddl in tran", true. Generally speaking, things done inside transactions involving DDL should be very short since they will lock up the frequently referenced system tables like sysobjects and in doing so, block the progress of other dataserver processes. Given that we've no way of knowing how much data needs to be copied, it could end up being a very long transaction which locks things up for everyone for a while. What's more, the DBAs will need to run that command on every database which contains tables that might contain a '#Target' table of this stored proc. If you were to use a transaction for the drop table it'd be a good idea to make it separate from any transaction handling the data insertion.
While you can do drop table commands in a transaction if the ddl in tran option is set, it's not possible to do select * into inside a transaction. Since select * into is a combination of table creation with insert, it would implicitly lock up the database (possibly for a while if there's a lot of data) if it were executed in a transaction.
If there are foreign key constraints on your #target table, you won't be able to just drop it without getting rid of the foreign key constraints first.
If you've got an 'id' column which relies upon a numeric identity type (often used as an autonumber feature to generate values for surrogate primary keys), be aware that you won't be able to copy the values from the '#Source' table's 'id' column across to the '#Target' table's id column.
I'd also check the size of your transaction log in any possible database which might hold a '#Target' table in relation to the size of any possible '#Source' table. Given that all the copying is being done in a single transaction, you may well find yourself copying a table so large that it blows out the transaction log in your prod dataserver, bringing all processes to a crashing halt. I've seen people using chunking to achieve this over particularly large tables, but then you end up needing to put your own checks into the code to make sure that you've actually captured a consistent snapshot of the table.
Just a thought - if this is being used to get snapshots, how about BCP? That could be used to dump out the contents of the table giving you the snapshot you're looking for. If you use the -c option you'd even get it in a human readable form.
All the best,
Stuart
This line seems a bit dangerous:
set #dropStmt = 'drop table ' + #target
What if the target table doesn't exist?
I'd try to safeguard that somehow - something like:
set #dropStmt =
'if object_id(' + #target + ') IS NOT NULL DROP TABLE ' + #target
That way, only if the call to OBJECT_ID(tablename) doesn't return NULL (that means: table doesn't exist) and the table is guaranteed to exist, issue the DROP TABLE statement.
Firstly, replace all the code like
set #source = '[' + #sdb + '].[dbo].[' + #stable + ']'
with code like
set #source = QuoteName(#sdb) + '.[dbo].' + QuoteName(#stable)
Secondly, your procedure assumes all objects are owned by dbo - this may not be the case.
Thirdly, your variable names are too short at 30 characters - 128 is the length of sysname.
I find the whole process you wrote to be terribly dangerous. Even if this is running from the database and not by the user, dynamic SQL is a poor practice! In databases using this to be able to do this to any table anytime is dangerous and would out and out be forbidden in the databases I work with. It is way too easy to accidentally drop the wrong tables! Nor is it possible to correctly test all possible values that the sp could run with, so this could be buggy code as well and you won't know until it has been in production.
Further, in dropping and recreating with select into, you must not have indexes or feoriegn key constraints or the things you need to havefor performance and data integrity. BAD BAD IDEA in general (OK if these are just staging tables of some type but not for anything else).
This is a task for SSIS. We save our SSIS packages and commit them to Subversion just like everything else. We can do a diff on them (they are just XML files) and we can tell what is running on prod and what configuration we are using.
You should not drop and recreate tables unless they are relatively small. You should update existing records, delete records no longer needed, and only add new ones. If you havea million records and 27000 have changed, 10 have been deleted, and 3000 are new, why drop and insert all 1,000,000 records. It is wasteful of server resources, could cause locking and blocking issues, and could create issues if the users are looking at the tables at the time you run this and the data suddenly disappears and takes some minutes to come back. Users get cranky about that.
Brief history:
I'm writing a stored procedure to support a legacy reporting system (using SQL Server Reporting Services 2000) on a legacy web application.
In keeping with the original implementation style, each report has a dedicated stored procedure in the database that performs all the querying necessary to return a "final" dataset that can be rendered simply by the report server.
Due to the business requirements of this report, the returned dataset has an unknown number of columns (it depends on the user who executes the report, but may have 4-30 columns).
Throughout the stored procedure, I keep a column UserID to track the user's ID to perform additional querying. At the end, however, I do something like this:
UPDATE #result
SET Name = ppl.LastName + ', ' + ppl.FirstName
FROM #result r
LEFT JOIN Users u ON u.id = r.userID
LEFT JOIN People ppl ON ppl.id = u.PersonID
ALTER TABLE #result
DROP COLUMN [UserID]
SELECT * FROM #result r ORDER BY Name
Effectively I set the Name varchar column (that was previously left NULL while I was performing some pivot logic) to the desired name format in plain text.
When finished, I want to drop the UserID column as the report user shouldn't see this.
Finally, the data set returned has one column for the username, and an arbitrary number of INT columns with performance totals. For this reason, I can't simply exclude the UserID column since SQL doesn't support "SELECT * EXCEPT [UserID]" or the like.
With this known (any style pointers are appreciated but not central to this problem), here's the problem:
When I execute this stored procedure, I get an execution error:
Invalid column name 'userID'.
However, if I comment out my DROP COLUMN statement and retain the UserID, the stored procedure performs correctly.
What's going on? It certainly looks like the statements are executing out of order and it's dropping the column before I can use it to set the name strings!
[Edit 1]
I defined UserID previously (the whole stored procedure is about 200 lies of mostly irrelevant logic, so I'll paste snippets:
CREATE TABLE #result ([Name] NVARCHAR(256), [UserID] INT);
Case sensitivity isn't the problem but did point me to the right line - there was one place in which I had userID instead of UserID. Now that I fixed the case, the error message complains about UserID.
My "broken" stored procedure also works properly in SQL Server 2008 - this is either a 2000 bug or I'm severely misunderstanding how SQL Server used to work.
Thanks everyone for chiming in!
For anyone searching this in the future, I've added an extremely crude workaround to be 2000-compatible until we update our production version:
DECLARE #workaroundTableName NVARCHAR(256), #workaroundQuery NVARCHAR(2000)
SET #workaroundQuery = 'SELECT [Name]';
DECLARE cur_workaround CURSOR FOR
SELECT COLUMN_NAME FROM [tempdb].INFORMATION_SCHEMA.Columns WHERE TABLE_NAME LIKE '#result%' AND COLUMN_NAME <> 'UserID'
OPEN cur_workaround;
FETCH NEXT FROM cur_workaround INTO #workaroundTableName
WHILE ##FETCH_STATUS = 0
BEGIN
SET #workaroundQuery = #workaroundQuery + ',[' + #workaroundTableName + ']'
FETCH NEXT FROM cur_workaround INTO #workaroundTableName
END
CLOSE cur_workaround;
DEALLOCATE cur_workaround;
SET #workaroundQuery = #workaroundQuery + ' FROM #result ORDER BY Name ASC'
EXEC(#workaroundQuery);
Thanks everyone!
A much easier solution would be to not drop the column, but don't return it in the final select.
There are all sorts of reasons why you shouldn't be returning select * from your procedure anyway.
EDIT: I see now that you have to do it this way because of an unknown number of columns.
Based on the error message, is the database case sensitive, and so there's a difference between userID and UserID?
This works for me:
CREATE TABLE #temp_t
(
myInt int,
myUser varchar(100)
)
INSERT INTO #temp_t(myInt, myUser) VALUES(1, 'Jon1')
INSERT INTO #temp_t(myInt, myUser) VALUES(2, 'Jon2')
INSERT INTO #temp_t(myInt, myUser) VALUES(3, 'Jon3')
INSERT INTO #temp_t(myInt, myUser) VALUES(4, 'Jon4')
ALTER TABLE #temp_t
DROP Column myUser
SELECT * FROM #temp_t
DROP TABLE #temp_t
It says invalid column for you. Did you check the spelling and ensure there even exists that column in your temp table.
You might try wrapping everything preceding the DROP COLUMN in a BEGIN...COMMIT transaction.
At compile time, SQL Server is probably expanding the * into the full list of columns. Thus, at run time, SQL Server executes "SELECT UserID, Name, LastName, FirstName, ..." instead of "SELECT *". Dynamically assembling the final SELECT into a string and then EXECing it at the end of the stored procedure may be the way to go.
Is there a clean way of cloning a record in SQL that has an index(auto increment). I want to clone all the fields except the index. I currently have to enumerate every field, and use that in an insert select, and I would rather not explicitly list all of the fields, as they may change over time.
Not unless you want to get into dynamic SQL. Since you wrote "clean", I'll assume not.
Edit: Since he asked for a dynamic SQL example, I'll take a stab at it. I'm not connected to any databases at the moment, so this is off the top of my head and will almost certainly need revision. But hopefully it captures the spirit of things:
-- Get list of columns in table
SELECT INTO #t
EXEC sp_columns #table_name = N'TargetTable'
-- Create a comma-delimited string excluding the identity column
DECLARE #cols varchar(MAX)
SELECT #cols = COALESCE(#cols+',' ,'') + COLUMN_NAME FROM #t WHERE COLUMN_NAME <> 'id'
-- Construct dynamic SQL statement
DECLARE #sql varchar(MAX)
SET #sql = 'INSERT INTO TargetTable (' + #cols + ') ' +
'SELECT ' + #cols + ' FROM TargetTable WHERE SomeCondition'
PRINT #sql -- for debugging
EXEC(#sql)
There's no easy and clean way that I can think of off the top of my head, but from a few items in your question I'd be concerned about your underlying architecture. Maybe you have an absolutely legitimate reason for wanting to do this, but usually you want to try to avoid duplicates in a database, not make them easier to cause. Also, explicitly naming columns is usually a good idea. If you're linking to outside code, it makes sure that you don't break that link when you add a new column. If you're not (and it sounds like you probably aren't in this scenario) I still prefer to have the columns listed out because it forces me to review the effects of the change/new column - even if it's just to look at the code and decide that adding the new column is not a problem.
DROP TABLE #tmp_MyTable
SELECT * INTO #tmp_MyTable
FROM MyTable
WHERE MyIndentID = 165
ALTER TABLE #tmp_MyTable
DROP Column MyIndentID
INSERT INTO MyTable
SELECT *
FROM #tmp_MyTable
This also deals with a unique key projectnum as well as the primary key.
CREATE TEMPORARY TABLE projecttemp SELECT * FROM project WHERE projectid='6';
ALTER TABLE projecttemp DROP COLUMN projectid;
UPDATE projecttemp SET projectnum = CONCAT(projectnum, ' CLONED');
INSERT INTO project SELECT NULL,projecttemp.* FROM projecttemp;
You could create an insert trigger to do this, however, you would lose the ability to do an insert with an explicit ID. It would, instead, always use the value from the sequence.
You could create a trigger to do it for you. To make sure that trigger only works for cloning, you could create a separate username CLONE and log in with it. Or, even better, if your DBMS supports it, create a role named CLONE and any user can log in using that role and do the cloning. The trigger code would be something like:
if (CURRENT_ROLE = 'CLONE') then
new.ID = assign new id from generator/sequence
Of course, you would grant that role only to the users who are allowed to clone records.