I'm trying to figure out how to create a looped truncate to truncate (or delete) the data in specific tables only. There are something like 30 or so tables that need to be truncated, and I want to avoid just using a list of truncate statements. I can't seem to find any good examples of doing this, and no matter what I try, I get a "right truncated" error.
Example 1:
FOR anlyc_tables AS curs CURSOR FOR
SELECT table_name FROM systable WHERE table_name LIKE 'table_to_truncate_prefix%'
DO EXECUTE (
'TRUNCATE TABLE ' + table_name
);
END FOR;
This one throws no errors, but completes in .078 seconds and doesn't actually truncate anything.
Example 2:
ALTER PROCEDURE truncate_analytics()
BEGIN
DECLARE #table_name VARCHAR;
DECLARE curs DYNAMIC SCROLL CURSOR FOR SELECT table_name FROM systable WHERE table_name LIKE 't_anlyc%';
OPEN curs WITH HOLD;
FETCH NEXT curs INTO #table_name;
WHILE(sqlstate = 0) LOOP
FETCH NEXT curs INTO #table_name;
TRUNCATE TABLE table_name;
END LOOP;
END
GO
CALL truncate_analytics()
GO
Results in the "right truncated" error and the tables are not truncated.
I think I'm missing something really obvious here, but I don't have a ton of experience with SQL scripting in this manner, and can't seem to find any working examples of this to prove it's even possible.
Can anyone point me in the right direction?
There may be a few things at work that are preventing the code from working.
With your second example, the text "TRUNCATE TABLE table_name" is going to look for a table named table_name. table_name won't be treated for it's variable value, but will be considered a object identifier. FWIW, I'm not sure how this would cause a right truncation error.
Depending on what you consider the prefix for a table, you might not be getting the list of tables you expect. If you are considering the user to be the prefix (i.e. dba.my_table), your select isn't going to grab the necessary tables; there can be multiple tables with the same name, just different owners. If you want to select based on user, you'll need to join to the SYSUSER view.
Lastly, using a cursor for this can be done. It would require WITH HOLD as you already have, because the TRUNCATE TABLE statement issues an implicit commit and closes cursors otherwise. (Note that WITH HOLD requires explicit closing of the cursor, otherwise it will remain open until connection close.) Another option is to build a string of statements to execute, and use the execute or EXECUTE IMMEDIATE on them as a batch.
Hope This Helps,
Tyson
Related
I've only been writing DB2 procedures for a few days, but trying to do a "batch delete" on a given table. My expected logic is:
to open a cursor
walk through it until EOF
issue a DELETE on each iteration
For sake of simplifying this question, assume I only want to issue a single COMMIT (of all DELETEs), after the WHILE loop is completed (ie. once cursor reaches EOF). So given the code sample below:
CREATE TABLE tableA (colA INTEGER, ...)
CREATE PROCEDURE "SCHEMA"."PURGE_PROC"
(IN batchSize INTEGER)
LANGUAGE SQL
SPECIFIC SQL140207163731500
BEGIN
DECLARE tempID INTEGER;
DECLARE eof_bool INTEGER DEFAULT 0;
DECLARE sqlString VARCHAR(1000);
DECLARE sqlStmt STATEMENT;
DECLARE myCurs CURSOR WITH HOLD FOR sqlStmt;
DECLARE CONTINUE HANDLER FOR SQLSTATE '02000' SET eof_bool = 1;
SET sqlString = 'select colA from TableA';
PREPARE sqlStmt FROM sqlString;
OPEN myCurs;
FETCH myCurs INTO tempID;
WHILE (eof_bool = 0) DO
DELETE FROM TableA where colA = tempID;
FETCH myCurs INTO tempID;
END WHILE;
COMMIT;
CLOSE myCurs;
END
Note: In my real scenario:
I am not deleting all records from the table, just certain ones based on some additional criteria; and
I plan to perform a COMMIT every N# iterations of the WHILE loop (say 500 or 1000), not the entire mess like above; and
I plan to DELETE against multiple tables, not just this one;
But again, to simplify, I tested the above code, and what I'm seeing is that the DELETEs seem to be getting committed 1-by-1. I base this on the following test:
I pre-load the table with (say 50k) records;
then run the purge storedProc which takes ~60 secs to run;
during this time, from another sql client, I continuously "SELECT COUNT(*) FROM tableA" and see count reducing incrementally.
If all DELETEs were committed at once, I would expect to see the record count(*) only drop from to 0 at the end of the ~60 seconds. That is what I see with comparable SPs written for Oracle or SQLServer.
This is DB2 v9.5 on Win2003.
Any ideas what I'm missing?
You are missing the difference in concurrency control implementation between the different database engines. In an Oracle database another session would see data that have been committed prior to the beginning of its transaction, that is, it would not see any deletes until the first session commits.
In DB2, depending on the server configuration parameters (e.g. DB2_SKIPDELETED) and/or the second session isolation level (e.g. uncommitted read) it can in fact see (or not see) data affected by in-flight transactions.
If your business logic requires different transaction isolation, speak with your DBA.
It should be pointed out that you're deleting "outside of the cursor"
The right way to delete using the cursor would be using a "positioned delete"
DELETE FROM tableA WHERE CURRENT OF myCurs;
The above deletes the row just fetched.
I'm trying to create a stored procedure (or a a trigger, function, anything) to check if all the tables in a database is accessible.
My idea is to get all the tables in database, and then try to acess them with a simple select, if this succeeds for all the tables, everything is supposed to be ok.
I couldn't think of anything else to solve this problem, but I don't know how to do this, or all this intead.
1 - to get all the tables name I did:
SELECT RDB$RELATION_NAME TABLE
FROM RDB$RELATIONS
WHERE RDB$VIEW_BLR IS NULL
AND (RDB$SYSTEM_FLAG IS NULL OR RDB$SYSTEM_FLAG = 0)
ORDER BY TABLE
Now I just need to create the SELECT statement to each table, and run a query:
SELECT FIRST 1 * FROM [TABLE];
while it's ok, it continues, if all the tables are accessible, my database is ok.
Can anybody help me with this? Is this the correct aproach to solve this problem?
As a_horse_with_no_name commented, this is really strange request... if you see the table in the RDB$RELATIONS you can be pretty sure the table exists in the database. If the table is listed in the DB metadata but actually doesn't exist then the DB is corrupted and your idea to use select to check it's "accessibilty" is pointless... Also, the table might be there but the user might not have select right for it, IOW you need to take the user rights into account too.
Anyway, you can use the EXECUTE STATEMENT to execute dynamically built DSQL statement, something like
declare stmt varchar(1024);
declare ctab varchar(31);
BEGIN
FOR SELECT RDB$RELATION_NAME
FROM RDB$RELATIONS
WHERE RDB$VIEW_BLR IS NULL AND (RDB$SYSTEM_FLAG IS NULL OR RDB$SYSTEM_FLAG = 0)
INTO :ctab DO BEGIN
stmt = 'select ... from ' || ctab;
execute statement stmt;
END;
END
To check is the database corrupted you should use the gfix utility with -validate option.
The issue I'm facing is that I have a stored procedure (lets call it sp_one), which during it's run calls another stored procedure (lets call it sp_two).
I'd only like the resultset from sp_one to be returned at the end, and not those from sp_two. I imagine there is a way to capture the results from sp_two that will prevent them from also being returned but haven't been able to figure the syntax for this.
Any ideas?
Some pseudo code which captures the essence of what is going on (not my actual code):
CREATE PROCEDURE sp_two AS
BEGIN
update Users
set is_valid = 0
select * from Users
END
CREATE PROCEDURE sp_one
AS
BEGIN
exec sp_two
select * from Users
END
exec sp_one
The result of running exec sp_one is the resultset from sp_two, then the results from sp_one. (eg. the users table twice).
First of all, here is a similiar question
I don't recommand to use this kind of solution, because it could be a bottleneck easily. I would say you should focus on to make the dataprocessing on a much clearer way (however I understand that your question's example is just a theoretical example)
But if you really want to use something like this I would say measure the danger of the returning rows:
1: How many rows returns?
2: How wide is the returning set?
And if you think "ok it is not a big deal", then I would say use a memory table instead of temp table (do not make physical writes):
DECLARE #users TABLE (...fields here...)
INSERT INTO #users
EXEC sp_two
In sp_one, you can use
CREATE TABLE #temporaryusers (Usertable fields here)
INSERT INTO #temporaryusers
EXEC sp_two
DROP TABLE #temporaryusers
to swallow your results.
I'm trying to find a way to do a conditional DELETE on an InnoDB table which contains millions of records, without locking it (thus not bringing the website down).
I've tried to find information on mysql.com, but to no avail. Any tips on how to proceed?
I don't think it is possible to delete without locking. That said, I don't think locking the record you want to delete is a problem. What would be a problem is locking other rows.
I found some information on that subject here: http://dev.mysql.com/doc/refman/5.0/en/innodb-locks-set.html
What I would suggest, is to try and do a million single row deletes. I think that if you do all those in a single transaction, performance should not hurt too much. so you would get something like:
START TRANSACTION;
DELETE FROM tab WHERE id = 1;
..
..
DELETE FROM tab WHERE id = x;
COMMIT;
You can generate the required statments by doing something like
SELECT CONCAT('DELETE FROM tab WHERE id = ', id)
FROM tab
WHERE <some intricate condition that selects the set you want to delete>
So the advantage over this method instead of doing:
DELETE FROM tab
WHERE <some intricate condition that selects the set you want to delete>
is that in the first approach you only ever lock the record you're deleting, whereas in the second approach you could run the risk of locking other records that happen to be in the same range as the rows you are deleteing.
If it fits your application, then you could limit the number of rows to delete, and setup a cronjob for repeating the deletion. E.g.:
DELETE FROM tab WHERE .. LIMIT 1000
I found this to be good compromise in a similar scenario.
I use procedure to delete
create procedure delete_last_year_data()
begin
DECLARE del_row varchar(255);
DECLARE done INT DEFAULT 0;
declare del_rows cursor for select CONCAT('DELETE FROM table_name WHERE id = ', id)
from table_name
where created_time < '2018-01-01 00:00:00';
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
open del_rows;
repeat
fetch del_rows into del_row;
if not done
then
set #del = del_row;
prepare stmt from #del;
execute stmt;
DEALLOCATE PREPARE stmt;
end if;
until done end repeat;
close del_rows;
end //
I'm after a simple stored procedure to drop tables. Here's my first attempt:
CREATE PROC bsp_susf_DeleteTable (#TableName char)
AS
IF EXISTS (SELECT name FROM sysobjects WHERE name = #TableName)
BEGIN
DROP TABLE #TableName
END
When I parse this in MS Query Analyser I get the following error:
Server: Msg 170, Level 15, State 1, Procedure bsp_susf_DeleteTable, Line 6
Line 6: Incorrect syntax near '#TableName'.
Which kind of makes sense because the normal SQL for a single table would be:
IF EXISTS (SELECT name FROM sysobjects WHERE name = 'tbl_XYZ')
BEGIN
DROP TABLE tbl_XYZ
END
Note the first instance of tbl_XYZ (in the WHERE clause) has single quotes around it, while the second instance in the DROP statement does not. If I use a variable (#TableName) then I don't get to make this distinction.
So can a stored procedure be created to do this? Or do I have to copy the IF EXISTS ... everywhere?
You should be able to use dynamic sql:
declare #sql varchar(max)
if exists (select name from sysobjects where name = #TableName)
BEGIN
set #sql = 'drop table ' + #TableName
exec(#sql)
END
Hope this helps.
Update: Yes, you could make #sql smaller, this was just a quick example. Also note other comments about SQL Injection Attacks
Personally I would be very wary of doing this. If you feel you need it for administrative purposes, please make sure the rights to execute this are extremely limited. Further, I would have the proc copy the table name and the date and the user executing it to a logging table. That way at least you will know who dropped the wrong table. You may want other protections as well. For instance you may want to specify certain tables that cannot be dropped ever using this proc.
Further this will not work on all tables in all cases. You cannot drop a table that has a foreign key associated with it.
Under no circumstances would I allow a user or anyone not the database admin to execute this proc. If you havea a system design where users can drop tables, there is most likely something drastically wrong with your design and it should be rethought.
Also, do not use this proc unless you have a really, really good backup schedule in place and experience restoring from backups.
You'll have to use EXEC to execute that query as a string. In other words, when you pass in the table name, define a varchar and assign the query and tablename, then exec the variable you created.
Edit: HOWEVER, I don't recommend that because someone could pass in sql rather than a TableName and cause all kinds of wonderful problems. See Sql injection for more information.
Your best bet is to create a parameterized query on the client side for this. For example, in C# I would do something like:
// EDIT 2: on second thought, ignore this code; it probably won't work
SqlCommand sc = new SqlCommand();
sc.Connection = someConnection;
sc.CommandType = Command.Text;
sc.CommandText = "drop table #tablename";
sc.Parameters.AddWithValue("#tablename", "the_table_name");
sc.ExecuteNonQuery();