Hope you can help me with the issue below.
DETAILS: I am trying to create a stored procedure for my Azure Data Factory project where I can pass a variable table name through to tell SQL Server to delete rows.
This all is part of an ERP tool with transactional data. I would delete rows that are in the target, but changed in the source.
Example: rowid 1 - €100 in the target but changes in the source later to €80. Then I need to delete that row with €100 in the target and copy the new value from the source with the same rowid to the target
I'm migrating from a Oracle environment to a Microsoft environment. In Oracle I've this SQL for the stored procedure:
create or replace procedure AAAAA_delete
as
cursor c_del
is
select 'delete from ' ||table_name ||' a where not exists (select 1 from ' ||replace(table_name,'AAAAA','BBBBB') ||' b where a.rowid = b.rowid)' deletions
from all_tables
where table_name like 'AAAAA%'
and table_name not like 'AAAAA_LOAD%'
and table_name not in ('AAAAA_TIME','AAAAA_CONFIGURATION')
order by 1;
begin
for r_del in c_del
loop
execute immediate r_del.deletions;
commit;
end loop;
end;
** the names have been changed to AAAA and BBBB on purpose to publish the code **
I'm wondering how i can transform this into MS SQL.
WHY?: My whole ETL process in Azure Data Factory is based on variables. All the table names etc are stored in a metadata table. And it depends on +- 70 tables.
QUESTION: Is it possible to create a generic stored procedure to drop rows depending on the name I pass through.
Yes your code can be adapted for MS SQL;
create procedure AAAAA_delete AS
DECLARE #VARA NVARCHAR(100);
DECLARE #EXEC NVARCHAR(1000);
DECLARE CURSER CURSOR
LOCAL STATIC READ_ONLY FORWARD_ONLY
FOR
SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES
where table_name like 'AAAAA%'
and table_name not like 'AAAAA_LOAD%'
and table_name not in ('AAAAA_TIME','AAAAA_CONFIGURATION')
order by 1
OPEN CURSER
FETCH NEXT FROM CURSER INTO #VARA
WHILE ##FETCH_STATUS = 0
BEGIN
SET #EXEC = N'delete from ' +#VARA +' a where not exists (select 1 from ' + replace(#VARA,'AAAAA','BBBBB') +' b where a.rowid = b.rowid)'
EXEC(#EXEC)
FETCH NEXT FROM CURSER INTO #VARA
END
CLOSE CURSER
DEALLOCATE CURSER
Related
We have production database above 100Gig. I want to duplicate this database and give it to every developer to test its code, but the size is too large. Is there anyway I can backup just top 1000 rows with FK's and restore it to new DB? Or duplicate the DB first and delete all records from all tables, but keep 1000 rows with FK's or any other way to keep size below 5Gig.
I did search, but none of solutions were for tables having foreign keys.
Thanks,
Basheer
This the IDEA:
First:
Create new database:
Second:
Select small records only like:
select top 500 from allYourTables
then insert to each every table to your new Database Created.
Third:
Dump the new database and give to its every developer
Hope it helps:
Assuming that you have a new db_to_dev Database name and you are working to your current database:
This procedure will insert all the data from your working database, making sure that you had already a database created. db_to_dev:
Using Information_Schema you can select all your tables:
CREATE PROCEDURE PROC_TRANSFER_DATA #NUM_OF_RECORDS nvarchar(255) as
BEGIN
SET NOCOUNT ON;
DECLARE #message varchar(80), #tablename nvarchar(50);
Declare #sqlstmt nvarchar(255);
PRINT '-------- List of tables --------';
DECLARE Schema_cursor CURSOR FOR
SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES
OPEN Schema_cursor
FETCH NEXT FROM Schema_cursor INTO #tablename
IF ##FETCH_STATUS <> 0
PRINT ' <<None>>'
WHILE ##FETCH_STATUS = 0
BEGIN
SELECT #message = ' ' + #tablename
set #sqlstmt = 'select top ' + #NUM_OF_RECORDS + ' * into [db_to_dev].[dbo].['+ #tablename +'] from ' + #tablename;
EXEC sp_executesql #sqlstmt
PRINT #message
FETCH NEXT FROM Schema_cursor INTO #tablename
END
CLOSE Schema_cursor
DEALLOCATE Schema_cursor
END
To use:
With an option parameter:
EXEC PROC_TRANSFER_DATA '500'
Parameter value is depend on you if how many records you want to transfer into your new database db_to_dev.
This Stored Proc is tested.
Good luck
There are a number of projects on github which seek to do exactly that: make a subset that preserves referential integrity. Here is one such project:
https://github.com/18F/rdbms-subsetter
The db being accessed is on Snowflake; not certain on the storage details behind the scenes.
I have a query right now that creates a new view from 41 data tables stored in separate schemas under the same database, looks something like this:
CREATE VIEW all_data AS
SELECT * FROM db.schema1.data UNION ALL
SELECT * FROM db.schema2.data UNION ALL
SELECT * FROM db.schema3.data
This query is run daily. My issue is I get new data tables added every few days and I have to go manually edit the query to include those new tables, as they're stored under separate schemas (and the naming scheme for the schemas isn't consistent either, for reasons outside my control). Is there a way I can select all the schemas inside a database with a subquery that would allow me to run the query daily without needing manual updates when new schemas + tables are added?
I'd like the resulting query to have a structure somewhat like
CREATE VIEW all_data as
SELECT * FROM [SELECT schemas from db].data
but not sure how that would work, and how to union the resulting data correctly.
Unfortunately, in Snowflake you can't dynamically construct SQL statements (yet). You can of course do what you want to achieve via a script in one of the supported languages (e.g. Python, JS), by first finding all the schemas and then constructing a full SQL statement.
Hope this helps.
You can definitely query the table and schema list available. SQL Authority has a good article on it:
http://blog.sqlauthority.com/2009/06/17/sql-server-list-schema-name-and-table-name-for-database/
In short the query winds up being along these lines to pull the list of tables and schema:
SELECT '['+SCHEMA_NAME(schema_id)+'].['+name+']'
AS SchemaTable
FROM sys.tables
Though you will have to add a database name to the where clause to point to the proper DB.
With the release of Snowflake Scripting dynamic recreation of the view inside Snowflake is now very possible.
create database dynamic_views;
create schema dynamic_views.schema_base;
create schema dynamic_views.schema1;
create table dynamic_views.schema1.data(id int) as select * from values (1);
We can use the INFORMATION_SCEMA.TABLES to find all DATA tables:
SELECT table_schema
FROM dynamic_views.information_schema.tables
WHERE table_name = 'DATA';
TABLE_SCHEMA
SCHEMA1
and now push that into a cursor and build up a view creation SQL
This SQL needs to be run in the new Snowsight Console see (Working with Classic Console):
declare
sql text;
add_union boolean := false;
c1 cursor for SELECT TABLE_SCHEMA
FROM dynamic_views.information_schema.TABLES
WHERE TABLE_NAME = 'DATA';
begin
sql := 'CREATE OR REPLACE VIEW dynamic_views.schema_base.all_data AS ';
for record in c1 do
if (add_union) then
sql := sql || 'UNION ALL ';
end if;
sql := sql || 'SELECT * FROM dynamic_views.'|| record.TABLE_SCHEMA ||'.data ';
add_union := true;
end for;
EXECUTE IMMEDIATE sql;
return sql;
end;
;
and we can use it:
select * from dynamic_views.schema_base.all_data;
ID
1
and add more:
create schema dynamic_views.schema2;
create table dynamic_views.schema2.data(id int) as select * from values (2);
rebuild:
anonymous block
CREATE OR REPLACE VIEW dynamic_views.schema_base.all_data AS SELECT * FROM dynamic_views.SCHEMA1.data UNION ALL SELECT * FROM dynamic_views.SCHEMA2.data
use it again:
select * from dynamic_views.schema_base.all_data;
ID
1
2
Note: You should not use SELECT * in production as the order of the table columns will be dependent of the create orders, and if newer tables have a different shape you view will become invalid.
So the explicit form really should be used:
'SELECT column1, column2, column4 FROM dynamic_views.'|| record.TABLE_SCHEMA ||'.data ';
for anyone who want the solution for this question this is my Idea with useing FETCH
Declare #str nvarchar(maX)
Declare #i int
Set #i =(Select max(id ) from Clinics );
set #str='';
declare #Id int
declare cur CURSOR LOCAL for
select [Id] from [dbo].[Clinics]
GROUP BY [Id]
open cur
fetch next from cur into #Id
while ##FETCH_STATUS = 0 BEGIN
if #i>#id
begin
set #str=#str+ 'sELECT '+ LTRIM(RTRIM(Convert(varchar(6),#Id))) + ',* fROM ' + quotename(LTRIM(RTRIM(CONVERT(VARCHAR(8),#Id))))+'.[Clinic_Benefits] UNION ALL ';
end
else
begin
set #str=#str+ 'sELECT '+ LTRIM(RTRIM(Convert(varchar(6),#Id))) + ',* fROM ' + quotename(LTRIM(RTRIM(CONVERT(VARCHAR(8),#Id))))+'.[Clinic_Benefits] ';
end
fetch next from cur into #Id
END
close cur
deallocate cur
print #str;
exec (#str);
I am using SQL Server 2008. My question is, is it possible, given a table name, to construct a query that returns a list of databases that contain that table WITH values (meaning it is not empty)?
For example, I have a table called tbl_Name. I have 100 databases, and in 90 of them tbl_Name is empty. Can I get a list of the 10 databases where tbl_Name has values?
You can do this with a cursor, and the stored procedure sp_msforeachdb.
sp_msforeachdb is, as the name suggests, a proc that runs something for each database. You can use this to list all db's with a given table name:
sp_msforeachdb 'SELECT "?" AS db from [?].sys.tables WHERE name = ''tbl_Name'''
Inserting those records into a temp table makes it easy to iterate over in a cursor:
DROP TABLE #db_List
DROP TABLE #Not_Empty
GO
CREATE TABLE #db_List (db VARCHAR(MAX))
CREATE TABLE #Not_Empty (db VARCHAR(MAX))
GO
sp_msforeachdb 'INSERT INTO #db_List SELECT "?" AS db from [?].sys.tables WHERE name = ''tbl_Name'''
GO
SET NOCOUNT ON
DECLARE #Iterator varchar(MAX)
,#strSQL varchar(MAX)
DECLARE xyz CURSOR
FOR
--Select stuff to iterate over
SELECT *
FROM #db_List
OPEN xyz
FETCH NEXT FROM xyz
INTO #Iterator
WHILE ##FETCH_STATUS = 0
BEGIN
--Do stuff
SET #strSQL = 'INSERT INTO #Not_Empty
SELECT '''+#Iterator+'''
WHERE EXISTS (SELECT TOP 1 * FROM '+#Iterator+'..tbl_Name)
'
Exec (#strSQL)
FETCH NEXT FROM xyz
INTO #Iterator
END
CLOSE xyz
DEALLOCATE xyz
GO
SET NOCOUNT OFF
Quick summary:
Start in master DB select from sys.databases, get all the DB's, cursor thru each one (USE the DB). Use EXEC( sqlStatement ) to perform your command etc...
Now that you're in the DB (Each DB has a table named sys.objects, so your goal from the above paragraph is to simply issue a command like EXEC(USE dbName) ) list through the list of tables, select from sys.objects, look for object types of 'U' (user table). Get the name, construct your SELECT * FROM INTO #yourTemp WHERE blah blah blah insert your results in a #temp table you created at the very beginning.. Issue the SQL via EXEC()
Maybe there's a better way about this but this has to be somewhat dynamic.
From a vb.net form I need to restore or replace data from one table to another. The two tables are identical except for a couple different columns.
First I wrote some SQL to grab the column names of the table passed in. Then through ordinal position I get only the tables I want values from. I store these tables names in a temp table.
Now I want to get those values from the backup table using the temp table column names and place them in the master table.
So I guess I suppose I need a cursor to loop through in some way.. I haven't touched a cursor since college and wow.
I'll embarrass myself and post my current code.
SET #getColCURSOR = CURSOR FOR
SELECT name
FROM #MyTempTable --created previously as table only holding column names
OPEN #getColCURSOR
FETCH NEXT FROM #getColCURSOR
INTO #columnName
WHILE ##FETCH_STATUS = 0
BEGIN
select #columnName --this variable should as a column name and change
from AUDIT_TABLE a where a.ID = 7 -- 7 is just for testing is dynamic variable
FETCH NEXT FROM #getColCURSOR
INTO #columnName
END
CLOSE #getColCURSOR
DEALLOCATE #getColCURSOR
I'm not going to comment on whether this could be done without a cursor, since I'm a bit lost on what you're trying to do. But one issue with your cursor is that you can't parameterize a column name in a select statement. So you'll need to replace this:
WHILE ##FETCH_STATUS = 0
BEGIN
select #columnName --this variable should as a column name and change
from AUDIT_TABLE a where a.ID = 7 -- 7 is just for testing is dynamic variable
FETCH NEXT FROM getColCURSOR
INTO #columnName
END
--with dynamic SQL like this:
DECLARE #SQL nvarchar(max)
WHILE ##FETCH_STATUS = 0
BEGIN
set #SQL =
N'select ' + QUOTENAME(#columnName) + ' from AUDIT_TABLE a where a.ID = 7'
EXEC (#SQL); --w/o brackets assumes you've calling a stored proc
FETCH NEXT FROM getColCURSOR
INTO #columnName
END
That could possibly introduce other issues, since dynamic SQL statements execute in their own scope. I'd definitely encourage you to look into whether there's a set-based solution to this, since using dynamic SQL will make this even messier, and I don't think you'll be able to escape dynamic SQL if you want to use a variable for column names.
Is there any way to truncate all tables from a specific MySQL database name without using any other language than SQL? I mean no linux shell scripts. (why? because it will run on windows, MacOSX and linux servers).
the problem is that the client its selecting the database name from a list in a control panel webpage (wich will be displaying MySQL databases from different servers *nix and windows), and then he will want to truncate all the tables inside that database (yes that is the main task of the web form).
Alex
Ok, I solved it by myself here is the stored procedure :)
BEGIN
DECLARE done BOOLEAN DEFAULT FALSE;
DECLARE truncatestmnt TEXT; -- this is where the truncate statement will be retrieved from cursor
-- This is the magic query that will bring all the table names from the database
DECLARE c1 CURSOR FOR SELECT Concat('TRUNCATE TABLE ', TABLE_NAME) FROM INFORMATION_SCHEMA.TABLES WHERE INFORMATION_SCHEMA.TABLES.TABLE_SCHEMA = "#DatabaseName";
DECLARE CONTINUE HANDLER FOR SQLSTATE '02000' SET done = TRUE;
OPEN c1;
c1_loop: LOOP
FETCH c1 INTO truncatestmnt;
IF `done` THEN LEAVE c1_loop; END IF;
SET #x = truncatestmnt;
PREPARE stm1 FROM #x;
EXECUTE stm1;
END LOOP c1_loop;
CLOSE c1;
END
What I am making its calling all tables from the given database, this will help if the tables inside the given database have no pattern to follow.
So by calling DECLARE c1 CURSOR FOR SELECT Concat('TRUNCATE TABLE ', TABLE_NAME) FROM INFORMATION_SCHEMA.TABLES WHERE INFORMATION_SCHEMA.TABLES.TABLE_SCHEMA = "#DatabaseName"; and saving results into a cursor I can fetch all the TRUNCATE TABLE x statements generated by the "n" quantity of tables inside the given database, then by just preparing and executing each statement in the cursor it will truncate all the tables inside the given database.
BTW #DatabaseName must be given as parameter to the stored procedure
Hope this helps someone else too :)
Alex
create procedure drop_tables_like(pattern varchar(255), db varchar(255))
begin
select #str_sql:=concat('drop table ', group_concat(table_name))
from information_schema.tables
where table_schema=db and table_name like pattern;
prepare stmt from #str_sql;
execute stmt;
drop prepare stmt;
end
then call
call drop_tables_like('%', 'dababase_name')