Find all ERDs containing table with specified name - sql

Is it possible that I can query for names of all ERDs (Entity Relationship Diagram) which contain table with name
Like '%mytable%'
Something like this:
select *
from <ERD objects>
where tableName like '%%'
Actually, I have a large Database with lots of ERDs. So, to understand the scope of a table I want to browse ERDs of that specific table.

As I understand you need a list of tables contained in a database diagrams. There you can help this article. I will add part of it here:
The diagram itself is stored in a binary field. And you can not convert it into readable form without difficulties. For deserializing this field (called definition) we need 2 functions:
CREATE FUNCTION [dbo].[Tool_VarbinaryToVarchar_Text]
(
#VarbinaryValue VARBINARY(max),
#bitASCIIOnly BIT = 0
)
RETURNS VARCHAR(max) AS
BEGIN
DECLARE #NumberOfBytes INT
SET #NumberOfBytes = DATALENGTH(#VarbinaryValue)
-- PART ONE --
IF (#NumberOfBytes > 4)
BEGIN
DECLARE #FirstHalfNumberOfBytes INT
DECLARE #SecondHalfNumberOfBytes INT
SET #FirstHalfNumberOfBytes = #NumberOfBytes/2
SET #SecondHalfNumberOfBytes = #NumberOfBytes - #FirstHalfNumberOfBytes
-- Call this function recursively with the two parts of the input split in half
RETURN dbo.Tool_VarbinaryToVarchar_Text(CAST(SUBSTRING(#VarbinaryValue, 1 , #FirstHalfNumberOfBytes) AS VARBINARY(max)),#bitASCIIOnly)
+ dbo.Tool_VarbinaryToVarchar_Text(CAST(SUBSTRING(#VarbinaryValue, #FirstHalfNumberOfBytes+1 , #SecondHalfNumberOfBytes) AS VARBINARY(max)),#bitASCIIOnly)
END
IF (#NumberOfBytes = 0)
BEGIN
RETURN '' -- No bytes found, therefore no 'hex string' is returned
END
-- PART TWO --
DECLARE #HighByte INT
-- #NumberOfBytes <= 4 (four or less characters/8 hex digits were input)
-- eg. 88887777 66665555 44443333 22221111
-- We'll process ONLY the right-most (least-significant) Byte, which consists
-- of eight bits
-- 2. Carve off the rightmost eight bits/single hex digit (ie 22221111)
-- Divide by 16 does a shift-left (now processing 2222)
SET #HighByte = CAST(#VarbinaryValue AS INT) & 255
IF #bitASCIIOnly = 1 AND (#HighByte < 32 OR #HighByte > 126) SET #HighByte=13;
-- 3. Trim the byte (two hex values) from the right (least significant) input Binary
-- in preparation for further parsing
SET #VarbinaryValue = SUBSTRING(#VarbinaryValue, 1, (#NumberOfBytes-1))
-- 4. Recursively call this method on the remaining Binary data, concatenating the text
-- 'value' we just decoded as their ASCII character representation
-- ie. we pass 88887777 66665555 44443333 back to this function, adding X to the result string
RETURN dbo.Tool_VarbinaryToVarchar_Text(#VarbinaryValue,#bitASCIIOnly) +
CHAR(#HighByte)
END
And:
CREATE FUNCTION [dbo].[fnTool_ScriptDiagram2005_Text]()
RETURNS
#tblOut TABLE
(
-- Add the column definitions for the TABLE variable here
diagramname NVARCHAR(128),
diagram_id INT PRIMARY KEY,
diagram_text VARCHAR(MAX),
diagram_ASCII VARCHAR(MAX)
)
AS
BEGIN
DECLARE #name NVARCHAR(128);
DECLARE #diagram_id INT;
DECLARE #index INT;
DECLARE #size INT;
DECLARE #chunk INT;
DECLARE #line VARCHAR(MAX);
DECLARE #lineASC VARCHAR(MAX);
DECLARE #CurrentPos INT;
SELECT #CurrentPos = MIN(diagram_id) FROM dbo.sysdiagrams;
WHILE (#CurrentPos IS NOT NULL)
BEGIN
-- Set start index, and chunk 'constant' value
SET #index = 1; --
SET #chunk = 32; -- values that work: 2, 6
-- values that fail: 15,16, 64
SELECT #diagram_id = diagram_id,
#size = DATALENGTH(definition),
#name = name
FROM dbo.sysdiagrams
WHERE diagram_id = #CurrentPos;
-- Now with the diagram_id, do all the work
SET #line = '';
SET #lineASC = '';
WHILE #index < #size
BEGIN
-- Output as many UPDATE statements as required to append all the diagram binary
-- data, represented as hexadecimal strings
SELECT #line = #line + dbo.Tool_VarbinaryToVarchar_Text(SUBSTRING (definition, #index, #chunk),0),
#lineASC = #lineASC + dbo.Tool_VarbinaryToVarchar_Text(SUBSTRING (definition, #index, #chunk),1)
FROM dbo.sysdiagrams
WHERE diagram_id = #CurrentPos;
SET #index = #index + #chunk;
END
INSERT INTO #tblOut (diagramname, diagram_id, diagram_text, diagram_ASCII)
VALUES (#name, #diagram_id, #line, REPLACE(#lineASC,CHAR(13),''));
SELECT #CurrentPos = MIN(diagram_id)
FROM dbo.sysdiagrams
WHERE diagram_id > #CurrentPos;
END
RETURN;
END
After that you can run:
SELECT *
FROM [dbo].[fnTool_ScriptDiagram2005_Text] ()
WHERE diagram_ASCII LIKE '%TableToFind%'
For example I have created diagram TestDiagram with 2 tables named whatever and IE_Stat. In return query:
Root EntrypfoBCompObj_ !"#$%&'()*+,-.123456789:(}5n]4o[\0V?[?i???V?[?i??T,,,4") -bH''Uu94941#xV4XdboIE_StatMicrosoft DDS Form 2.0Embedded Object9q&sch_labels_visibled(ActiveTableViewMode1 TableViewMode:0:4,0,28DdsStreamSchema UDV Default&/DSREF-SCHEMA-CONTENTS,0Schema UDV Default Post V66;4,0,2310,1,1890,5,1260 TableViewMode:12,0,284,0,2805 TableViewMode:22,0,284,0,2310 TableViewMode:32,0,284,0,2310 TableViewMode:4>4,0,284,0,2310,12,2730,11,1680(ActiveTableViewMode1 TableViewMode:0:4,0,284,0,2310,1,1890,5,1260 TableViewMode:12,0,284,0,2805 TableViewMode:22,0,284,0,2310 TableViewMode:32,0,284,0,2310 TableViewMode:4>4,0,284,0,2310,12,2730,11,1680NaQW9 LHEData Source=********;Initial Catalog=test;Integrated Security=True;MultipleActiveResultSets=False;TrustServerCertificate=True;Packet Size=4096;Application Name="Microsoft SQL Server Management Studio"TestDiagram&whateverdbo$IE_StatdbokE7d2pN{1634CDD7-0888-42E3-9FA2-B6D32563B91D}bR
you can see both table names.

Related

Related to string manipulation

I have a table of crop which contain a column with crop name For Example Fruit and Nuts, Vegetables and Melons. I have to use T-SQL and remove the 'and' insert the spaces with a underscore and a text crop.groups should be added in front. For example Fruit and Nuts will become crop.groups.fruit_nuts. I have to to do these for all the crops together in SQL. When we run the query a separate column with these transformation should be generated. Also I have to create a function doing all this to me.
My Function:
CREATE function dbo.translationkey_v10
(
#column_name nchar(15)
)
returns nvarchar(1000)
as
begin
DECLARE #numletters int;
DECLARE #counter int;
DECLARE #str nvarchar(6);
DECLARE #newcolumn_name nvarchar(50)
SET #numletters = LEN(#column_name);
SET #counter = 1;
SET #newcolumn_name = '';
WHILE #counter <= #numletters
BEGIN
-- a. read next character:
----------------------------------------------------------------
SET #str = LOWER(SUBSTRING(#column_name, #counter, 1));
-- b. search for and in the string 'and':
----------------------------------------------------------------
IF (#str LIKE '%and%')
BEGIN
SET #str = REPLACE(#str,' and','')
END
-- c. check for space:
----------------------------------------------------------------
IF UNICODE(#str) = 32
BEGIN
SET #str = '_';
END
SET #newcolumn_name = #newcolumn_name + #str;
SET #counter = #counter + 1;
END
RETURN CONCAT('crop.groups.',#newcolumn_name)
END
Sample data
Crop name : Beverages and spice crops
Translation output : crop.group.beverages_spice_crops
This would seem very simple using replace:
select cropname, Concat('crop.groups.', Replace(cropname, ' and ','_'))
from crop

data length issue from SQL Server to Oracle with non english characher

we have 2 applications. One application uses SQL Server as the backend and the other application uses Oracle.
In the first application the user can enter some information and the 2nd application gets the data from SQL Server and insert it into oracle.
The problem is that the user can enter in any language following table shows sample data
Table in SQL Server
For instance user has entered Chinese characters in address field and length is 10,
Oracle Table
Address is not inserted here because length of address exceeds to 12, in oracle special character considering as 3 length.
I want to substring character (with non english and with english). How can I achieve that? I have written function which written number of special character.
how to get only 5 charachters from #nstring
Try to define the column in Oracle as VARCHAR2(10 CHAR). That changes the length semantics from bytes to characters. So the column will be able to accept 10 characters not just 10 bytes, which might be to short if there are special characters in the string.
declare #nstring NVARCHAR(MAX)=N'理,551'
declare #lenSQL int = len(#nstring)
declare #oracleLen int = #lenSQL +(2 * [dbo].[CountNonEnglishfromString](#nstring))
declare #OracleMaxLen int = 5; -- change as per len required
declare #newString nvarchar(max);
if(#OracleMaxLen < #oracleLen)
begin
declare #olen int =0
declare #count int =1;
WHILE ( #count <= #lenSQL)
BEGIN
declare #ch nvarchar(1) =(SELECT SUBSTRING(#nstring,#count,#count) AS ExtractString);
declare #isSpecialChar int = [dbo].[CountNonEnglishfromString](#ch)
if(#isSpecialChar = 1)
begin
set #olen = #olen+3;
end
else
begin
set #olen = #olen+1;
end
if(#OracleMaxLen < #olen)
begin
break
end
set #newString =CONCAT(#newString , #ch)
set #count = #count +1
End
End
else
begin
set #newString = #nstring
end
select isnull(#newString,'') as 'new string';

Scalar Variable not Populating

Here's the story. I'm trying to pull metadata from a master table and from a series of tables whose names are based on values in the master table. There is no foreign key.
If there was a key, it is that the primary key from from the master table is appended to the end of the child table. The master table is hsi.keytypetable. The child tables are hsi.keyitemxxx where the xxx is a value (keytypenum) pulled from the master table.
All I'm trying to pull from the child table right now is a count of values. In the current form, the query, #sql1, is failing to populate #keytypenum, although when I look at query itself, and run it in a separate window, it works like a champ. The problem continues in the second query, #sql2, where I am getting the error,
Must declare the scalar variable "#keytypenum"
As far as I can tell, I've declared the thing. I'm guessing I have a similar problem with syntax in each query.
SET CONCAT_NULL_YIELDS_NULL OFF
declare #keytypedata table
(
Keyword varchar(50),
DateType varchar(50),
"Length" int,
"Count" int
)
declare #keywordcount int
declare #x int = 1
declare #keytypenum int
declare #sql1 varchar(max)
declare #sql2 varchar(max)
/* Determine how many records are in the master table so that I can cycle thru each one, getting the count of the child tables. */
Select #keywordcount = (Select count(*) from hsi.keytypetable)
/* #x is the counter. I'll cycle through each row in the master using a WHILE loop */
WHILE #x < #keywordcount+1
BEGIN
/* One row at a time, I'll pull the KEYTYPENUM and store it in #keytypenum. (I don't really need the order by, but I like having things in order!)
** I take the rows in order b using my counter, #x, as the offset value and fetch only 1 row at a time. When I run this query in a separate screen,
** it works well, obviously with providing a fixed offset value. */
set #sql1 =
'Set #keytypenum =
(Select
KEYTYPENUM
from hsi.keytypetable
order by KEYTYPENUM
OFFSET ' + cast(#x as varchar(4)) + ' ROWS
FETCH NEXT 1 ROWS ONLY)'
EXEC(#sql1)
/* For debugging purposes, I wanted to see that #keytypenum got assigned. This is working. */
print 'KeyTypeNum: '+cast(#keytypenum as varchar(4))
/* I don't know why I had to be my table variable, #keytypedata, in single quotes at the beginning, but it wouldn't work if
** I didn't. The problem comes later on with restricting the query by the aforementioned #keytypenum. Remember this variable is an INT. All values
** for this field are indeed integers, and there are presently 955 rows in the table. The maximum value is 1012, so we're safe with varchar(4).
*/
SET #sql2 =
'Insert into ' + '#keytypedata' + '
Select
keytype,
CASE
WHEN k.datatype = 1 THEN ''Numeric 20''
WHEN k.datatype = 2 THEN ''Dual Table Alpha''
WHEN k.datatype = 3 THEN ''Currency''
WHEN k.datatype = 4 THEN ''Date''
WHEN k.datatype = 5 THEN ''Float''
WHEN k.datatype = 6 THEN ''Numeric 9''
WHEN k.datatype = 9 THEN ''DateTime''
WHEN k.datatype = 10 THEN ''Single Table Alpha''
WHEN k.datatype = 11 THEN ''Specific Currency''
WHEN k.datatype = 12 THEN ''Mixed Case Dual Table Alpha''
WHEN k.datatype = 13 THEN ''Mixed Case Single Table Alpha''
END,
keytypelen,
(Select count(*) from hsi.keyitem' + cast(#keytypenum as varchar(4)) + ')
FROM
hsi.keytypetable k
where
k.keytypenum = ' + cast(#keytypenum as varchar(4))+''
/* Printing out where I am with cycling thru the master table, just for troubleshooting*/
print #x
/* Increment the counter*/
set #x = #x + 1
END
/* This query is simply to display the final results. */
select *
from #keytypedata
order by 1
/* Print statements below are for troubleshooting. They should show what the 2 queries currently look like. */
Print #sql1
Print #sql2
Yeah, you can't do that. Your variables are going out of scope.
Each variable has it's scope restricted to it's own session, and exec() basically creates a new session.
This will throw and error that #x is undefined:
declare #x int = 1
exec ('select #x')
As will this:
exec ('declare #x int = 2')
exec ('select #x')
And this:
exec ('declare #x int = 2')
select #x
You'd have to do it like this:
exec ('declare #x int = 2; select #x')
Or otherwise pass the results back somehow. The sp_executesql suggestion in #TT.'s answer is a good idea.
When you declare variables, the visibility is restricted to the scope in which they are declared. When you EXEC a statement, a new session and thereby a new scope is created.
What you need to do is output the scalar variable using an OUTPUT parameter in a call to sp_executesql:
SET #sql='Set #keytypenum = ...';
EXEC sp_executesql #sql,N'#keytypenum INT OUT', #keytypenum OUTPUT;
Note that the #sql variable needs to be an NVARCHAR.
You can find more examples here.

SQL Server update statement performance

I have problem in optimizing an SQL query to do some data cleansing.
In fact, I have a table which is a sort of referential of a multiple special characters and word. Let's call it ABNORMAL(ID,PATTERN)
I have also another table INDIVIDUALS containing a column (NAME) which I want to clean by removing from it all characters that exist in the table ABNORMAL.
Currently, I have tried to use update statements, but I'm not sure if there is a better way to do this.
Approach one
Use a while loop to build a replace containing all characters from ABNORMALS by a blank '' and do one update using the built-in REPLACE
DECLARE #REPLACE_EXPRESSION nvarchar(max) ='REPLACE(NAME,'''','''')'
DECLARE #i int = 1
DECLARE #nbr int = (SELECT COUNT(*) FROM ABNORMAL)
-- CURRENT_CHARAC
DECLARE #CURRENT_CHARAC nvarchar(max)
-- NEW REPLACE EXPRESSION TO IMBRICATE INTO THE REPLACE EXPRESSION VARIABLE
DECLARE #CURR_REP NVARCHAR(max)
-- STRING TO BUILD AN SQL QUERY CONTAINING THE REPLACE EXPRESSION
DECLARE #UPDATE_QUERY nvarchar(max)
WHILE #i < #nbr
BEGIN
SELECT #CURRENT_CHARAC=PATTERN FROM CLEANSING_STG_PRISM_FRA_REF_UNSIGNIFICANT_VALUES WHERE ID_PATTERN=#i ;
SET #REPLACE_EXPRESSION = REPLACE(#REPLACE_EXPRESSION ,'NAME','REPLACE(NAME,'+''''+#CURRENT_CHARAC+''''+','''')')
set #i=#i+1
END
SET #UPDATE_QUERY = 'UPDATE INDIVIDUAL SET NAME ='+ #REPLACE_EXPRESSION
EXEC sp_executesql #UPDATE_QUERY
Approach two
Use a while loop to select every character in abnormal and do an update using replace containing the characters to remove:
DECLARE #i int = 1
DECLARE #nbr int = (SELECT COUNT(*) FROM ABNORMAL)
-- CURRENT_CHARAC
DECLARE #CURRENT_CHARAC nvarchar(max)
-- STRING TO BUILD AN SQL QUERY CONTAINING THE REPLACE EXPRESSION
DECLARE #UPDATE_QUERY nvarchar(max)
WHILE #i < #nbr
BEGIN
SELECT #CURRENT_CHARAC=PATTERN FROM CLEANSING_STG_PRISM_FRA_REF_UNSIGNIFICANT_VALUES WHERE ID_PATTERN=#i ;
UPDATE INDIVIDUAL
SET NAME = REPLACE(NAME,#CURRENT_CHARAC,'')
SET #i=#i+1
END
I already tested both approaches for 2 millions records, and I found that the first approach is faster than the second. I would know if you have already done something similar and new (better) ideas to try.
If you are using SQL Server 2017 you could use TRANSLATE and avoid dynamic SQL:
SELECT i.*
, REPLACE(TRANSLATE(i.NAME, f, REPLICATE('!', s.l)), '!', '') AS cleansed
FROM INDIVIDUALS i
OUTER APPLY (SELECT STRING_AGG(PATTERN, '') AS f
,LEN(STRING_AGG(PATTERN,'')) AS l
FROM ABNORMAL) AS s
DBFiddle Demo
Anyway 1st approach is better becasue you do one UPDATE, with second approach you remove characters one character at time (so you will have multiple UPDATE).
I would also track transaction log growth with both approaches.
If there's not too many characters that to be cleaned, then this trick might work.
Basically, you build 1 big update statement with a replace for each value in the table with the characters to be removed.
Example code:
Test data (using temp tables)
create table #ABNORMAL_CHARACTERS (id int identity(1,1), chr varchar(30));
insert into #ABNORMAL_CHARACTERS (chr) values ('!'),('&'),('#');
create table #INDIVIDUAL (id int identity(1,1), name varchar(30));
insert into #INDIVIDUAL (name) values ('test 1 &'),('test !&#2'),('test 3');
Code:
declare #FieldName varchar(30) = 'name';
declare #Replaces varchar(max) = #FieldName;
declare #UpdateSQL varchar(max);
select #Replaces = concat('replace('+#Replaces+', ', ''''+chr+''','''')') from #ABNORMAL_CHARACTERS order by id;
set #UpdateSQL = 'update #INDIVIDUAL
set name = '+#Replaces + '
where exists (select 1 from #ABNORMAL_CHARACTERS where charindex(chr,name)>0)';
exec (#UpdateSQL);
select * from #INDIVIDUAL;
A test here on rextester
And if you would have a UDF that can do a regex replace.
For example here
Then the #Replaces variable could be simplified with only 1 RegexReplace function and a pattern.

Creating multiple UDFs in one batch - SQL Server

I'm asking this question for SQL Server 2008 R2
I'd like to know if there is a way to create multiple functions in a single batch statement.
I've made the following code as an example; suppose I want to take a character string and rearrange its letters in alphabetical order. So, 'Hello' would become 'eHllo'
CREATE FUNCTION char_split (#string varchar(max))
RETURNS #characters TABLE
(
chars varchar(2)
)
AS
BEGIN
DECLARE #length int,
#K int
SET #length = len(#string)
SET #K = 1
WHILE #K < #length+1
BEGIN
INSERT INTO #characters
SELECT SUBSTRING(#string,#K,1)
SET #K = #K+1
END
RETURN
END
CREATE FUNCTION rearrange (#string varchar(max))
RETURNS varchar(max)
AS
BEGIN
DECLARE #SplitData TABLE (
chars varchar(2)
)
INSERT INTO #SplitData SELECT * FROM char_split(#string)
DECLARE #Output varchar(max)
SELECT #Output = coalesce(#Output,' ') + cast(chars as varchar(10))
from #SplitData
order by chars asc
RETURN #Output
END
declare #string varchar(max)
set #string = 'Hello'
select dbo.rearrange(#string)
When I try running this code, I get this error:
'CREATE FUNCTION' must be the first statement in a query batch.
I tried enclosing each function in a BEGIN END block, but no luck. Any advice?
Just use a GO statement between the definition of the UDFs
Not doable. SImple like that.
YOu can make it is one statement using a GO between them.
But as the GO is a batch delimiter.... this means you send multiple batches, which is explicitly NOT Wanted in your question.
So, no - it is not possible to do that in one batch as the error clearly indicates.