Trouble replacing abbreviated text correctly in SQL - sql

I am trying to replace a bunch of difficult to decipher abbreviations with corresponding detailed descriptions. I have a table called Abbreviations that simply holds a list of abbreviations to look for and the corresponding descriptions they should be changed to. Additionally the "Replaced" table holds a list of unaltered abbreviated descriptions that I would like to change in a single column called "DescriptionCodes"
The data I am trying to change is a list of different teas. For instance the entry
"TADIN H-B GR" would be the abbreviation for "TADIN HERBAL BAG GREEN"
The SQL Code I am currently using looks like this:
BEGIN TRANSACTION
DECLARE #Desc varchar(500)
DECLARE #Abbr varchar(500)
DECLARE contact_cursor CURSOR FOR
SELECT Description, Abbrv FROM dbo.Abbreviations
OPEN contact_cursor
FETCH NEXT FROM contact_cursor
INTO #Desc, #Abbr
WHILE ##FETCH_STATUS = 0
BEGIN
PRINT 'Changing ' + #Abbr + ' to ' + #Desc
UPDATE Replaced
SET DescriptionCodes = REPLACE(DescriptionCodes, #Abbr, #Desc)
WHERE DescriptionCodes LIKE CONCAT('% ', #Abbr, ' %')
FETCH NEXT FROM contact_cursor
INTO #Desc, #Abbr
END
CLOSE contact_cursor
DEALLOCATE contact_cursor
COMMIT
Of course the problem I am running into is that once an abbreviation is applied the detailed description may contain a substring that matches another abbreviation that is then applied. For instance PKG might be changed to PACKAGE but PA is the abbreviation for PINEAPPLE as well, meaning that once that change is applied the final result would be PINEAPPLECKAGE. Every abbreviation has a single blank space on either side so I thought to use that fact to not update any abbreviation without a blank space on either side, hence the
CONCAT('% ', #Abbr, ' %') in my code. However, when I try this method basically nothing gets changed at all. I've been able to see limited success by removing that constraint but the other issue then happens. Any ideas as to how I could make sure that only occurrences of abbreviations with a blank space on either side are considered and updated?
UPDATE:
After trying several of the solutions posted here, I still haven't been able to get it to work and I'm not sure why. By all appearances this is correct. Here is a sample of some of the data I'm working with:
JUSTEA HBL PKG CMCL CHM LG LR 1.5OZ
PRIDE OF INDIA BG ASM B BKFST 25 CT
CTL BR H-B 7BLSM PP 1 CT
POSTI H-B HRH CRN 20 CT
DRS H-B EPGP LPLDS PTVP TGN 20 CT
ULTLC BG CHG 100 CT
PG TIPS BG D B 40 CT
RPBL R-B B HLDY FT BLD 6 CT
This is fine and looks like it should convert with no issues. Yet when I run the code with spaces indicated (As with "LIKE '% ' + #Abbr + ' %'", which was my first inclination) the data remains completely unchanged. If I remove them the data becomes completely unintelligible. For example the line beginning with PRIDE OF INDIA becomes
PRIDE OF INDIA(N) IRISH AFTERNOON BLACK AG ASIA PLUM RICOT SPICE(D) EARMINT BLACK BLACK KFST 25 CURRANT AN AID N T
I feel I should note that this data was imported from Excel Spreadsheets originally. Is there any chance that has anything to do with the spaces not being recognized?

Why use "LIKE" if you're going to add the spaces? just do:
WHERE DescriptionCodes = #Abbr

You can do this and it will achieve what you are trying to do:
LIKE '% ' + #Abbr + ' %'

I think you need to update each record of the Replaced table multiple times, and you need to consider the 4 locations of the abbreviations (Alone, First, In the middle, Last). Something like this:
DECLARE #Replaced TABLE ([DescriptionCodes] varchar(50))
DECLARE #Abbreviations TABLE ([Abbrv] varchar(50), [Description] varchar(50))
INSERT INTO #Replaced([DescriptionCodes]) VALUES ('TADIN H-B GR')
INSERT INTO #Replaced([DescriptionCodes]) VALUES ('PKG')
INSERT INTO #Replaced([DescriptionCodes]) VALUES ('PKG PA')
INSERT INTO #Abbreviations([Abbrv], [Description]) VALUES ('H-B', 'HERBAL BAG')
INSERT INTO #Abbreviations([Abbrv], [Description]) VALUES ('GR', 'GREEN')
INSERT INTO #Abbreviations([Abbrv], [Description]) VALUES ('PKG', 'PACKAGE')
INSERT INTO #Abbreviations([Abbrv], [Description]) VALUES ('PA', 'PINAPPLE')
DECLARE #RowCount int;
WHILE 1 = 1
BEGIN
SET #RowCount = 0;
UPDATE r
SET r.[DescriptionCodes] = REPLACE(r.[DescriptionCodes], a.[Abbrv], a.[Description])
FROM #Replaced r join #Abbreviations a ON r.[DescriptionCodes] = a.[Abbrv];
SET #RowCount = #RowCount + ##RowCount;
UPDATE r
SET r.[DescriptionCodes] = REPLACE(r.[DescriptionCodes], ' ' + a.[Abbrv] + ' ', ' ' + a.[Description] + ' ')
FROM #Replaced r join #Abbreviations a ON r.[DescriptionCodes] like '% ' + a.[Abbrv] + ' %';
SET #RowCount = #RowCount + ##RowCount;
UPDATE r
SET r.[DescriptionCodes] = REPLACE(r.[DescriptionCodes], ' ' + a.[Abbrv],' ' + a.[Description])
FROM #Replaced r join #Abbreviations a ON r.[DescriptionCodes] like '% ' + a.[Abbrv];
SET #RowCount = #RowCount + ##RowCount;
UPDATE r
SET r.[DescriptionCodes] = REPLACE(r.[DescriptionCodes], a.[Abbrv] + ' ', a.[Description] + ' ')
FROM #Replaced r join #Abbreviations a ON r.[DescriptionCodes] like a.[Abbrv] + ' %';
SET #RowCount = #RowCount + ##RowCount;
IF #ROWCOUNT = 0 BREAK;
END
SELECT * FROM #Replaced

Related

How to select into table as string builder instead of using cursor in SQL

I am wondering if there is a faster way to handle the following code in SQL. Currently I am using SQL cursor to do a select and build a string of delimiter values, as a dynamic value of suggestion items?
Here is the snippet of SQL:
begin
set #cursor = cursor for
select top 5 Manufacturer,ManufacturerPartNumber,Description as ManufacturerDescription, CONVERT(money,Price) as Price,fms.Score
from Products_OurProducts_Products_View
open #cursor
fetch next from #cursor
into #CURSOR_Mfr,#CURSOR_Model,#CURSOR_Desc,#CURSOR_Price,#CURSOR_Score
while ##FETCH_STATUS = 0
begin
set #suggestionsStringBuilder += #CURSOR_Mfr + ',' + #CURSOR_Model + ',' + #CURSOR_Desc + ',' + convert(varchar(20),#CURSOR_Price) + ',' + convert(varchar(4),#CURSOR_Score) + '^'
fetch next from #SuggestionsListCursor
into #CURSOR_Mfr,#CURSOR_Model,#CURSOR_Desc,#CURSOR_Price,#CURSOR_Score
end
insert into BASE (Manufacturer, ManufacturerOrig, ManufacturerPartNumber,ManufacturerPArtNumberOrig,ManufacturerDescription, QWDescription, Serial,AssetID,Price,Score,ItemType,MfrFound,ModelFound,trained, SuggestionList,LineNumberIn)
values(#objectORIGMfr,#objectORIGMfr, #objectORIGModel, #objectORIGModel, #objectDescription, #objectDescription, '',#objectAssetID,'0.00',#topMaxScore,'NA','1','0',#trained,#suggestionsStringBuilder,#objectLineNumber)
close #cursor
deallocate #cursor
end
The code above is trying to build a dynamic column of delimiter values such as shown below:
Object Example:
Mfr,
Model,
Price,
Score,
Description,
Suggestions = 'Mfr,Model,Desc,Price^Mfr,Model,Description,Price^
A return model would truly be as follows:
BaseMfr:Fluke,
BaseModel:Tb1,
BaseDescription:'Multi meter item',
BasePrice:120.00,
Suggestions: "Fluke, Tc1, 'Desc', '120.00' ^ 'Fluke', 'T11', 'Desc', 220.00"
Can I do the string builder / cursor section without having to use a looping cursor? The idea behind this is we send in items to be priced. If the item is not found, we then build a list of suggestions to bring back to the user of what they may use in the system or so they can see if there is a typo in the data.
The suggestion list is just the rows found, separating the columns by a "," and separating entities by a "^".
Thanks very much in advance!
Thanks you all for the feedback and I appreciate the help even though I know I had a rough time explaining the question correctly. Thanks to the suggestion from Sean Lange, I was able to be directed in the correct direction and came up with this. Now I will test the performance of it to see if it is better or not. Here is the code:
select
SUBSTRING(
(select top 5
Manufacturer + ',' + ManufacturerPartNumber + ',' + Description +',' + CONVERT(VARCHAR,Price) +',' + CONVERT(varchar,fms.Score) +'^' as [text()]
from Products_OurProducts_Products_View
CROSS APPLY (
select
dbo.FuzzyControlMatch('Flooke', Manufacturer) AS score
) AS fms
order by fms.score desc
FOR XML PATH ('')
), 2, 1000) [Suggestions]
The above code produces the following string:
ARD BROOKE,WB808 10UNF,TORQUE SCREWDRIVER,70.00,50^WARD BROOKE,WB808 1146,TORQUE SCREWDRIVER,70.00,50^WARD BROOKE,WB808 1246,TORQUE SCREWDRIVER,70.00,50^WARD BROOKE,WB808 6UNC,TORQUE SCREWDRIVER,70.00,50^ROKEM TECHNOLOGIES,FIRESET,RC STANDARD,105.00,50^
Now I am not sure if I am handling this the best way, but this is what I was searching for. I will post a comment update to let the feed know if the performance is better or worse.
-Thanks-

I want to give information according to profession using 'Trigger'

'Trigger' I want to give information according to the profession using. For example; When I entered the engineer I added 'engineer'. This is his name and surname. ' as.
ALTER TRIGGER tigger_example
ON information
AFTER INSERT
AS
IF EXISTS(SELECT * FROM inserted WHERE Person_Job='Engineer')
BEGIN
PRINT 'Engineer added to list.'+
'Person Info:' +
'Name : ' + person_firstname + -- not work
'Surname : ' + person_lastname -- not work
END
ELSE IF EXISTS(SELECT * FROM inserted WHERE Person_Job='Architect')
BEGIN
PRINT 'Architect added to list.'+
'Person Info:' +
'Name : ' + person_firstname + -- not work
'Surname : ' + person_lastname -- not work
END
ELSE
BEGIN
PRINT 'An undefined contact has been added to the list.'
END
ERROR:
The name "person_firstname" is not permitted in this context. Valid expressions are constants, constant expressions, and (in some contexts) variables. Column names are not permitted.
You have to use "SELECT ... from inserted" for read field value.
example:
ALTER TRIGGER tigger_example
ON information
AFTER INSERT
AS
DECLARE #MSG NVARCHAR(MAX)
IF EXISTS(SELECT * FROM inserted WHERE Person_Job='Engineer')
BEGIN
SET #MSG ='Engineer added to list.'+CHAR(13)+CHAR(10)+'Person Info:'
SELECT #MSG=#MSG+CHAR(13)+CHAR(10)+'Name : ' + person_firstname + 'Surname : ' + person_lastname
FROM inserted WHERE Person_Job='Engineer'
print #MSG
END
ELSE IF EXISTS(SELECT * FROM inserted WHERE Person_Job='Architect')
BEGIN
SET #MSG ='Architect added to list.'+CHAR(13)+CHAR(10)+'Person Info:'
SELECT #MSG=#MSG+CHAR(13)+CHAR(10)+'Name : ' + person_firstname + 'Surname : ' + person_lastname
FROM inserted WHERE Person_Job='Architect'
print #MSG
END
ELSE
BEGIN
PRINT 'An undefined contact has been added to the list.'
END

Adding a space to the data

I have a stored procedure like the following:
declare #mySymbol as varchar(32)
DECLARE #RefGroupID AS VARCHAR(20)
declare #myJID as varchar(45)
declare Div_csr cursor for
select distinct Symbol2, RefGroupID, jnl_id
from #TDRows td
WHERE TD_CompanyCode_AU = 'A'
open Div_csr
fetch next from Div_csr into #mySymbol, #RefGroupID, #myJID
while ##FETCH_STATUS = 0 begin
SET #InfoMessage = #InfoMessage +CONVERT(varchar(30),getdate(), 120)+ ' ' + #SPname
+ ': RefGroupID:'+ convert(varchar(20),#RefGroupID,101)
+ ' Sym: ' + #mySymbol
+ ' JID: ' + #myJID
+ CHAR(13) + CHAR(10)
fetch next from Div_csr into #mySymbol, #RefGroupID, #myJID
end
close Div_csr
deallocate Div_csr
#mySymbol is the data I want to modified.
For example; #mySymbol is BXP PRB. You can see there is a space between BXP and PRB. I want to this space also show in the comment that was created using the stored procedure.
Right now, the commment's Sym: is parsed by data #mySymbol. But it does not have space now. I do not know why.. .
The comment looks like this:
"DIV2PAY Record receipt from CSH RecordDt=08/04/2017 Intended PayDt= 08/15/2017 Sym=BXPPRB Qty=-100 CashRate=0.328125 Kind= Cash Dividend RowID= 127278 CAEventID= 105226767
".
Could anyone help me with this?

T-SQL Procedure not working, issue with varchar to date conversion

When I run this SP, I get:
Msg 241, Level 16, State 1, Procedure PED_SP_PED_Updates, Line 22
Conversion failed when converting date and/or time from character string.
Here is the execution:
exec dbo.ped_sp_ped_updates
#CURRENTHICN='111111111A',
#DATERECEIVED = '20140904',
#FIELDTOBECHANGED='FIRST_NAME_MEMBER',
#CURRENTFIELDVALUE = 'MARY',
#NEWFIELDVALUE = 'MARYTEST',
#REQUESTEDBY = 'IPISORS',
#ID=156
I am not sure why, I'm casting the varchar back to a date for the comparison.
Please note, I have no problem being told a better way to do it, but it would be (I think) more helpful to my learning if I could, at least 'also', get a direct answer as to why my current proc isn't working. In addition to any helpful ideas as to why it should be done different, better, etc, etc. etc.
ALTER PROCEDURE [dbo].[PED_SP_PED_Updates]
#CurrentHicn VARCHAR(500),
#DateReceived VARCHAR(20),
#FieldToBeChanged VARCHAR(500),
#CurrentFieldValue VARCHAR(500),
#NewFieldValue VARCHAR (500),
#RequestedBy VARCHAR(10),
#ID int
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
-- Insert statements for procedure here
DECLARE #CurrentDBNote VARCHAR(MAX)
DECLARE #NewNote VARCHAR(MAX)
DECLARE #CountofHicn INT
SET #NEWNOTE = 'Isaac Pisors | ' + GetDate() + ' | '
+ 'Changing field: ' + #FieldToBeChanged + ' from ' + #CurrentFieldValue + ' to ' + #NewFieldValue
+ ', per ' + #RequestedBy + ' request. Also changing any related DOCS/FAXES records to correspond'
SET #CurrentDBNote=
(SELECT NOTES_GENERAL FROM PED_APPLICATIONS WHERE HICN_MEDICARE_NUMBER=#CurrentHicn AND (Cast(ISNULL(DATE_RECEIVED,'1900-01-01') as DATE)=CAST(#DateReceived AS DATE)))
--NOW ADD THE TWO:
SET #NewNote = #CurrentDBNote + CHAR(13) + #CurrentDBNote
--SEE IF THERE IS STILL A MATCHING RECORD
SET #CountofHicn=
(SELECT COUNT(*) FROM PED_APPLICATIONS WHERE HICN_MEDICARE_NUMBER=#CurrentHicn AND (CAST(ISNULL(DATE_RECEIVED,'1900-01-01') AS DATE)=CAST(#DateReceived AS DATE)))
IF #CountofHicn=0 --THERE IS NO LONGER A MATCHING RECORD - INSERT THAT NOTE AND CALL IT A DAY
BEGIN
UPDATE PED_PEDUPDATES SET COMPLETEDON=GetDate(), COMPLETEDBY='SSIS',
EXCEPTIONNOTE='Could not locate any records where HICN is ' + #CurrentHicn + ' and Date Received is ' + CAST(#DateReceived AS VARCHAR)
WHERE [ID]=#ID
END
ELSE --GO AHEAD AND DO THE UPDATE
BEGIN
UPDATE PED_APPLICATIONS SET #FieldToBeChanged = #NewFieldValue
WHERE HICN_MEDICARE_NUMBER=#CurrentHicn AND (CAST(ISNULL(DATE_RECEIVED,'1900-01-01') AS DATE)=CAST(#DateReceived AS DATE))
END
IF #FieldToBeChanged='HICN_MEDICARE_NUMBER' --THEN WE HAVE TO UPDATE DOCS TABLE, TOO
BEGIN
UPDATE PED_DOCS SET HICN_MEDICARE_NUMBER=#NewFieldValue
WHERE
(HICN_MEDICARE_NUMBER=#CurrentFieldValue AND (CAST(ISNULL(DATE_RECEIVED,'1900-01-01') AS DATE)=#DateReceived)) or
(HICN_MEDICARE_NUMBER=#CurrentFieldValue AND DATE_RECEIVED IS NULL)
END
IF #FieldToBeChanged='HICN_MEDICARE_NUMBER' --THEN OUR WHERE CLAUSE-HICN IS THE *NEW* HICN
BEGIN
UPDATE PED_APPLICATIONS SET NOTES_GENERAL=#NewNote
WHERE HICN_MEDICARE_NUMBER=#NewFieldValue AND (CAST(ISNULL(DATE_RECEIVED,'1900-01-01') AS DATE)=CAST(#DateReceived AS DATE))
END
ELSE --ELSE OUR WHERE CLAUSE-HICN IS THE *OLD* HICN
BEGIN
UPDATE PED_APPLICATIONS SET NOTES_GENERAL=#NewNote
WHERE HICN_MEDICARE_NUMBER=#CurrentHicn AND (CAST(ISNULL(DATE_RECEIVED,'1900-01-01') AS DATE)=CAST(#DateReceived AS DATE))
END
--FINALLY, UPDATE RECORD AS COMPLETE:
UPDATE PED_PEDUPDATES SET COMPLETEDON=GetDate(),COMPLETEDBY='SSIS' WHERE [ID]=#ID
END
GO
Short Term Fix
Instead of CAST(#DateReceived AS DATE), use CONVERT(date, #DateReceived, 112)
The value 112 is the style code for the yyyymmdd formated varchar you're using. See the cast and convert documentation for more details.
Also, you should verify that all values in the DATE_RECEIVED column of your table are in the correct format. Even one value that is not convertible will cause this error.
Proper Fix
#DateReceived should be passed in to the procedure as a date instead of a varchar.
The DATE_RECEIVED field in your table should be declared as a date instead of a varchar.
In general, avoid treating dates or times as strings in a database when there are native types for that purpose.

tsql Loop with external query

I am looping through all my databases and aggregating the results into an aggregates database.
In my loop I call
master.dbo.xp_cmdshell osql C:\whatever.SQL
As the loop progresses, the cmdshell takes longer and longer to execute. If I stop the loop and run a single aggregate for one database it executes quickly.
Is there anything I can add to my external SQL script to make it run faster? Maybe something to commit and free the records before the next loop? Or should I add some kind of a pause after every loop?
I want to use an external SQL file because it contains many update statements and it's more manageable for me.
Here's how I loop:
Update dbFoo.dbo.tblBar set Processed = 0
Go
WHILE EXISTS ( SELECT ID FROM dbFoo.dbo.tblBar WHERE Processed = 0)
BEGIN
SELECT #aRow = MIN(tblBar.ID) FROM dbFoo.dbo.tblBar
SELECT #aFoo1 = Foo1 FROM dbFoo.dbo.tblBar WHERE ID = #aRow
SELECT #aFoo2 = Foo2 FROM dbFoo.dbo.tblBar WHERE ID = #aRow
SELECT #aFoo3 = Foo3 FROM dbFoo.dbo.tblWhatever WHERE Foo = #aFoo
EXEC RunPreAgg #Foo1 = #aFoo1, #Foo2 = #aFoo2, #Foo3 = #aFoo3, #RetVal = #aRetVal OUTPUT
SELECT returning = #aRetVal
UPDATE dbFoo.dbo.tblBar SET Processed = 1 WHERE ID = #aRow
END
Then the RunPreAgg stored procedure basically does this:
if db_id('db' + #Foo1 + '_' + #Foo2) is not null
BEGIN
--This bat file creates the SQL File
select #sql = 'master.dbo.xp_cmdshell '''+#path+'wwwRunPreAgg.bat ' + #Foo1 + ' ' + #Foo2 + ' ' + #Foo3 + ''''
exec( #sql )
--execute
select #sql = 'master.dbo.xp_cmdshell ''osql -E -o '+#path+'output\tmp'+#Foo1+'_'+#Foo2+'.txt -i '+#path+'tmp' + #Foo1 + '.SQL'''
exec( #sql )
--This erases the SQL File
select #sql = 'master.dbo.xp_cmdshell '''+#path+'wwwCleanup.bat ' + #Foo1 + ' ' + #Foo2 + ''''
exec( #sql )
Set #retval = 'Done!'
END
ELSE
BEGIN
Set #retval = 'Err: No DataBase'
END
The variable names are changed to protect the innocent. The code works fine, I just need to optimize.
If it is the loops performance that is causing you trouble, you might try reducing the number of selects. Normally I dislike Cursors, but your loop might benefit from one. You can select all the values you need for the loop into memory, then loop through those values without having to run 3 or 4 selects per loop (of course if the performance hit is occurring inside the RunPreAgg SP, then this won't help):
DECLARE cFoos CURSOR FOR
SELECT tblBar.ID, tblBar.Foo1, tblBar.Foo2, tblWhatever.Foo3
FROM dbFoo.dbo.tblBar
INNER JOIN dbFoo.dbo.tblWhatever
ON tblWhatever.Foo = tblBar.Foo
WHERE tblBar.Processed = 0;
OPEN cFoos;
FETCH NEXT FROM cFoos INTO #aRow, #aFoo1, #aFoo2, #aFoo3;
WHILE ##FETCH_STATUS = 0
BEGIN
EXEC RunPreAgg #Foo1 = #aFoo1, #Foo2 = #aFoo2, #Foo3 = #aFoo3, #RetVal = #aRetVal OUTPUT
SELECT returning = #aRetVal
UPDATE dbFoo.dbo.tblBar SET Processed = 1 WHERE ID = #aRow
FETCH NEXT FROM cFoos INTO #aRow, #Foo1, #Foo2, #Foo3;
END
CLOSE cFoos;
DEALLOCATE cFoos;