We have an extremely large nvarchar(max) field that contains html. Within this html is an img tag.
Example:
<img style="float:right" src="data:image/png;base64,/9j/4AAQSkZJRgABAQEBLAEsAAD/7gAOQW....
The length of this column is 1645151, although what is being replace is a bit less than this, but not a lot.
What we are trying to do, is a replace in SQL on the column:
declare #url varchar(50) = 'myimageurl';
UPDATE table SET field =
CAST(REPLACE(CAST(field as NVARCHAR(MAX)),#source,'#url') AS NVARCHAR(MAX))
Where #source, is the above image bytes as string, which are assigned to an nvarchar(max) variable before running the replace. and dest is the url of an image, rather than the images bytes as string.
Although I still get the message string or binary data would be truncated.
Does anyone know if this is possible in SQL to replace strings as large as this.
I had the same error, but on a different function.
The fault was that my pattern has longer than my expression, which means that your search pattern will be truncated.
I hope this helps someone.
Also, make sure you put pattern and expression in the right location of your function.
Instead of doing the replace, can you rebuilt the entire field by parsing out the rest of the img tag?
Something like:
declare #Field nvarchar(max) = '<img style="float:right" src="data:image/png;base64,/9j/4AAQSkZJRgA....BAQEBLAEsAAD/7gAOQW" />'
declare #Source nvarchar(max) = 'data:image/png;base64,/9j/4AAQSkZJRgA....BAQEBLAEsAAD/7gAOQW'
declare #URL nvarchar(max) = 'www.img.img/img.png'
declare #Chars int = 20
select left(#Field,patindex('%' + left(#Source,#Chars) + '%', #Field) - 1) as HTMLStart
,#URL as ImgURL
,right(#Field,len(#Field) - patindex('%' + right(#Source,#Chars) + '%', #Field) - #Chars + 1) as HTMLEnd
If you were wanting to run this on a whole dataset at once, you would simply need to look for the src="data:image/png;base64, element and work backwards from there using a similar methodology to the above. Depends on how you are identifying which binary data to replace and what to replace it with.
Related
In TSQL, I need to format a string in a predefined format.
For eg:
SNO
STRING
FORMAT
OUTPUT
1
A5233GFCOP
*XXXXX-XXXXX
*A5233-GFCOP
2
K92374
/X-000XXXXX
/K-00092374
3
H91543987
XXXXXXXXX
H91543987
I am trying with FORMATMESSAGE() built in function.
For ex:
FORMATMESSAGE('*%s-%s','A5233','GFCOP')
FORMATMESSAGE('/%s-000%s','K','92374')
FORMATMESSAGE('%s','H91543987')
I am able to get the first argument by replace function but issue is second/third/fourth/.. arguments.
I don't know how to count respective X's between the various delimiters, so that I can use substring to pass in second/third/.. arguments. If I can count the respective # of X's from the Format column, I feel using substring we can get it but not sure how to count the respective X's.
Please let me know how to get through it or if there is any other simple approach.
Appreciate your help.
Thanks!
It's in theory quite simple, could probably be done set-based using string_split however that's not ideal as the ordering is not guaranteed. As the strings are fairly short then a scalar function should suffice. I don't think it can use function in-lining.
The logic is very simple, create a counter for each string, loop 1 character at a time and pull a character from one or the other into the output depending on if the format string is an X or not.
create or alter function dbo.fnFormatString(#string varchar(20), #format varchar(20))
returns varchar(20)
as
begin
declare #scount int=1, #fcount int=1, #slen int=len(#string), #flen int=Len(#format), #output varchar(20)=''
while #scount<=#slen or #fcount<=#slen
begin
if Substring(#format,#fcount,1)='X'
begin
set #output+=Substring(#string,#scount,1)
select #scount+=1, #fcount +=1
end
else
begin
set #output+=Substring(#format,#fcount,1)
set #fcount +=1
end
end
return #output
end;
select *, dbo.fnFormatString(string, [format])
from t
See working Fiddle
I am working on some string manipulation with PATINDEX to fix some incorrect time formatting in XML e.g. (2018-12-20T17:00:00-05:00).
The issue I am having is PATINDEX is finding a match to #Pattern in the #IncorrectMatchIndex string.
You can recreate the issue by running the following:
DECLARE #Pattern nvarchar(36) = '%<EstmatedTime>%T%-%</EstmatedTime>%',
#CorrectMatchIndex nvarchar(100) = '<DiscountedRate>263.34</DiscountedRate><EstmatedTime>2018-12-20T17:00:00-05:00</EstmatedTime></Rate>',
#CorrectMatchIndex2 nvarchar(94) = '<DiscountedRate>263.34</DiscountedRate><EstmatedTime>2018-12-20T17:00:00</EstmatedTime></Rate>',
#IncorrectMatchIndex nvarchar(296) = '<DiscountedRate>263.34</DiscountedRate><EstmatedTime>2018-12-20T17:00:00</EstmatedTime></Rate><Rate><Carrier>FedEx Freight</Carrier><Service>FEDEX_FREIGHT_PRIORITY</Service><PublishedRate>520.6</PublishedRate><DiscountedRate>272.04</DiscountedRate><EstmatedTime>2018-12-18T17:00:00</EstmatedTime>'
SELECT
PATINDEX(#Pattern, #CorrectMatchIndex) AS CorrectMatchIndex,
PATINDEX(#Pattern, #CorrectMatchIndex2) AS CorrectMatchIndex2,
PATINDEX(#Pattern, #IncorrectMatchIndex) AS IncorrectMatchIndex
At a pure guess, I suspect you want:
DECLARE #Pattern nvarchar(300) = '%<EstmatedTime>[1-2][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]T[0-9][0-9]:[0-9][0-9]:[0-9][0-9]-[0-9][0-9]:[0-9][0-9]</EstmatedTime>%'
This then returns 0 for IncorrectMatchIndex.
Of course, the comments are right, you should really be using XQUERY for this. I can't provide a sample for this, however, as none of the XML data you have supplied it valid XML (for example #CorrectMatchIndex ends with '</Rate>' but that node is never opened).
The #IncorrectMatchIndex string does not contain a match to %<EstmatedTime>%T%-%</EstmatedTime>% as far as I can see. There is no dash between the T and closing </EstmatedTime>
Yes there is. Because there is a second set of <EstimatedTime> tags later in the string, and there most certainly is a '-' character between the first T and the last </EstimatedTime>
I have created a data mask that finds a 16 digit number anywhere within a string and replaces all but the last four characters with X's.
But instead of manually setting the string I need to update all data within a column located in a table. Please see my code so far:
DECLARE
#NOTES AS VARCHAR(8000)
SET #NOTES = 'Returns the starting position of the first occurrence of a pattern in a specified expression, 1234567891234567 or zeros if the pattern is not found, on all valid text and character data types'
SELECT
REPLACE(#NOTES, SUBSTRING(#NOTES, PATINDEX('%1%2%3%4%5%6%7%8%9%', #NOTES), 16), 'XXXXXXXXXXXX' + RIGHT(SUBSTRING(#NOTES, PATINDEX('%1%2%3%4%5%6%7%8%9%', #NOTES),16),4)) AS REPLACEMENT
Any help would be much appreciated :-)
Create a function with your logic
CREATE FUNCTION MyMask(
#NOTES VARCHAR(8000))
returns varchar(8000)
BEGIN
RETURN
REPLACE(#NOTES, SUBSTRING(#NOTES, PATINDEX('%1%2%3%4%5%6%7%8%9%', #NOTES), 16), 'XXXXXXXXXXXX' + RIGHT(SUBSTRING(#NOTES, PATINDEX('%1%2%3%4%5%6%7%8%9%', #NOTES),16),4))
END
This is who you use it
update table
set field = dbo.myMask(field)
where some condition
The function provided by Horaciux, works re a static declared string, but the PATINDEX always sets to 0 when used in an update query.
The work around was to amend the implementation of the PATINDEX from PATINDEX('%1%2%3%4%5%6%7%8%9%' to PATINDEX('%[123456789]%' I have included the full function below:
CREATE FUNCTION [dbo].[MyMask](#NOTES VARCHAR(8000)) RETURNS VARCHAR(8000)
BEGIN
RETURN
REPLACE(#NOTES, SUBSTRING(#NOTES, PATINDEX('%[123456789]%', #NOTES), 16), 'XXXXXXXXXXXX' + RIGHT(SUBSTRING(#NOTES, PATINDEX('%[123456789]%', #NOTES),16),4))
END
I hope this is useful to others :-)
I've got a stored procedure I use to insert data from a csv. The data itself is a mix of types, some test, some dates, and some money fields. I need to guarantee that this data gets saved, even if it's formatted wrong, so, I'm saving them all to varchars. Later, once the data's been validated and checked off on, it will be moved to another table with proper datatypes.
When I do the insert into the first table, I'd like to do a check that sets a flag (bit column) on the row if it needs attention. For instance, if what should be a money number has letters in it, I need to flag that row and add the column name in an extra errormsg field I've got. I can then use that flag to find and highlight for the users in the interface the fields they need to edit.
The date parameters seem to be easy, I can just use IF ISDATE(#mydate) = '0' to test if that parameter could be converted from varchar to datetime. But, I can't seem to find an ISMONEY(), or anything that's remotely equivalent.
Does anyone know what to call to test if the contents of a varchar can legitimately be converted to money?
EDIT:
I haven't tested it yet, but what do you think of a function like this?:
CREATE FUNCTION CheckIsMoney
(
#chkCol varchar(512)
)
RETURNS bit
AS
BEGIN
-- Declare the return variable here
DECLARE #retVal bit
SET #chkCol = REPLACE(#chkCol, '$', '');
SET #chkCol = REPLACE(#chkCol, ',', '');
IF (ISNUMERIC(#chkCOl + 'e0') = '1')
SET #retVal = '1'
ELSE
SET #retVal = '0'
RETURN #retVal
END
GO
Update
Just finished testing the above code, and it works!
money is decimal in effect, so you test this way
Don't use ISNUMERIC out of the box though: it's unreliable. Use this:
ISNUMERIC(MyCOl + 'e0')
Note, if you have 6 decimal places then it will be lost on conversion to money
Other question with more info why: How to determine the field value which can not convert to (decimal, float,int) in SQL Server
Edit:
Can do it in one line if you want
ISNUMERIC(REPLACE(REPLACE(#chkCOl, '$', ''), ',', '') + 'e0')
Is it possible to store non-alphanumeric characters (more specifically line break characters) in a XML data type?
The code below illustrates my problem:
declare #a xml
declare #b nvarchar(max)
set #b = '<Entry Attrib="1'+CHAR(13)+'2" />'
print #b
set #a=convert(xml,#b,1)
set #b=convert(nvarchar, #a,1)
print #b
The output is:
<Entry Attrib="1
2" />
<Entry Attrib="1 2"/>
Is there any way I could keep the line break intact?
My actual problem is to store the value in a table (rather than a local variable), so maybe there's some setting for the corresponding XML column in my table that would do the job?
It would not be possible. The XML Data type is stored as an XML DOM Tree, not a string.
You would have to store it as a varchar instead if you want to keep whitespace.
My answer in the XSLT context should apply here:
XML parsed entities are often stored
in computer files which, for editing
convenience, are organized into lines.
These lines are typically separated by
some combination of the characters
carriage-return (#xD) and line-feed
(#xA).
So this might be what you are looking for:
set #b = '<Entry Attrib="1
2" />'
White space inside of an XML tag is not considered significant according to the XML specification, and will not be preserved. White space outside of the element will however:
declare #a xml
declare #b nvarchar(max)
set #b = '<Entry Attrib="12"> fo'+CHAR(13)+'o</Entry>'
print #b
set #a=convert(xml,#b,1)
set #b=convert(nvarchar(max), #a,1)
print #b
will output:
<Entry Attrib="12"> fo
o</Entry>
<Entry Attrib="12"> fo
o</Entry>