I have a table containing a column of raw email text including headers and message body. This may or may not include a url from my domain. If it does contain a url from my domain, I'd like to add a row to the result and increment the number of occurrences of that URL.
For example, the result should look like this:
Link Count
---- -----
a 19
b 5
c 1
What is a sensible approach to this?
I found a function on SQLMag (by Brian Moran) that may be of use:
Please note I haven't tested the function, so you might wanna do that yourself :)
CREATE function WordRepeatedNumTimes
(#SourceString varchar(8000),#TargetWord varchar(8000))
RETURNS int
AS
BEGIN
DECLARE #NumTimesRepeated int
,#CurrentStringPosition int
,#LengthOfString int
,#PatternStartsAtPosition int
,#LengthOfTargetWord int
,#NewSourceString varchar(8000)
SET #LengthOfTargetWord = len(#TargetWord)
SET #LengthOfString = len(#SourceString)
SET #NumTimesRepeated = 0
SET #CurrentStringPosition = 0
SET #PatternStartsAtPosition = 0
SET #NewSourceString = #SourceString
WHILE len(#NewSourceString) >= #LengthOfTargetWord
BEGIN
SET #PatternStartsAtPosition = CHARINDEX
(#TargetWord,#NewSourceString)
IF #PatternStartsAtPosition <> 0
BEGIN
SET #NumTimesRepeated = #NumTimesRepeated + 1
SET #CurrentStringPosition = #CurrentStringPosition + #PatternStartsAtPosition +
#LengthOfTargetWord
SET #NewSourceString = substring(#NewSourceString,
#PatternStartsAtPosition +
#LengthOfTargetWord, #LengthOfString)
END
ELSE
BEGIN
SET #NewSourceString = ''
END
END
RETURN #NumTimesRepeated
END
You can then use it in the following way:
DECLARE #link varchar(max)='http://YourLinkHere.com'
SELECT SUM(dbo.WordRepeatedNumTimes(field, #link))
FROM Table
The original article can be found here
Related
I have created a Function in SQL Server 2012 that I will use in a Check Constraint on a table.
The function works as expected if I do:
SELECT [dbo].[CheckValidCardnumberForShellTankingen] ('700678036658047691' ,'2925CA00-6DD5-4F9D-AB0E-AA15DBBD388B')
But when I try to set the expression in Check Constraint so:
([dbo].[CheckValidCardnumberForShellTankingen]([Volledig kaartnummer],[RollBackCode])=(1))
I get a Messaage: "Error validating constraint 'CK_MyConstraint'"
I use the Uniqueidentifier in a Where clause and the strange thing is if I replace the parameter with string containing the Uniqueidentifier I dont get this error.
Here is the Function:
-- =============================================
-- Author: Anders Pedersen
-- Create date: 2015-02-13
-- Description: Check of the Cardnumber of a transaction is valid.
-- =============================================
CREATE FUNCTION [dbo].[CheckValidCardnumberForShellTankingen]
(
-- Add the parameters for the function here
#Cardnumber NvarChar(50),
#RollBackCode NvarChar(200)
)
RETURNS BIT
AS
BEGIN
-- Declare the return variable here
DECLARE
#Result BIT
,#ResultLenght BIT
,#ResultPrefix BIT
,#CardLenght INT
,#SupplierID INT
,#UseCardnumber BIT
,#Prefix NvarChar(50)
-- Add the T-SQL statements to compute the return value here
SET #Result = 0
SET #ResultLenght = 0
SET #ResultPrefix = 0
SET #CardLenght = -1
SET #SupplierID = -1
SET #UseCardnumber = 0
SET #Prefix = ''
-- Get the UseCardnumber and the SupplierID
SELECT #UseCardnumber = C.UseCardNumber, #SupplierID = F.SupplierID
FROM Client C INNER JOIN
ClientFileUploads F ON C.ClientID = F.ClientID
WHERE F.RollBackCode = #RollBackCode
--WHERE F.RollBackCode = '2925CA00-6DD5-4F9D-AB0E-AA15DBBD388B'
-- Only carry out the check if the Client use Cards else set the check to True (1)
IF #UseCardnumber = 1
BEGIN
SELECT #CardLenght = [CardNumberLenght], #Prefix = ISNULL([Prefix],'') FROM [dbo].[Supplier] AS S WHERE S.SupplierID = #SupplierID
IF (#CardLenght IS NULL) OR (#CardLenght = 0)
BEGIN
SET #ResultLenght = 1
END
ELSE
BEGIN
IF (LEN(#Cardnumber) - #CardLenght)= 0
BEGIN
SET #ResultLenght = 1
END
ELSE
BEGIN
SET #ResultLenght = 0
END
END
IF SUBSTRING(#Cardnumber, 1, LEN(#Prefix)) = #Prefix
BEGIN
SET #ResultPrefix = 1
END
ELSE
BEGIN
SET #ResultPrefix = 0
END
IF ((#ResultLenght = 1) AND (#ResultPrefix = 1))
BEGIN
SET #Result = 1
END
ELSE
BEGIN
SET #Result = 0
END
END
ELSE
BEGIN
SET #Result = 1
END
-- Return the result of the function
RETURN #Result
END
GO
If #RollBackCode is a uniqueidentifier, I recommend making the parameter a uniqueidentifier and not a varchar.
As Rhys Jones points out, you shouldn't use a UDF in a check constraint.
See
https://dba.stackexchange.com/questions/22297/udf-in-check-constraint-downside
https://social.msdn.microsoft.com/Forums/sqlserver/en-US/078b720f-faac-425c-b51a-33bcecb263d2/check-constraint-with-udf-problem-with-lots-of-data?forum=transactsql
http://sqlblog.com/blogs/tibor_karaszi/archive/2009/12/17/be-careful-with-constraints-calling-udfs.aspx
If you need to check in a trigger and roll back -- SQL Server - After Insert/ For Insert - Rollback
This was my first attempt at a SQL function. I wrote it in VB and it works like a charm. When I translated it to SQL Server, it returns not what I expect. What the function is intended to do is to return a percentage match of two strings.
How I expected it to function is this:
accept two strings, compare the two, and based on my rating values, return a percentage of the match value...matchscore/max possiblescore
The length of the larger string is multiplied by 3. This is the max possiblescore.
Go character by character of the first string and find that character in the second string.
If the character is found in the same position in the second string, add three to the matchscore and move on to the next letter.
If the character is found in the second word, but not in the same position, add one to match score and move on to the next character.
If the character is not found in the second string, add nothing and move on to the next character.
Divide the matchscore by the max possiblescore. This returns a decimal value. I read RETURN only returns an integer, so I multiplied the division result by 100.
An example of what I expected is I compare "CAT" to "CART". My expected return is 7/12...0.58. Instead, I get 0. If I compare "CAT" to "CAT", I expect 9/9...1.00. Instead, I get 2.
(Note from 9/17/2014: I appreciate your input. I used what you suggested, and did one more major change, that doesn't affect what I asked about, other than getting the correct final answer is that I got rid of the second While Loop. Instead, I search for #strLetter in #strWord2. If it is found, then, I look to see if it is in the same position in #strWord2 as #strWord1. If it is, then I add 3, if not, I add 1. This sped up the function and made the count accurate.
Here is the code:
CREATE FUNCTION [dbo].[CompareWords]
(#strWord1 VARCHAR(2000), #strWord2 VARCHAR(2000))
RETURNS DECIMAL
AS
BEGIN
SET #strWord1 = UPPER(#strWord1)
SET #strWord2 = UPPER(#strWord2)
DECLARE #intLength INT
IF LEN(#strWord1) >= LEN(#strWord2)
BEGIN
SET #intLength = LEN(#strWord1)
END
ELSE
BEGIN
SET #intLength = LEN(#strWord2)
END
DECLARE #iWordLoop1 INT
DECLARE #iWordLoop2 INT
DECLARE #intWordLoop2 INT
DECLARE #intWordScore INT
DECLARE #intLetterScore INT
SET #intWordScore = 0
SET #intWordLoop2 = Len(#strWord2)
DECLARE #strLetter VARCHAR(1000)
DECLARE #count1 INT
SET #count1 = 0
SET #iWordLoop1 = Len(#strWord1)
WHILE (#count1 < #iWordLoop1)
BEGIN
SET #strLetter = SUBSTRING(#strWord1, #count1+1, 1)
SET #intLetterScore = 0
DECLARE #count2 INT
SET #count2 = 0
SET #iWordLoop2 = Len(#strWord2)
WHILE (#count2 < #iWordLoop2)
BEGIN
If #strLetter = SUBSTRING(#strWord2, #count2+1, 1)
BEGIN
If #iWordLoop1 = #iWordLoop2
BEGIN
SET #intLetterScore = 3
SET #iWordLoop2 = Len(#strWord2)
END
ELSE
BEGIN
SET #intLetterScore = 1
END
END
SET #intWordScore = #intWordScore + #intLetterScore
SET #count2 = (#count2 + 1)
END
SET #count1 = (#count1 + 1)
END
DECLARE #sinScore DEC
SET #sinScore = (#intWordScore / (3 * #intLength)) * 100
RETURN #sinSCore
END;
The most significant changes I made were to
reset the intLetterScore to 0 after it's been used in the intWordScore calculation. Without it being reset, the same value was being used each time the inner loop and the character was not matched.
move the multiplication by 100 into the
brackets in the calculation of sinScore.
As referred to in a previous post, because you are doing integer multiplication, the decimal portion is truncated from the calculation. By growing the wordScore by a factor of 100, it is much more likely to be larger than the length and yield a result which is non-zero.
Multiplying outside the brackets has the multiplies the integer result of the division score by length. If this answer is already zero, then the multiplication result is also zero.
Other changes I made are commented in the code: the variable intWordLoop2 has no effect on the calculation and can be removed; strLetter can be declared as a Char(1) instead of VarChar(1000).
CREATE FUNCTION [dbo].[CompareWords]
(#strWord1 VARCHAR(2000), #strWord2 VARCHAR(2000))
RETURNS DECIMAL
AS
BEGIN
SET #strWord1 = UPPER(#strWord1)
SET #strWord2 = UPPER(#strWord2)
--Set #intLength (maxLength as len of word1 or word2)
DECLARE #intLength INT --maxLength
IF LEN(#strWord1) >= LEN(#strWord2)
BEGIN
SET #intLength = LEN(#strWord1)
END
ELSE
BEGIN
SET #intLength = LEN(#strWord2)
END
DECLARE #iWordLoop1 INT, #iWordLoop2 INT--, #intWordLoop2 INT --This variable doesn't impact the calculation
DECLARE #intWordScore INT
DECLARE #intLetterScore INT
SET #intWordScore = 0
--SET #intWordLoop2 = Len(#strWord2)--this value is not used anywhere else, so removing makes no difference.
--DECLARE #strLetter VARCHAR(1000)
DECLARE #strLetter CHAR(1)--there is no need for 1000 characters since we're only ever assigning a single character to this
DECLARE #count1 INT
SET #count1 = 0
SET #iWordLoop1 = Len(#strWord1)
WHILE (#count1 < #iWordLoop1)
BEGIN
SET #strLetter = SUBSTRING(#strWord1, #count1+1, 1)
SET #intLetterScore = 0
DECLARE #count2 INT
SET #count2 = 0
SET #iWordLoop2 = Len(#strWord2)
WHILE (#count2 < #iWordLoop2)
BEGIN
If #strLetter = SUBSTRING(#strWord2, #count2+1, 1)
BEGIN
If #iWordLoop1 = #iWordLoop2
BEGIN
SET #intLetterScore = 3
SET #iWordLoop2 = Len(#strWord2)
END
ELSE
BEGIN
SET #intLetterScore = 1
END
END
SET #intWordScore = #intWordScore + #intLetterScore
SET #intLetterScore = 0
SET #count2 = (#count2 + 1)
END
SET #count1 = (#count1 + 1)
END
DECLARE #sinScore DEC
SET #sinScore = (#intWordScore*100 / (3 * #intLength))
RETURN #sinSCore
END;
select dbo.comparewords ('Cat','cart')
I am trying to extract an ID from a URL and running into some issues. The URL's will look something like this:
http://www.website.com/news/view.aspx?id=95
http://www.website.com/news/view.aspx?id=20&ReturnURL=%2fnews%2fview.aspx%3fid%3d20
I am trying to return back the number following "?id=" and nothing after the number. I will then convert it to an INT in reference to another table. Any suggestions as how to do this properly?
Use charindex to find the position of ?id and stuff to remove the characters that is before ?id. Then you use left to return the characters to the next &
declare #T table
(
URL varchar(100)
);
insert into #T values
('http://www.website.com/news/view.aspx?id=95'),
('http://www.website.com/news/view.aspx?id=20&ReturnURL=%2fnews%2fview.aspx%3fid%3d20');
select left(T2.URL, charindex('&', T2.URL) - 1) as ID
from #T as T
cross apply (select stuff(T.URL, 1, charindex('?id', T.URL) + 3, '')+'&') as T2(URL);
Here is an option that you can use when you want to find the value of any parameter value within a URL, it also supports parsing text values that contain a URL
DECLARE #Param varchar(50) = 'event'
DECLARE #Data varchar(8000) = 'User Logged into https://my.website.org/default.aspx?id=3066&event=49de&event=true from ip'
DECLARE #ParamIndex int = (select PatIndex('%'+#param+'=%', #Data)+LEN(#param)+1)
-- #ParamValueSubstring chops off everthing before the first instance of the parameter value
DECLARE #ParamValueSubstring varchar(8000) = SubString(#Data, #ParamIndex, 8000)
SELECT #ParamValueSubstring as ParamSubstring
DECLARE #SpaceIndex int = (SELECT CHARINDEX(' ', #ParamValueSubstring))-1
DECLARE #AmpIndex int = (SELECT CHARINDEX('&', #ParamValueSubstring))-1
DECLARE #valueLength int = -1
-- find first instance of ' ' or & after parameter value
IF #SpaceIndex = -1
SET #valueLength = #AmpIndex
ELSE IF #AmpIndex = -1
SET #valueLength = #SpaceIndex
ELSE IF #SpaceIndex < #AmpIndex
SET #valueLength = #SpaceIndex
ELSE
SET #valueLength = #AmpIndex
IF(#valueLength = -1) -- check to see if there was no space or '&' following the parameter value
BEGIN
SET #valueLength = 8000
END
select Left(#ParamValueSubstring, #valueLength) as ParamValue
-- approach similar to idea function found here http://blogs.lessthandot.com/index.php/DataMgmt/DataDesign/extracting-numbers-with-sql-server/
I'm not totally clear on what you're asking. Are you asking how to get the value of id from the url when you are in the asp.net application? Then in the code behind you can
In c#
string id = Request.QueryString["id"]; // returns null if id not found
Reference
From this SO question Try this for integers:
int id;
if (!int.TryParse(Request.QueryString["id"], out id))
{
// error case
}
You could do it in an SQL function, like this:
declare #URL varchar(100)
--set #URL = 'http://www.website.com/news/view.aspx?id=95'
set #URL = 'http://www.website.com/news/view.aspx?id=20&ReturnURL=%2fnews%2fview.aspx%3fid%3d20'
Set #URL = CASE charindex('&',#URL)
WHEN 0 then #URL else substring(#url,1,charindex('&',#URL)-1) end
select #URL,SUBSTRING(#URL,CHARINDEX('?id=',#URL)+4,99)
Both examples are in there, comment either one to see result
I am trying to set a column name to a declared variable but i keep getting invalid column name message. what is the right syntax? Here is the query
Declare #APPSHELTER Table
(tid int Identity(1,1),App_Id Numeric,PrimaryName Varchar(300),--EmployerAdress Varchar(500), rent_amt varchar(20),house_payment_amt varchar(20),ins_amt varchar(20),utilities_amt varchar(20),Trash_Collection_amt varchar(20),Sewerage_amt varchar(20),Telephone_amt varchar(20),water_amt varchar(20),other_amt varchar(20), total varchar(20), property_taxes_amt varchar(20),insurance_amt varchar(20), other_house_amt varchar(20), gas_amt varchar(20), elec_amt varchar(20), otherfuel_amt varchar(20))
DECLARE #rent_amt_h NUMERIC
DECLARE #house_payment_amt_h NUMERIC
DECLARE #insurance_amt_h NUMERIC
DECLARE #property_taxes_amt_h NUMERIC
DECLARE #Other_house_amt_h NUMERIC
DECLARE #gas_amt_u NUMERIC
DECLARE #elec_amt_u NUMERIC
DECLARE #otherfuel_amt_u NUMERIC
DECLARE #Trash_Collection_amt_u NUMERIC
DECLARE #Sewerage_amt_u NUMERIC
DECLARE #Telephone_amt_u NUMERIC
DECLARE #water_amt_u NUMERIC
DECLARE #other_amt_u NUMERIC
DECLARE #total_u NUMERIC
insert into #APPSHELTER(App_Id,PrimaryName,rent_amt,house_payment_amt,ins_amt,utilities_amt,Trash_Collection_amt,Sewerage_amt,Telephone_amt,water_amt,other_amt,total, property_taxes_amt, insurance_amt, other_house_amt, gas_amt, elec_amt, otherfuel_amt )
select #app_id,
ISNULL((select top 1 replace(first_name,'''','''''') + ' ' + isnull(replace(middle_name,'''',''''''),'') + ' '+replace(last_name,'''','''''')
from app_member (nolock)
where app_id = #app_id and msn=1),'') AS PrimaryName,
isnull(rent_amt,'0' ) AS rent_amt,
isnull(house_payment_amt,'0') AS house_payment_amt,
isnull((insurance_amt+property_taxes_amt),'0') AS ins_amt,
--isnull(HC_Costs_amt,'0') AS utilities_amt,
isnull(gas_amt,'0') + isnull(elec_amt,'0') + isnull(otherfuel_amt,'0') as utilities_amt,
isnull(Trash_Collection_amt,'0') AS Trash_Collection_amt,
isnull(Sewerage_amt,'0') AS Sewerage_amt,
isnull(Telephone_amt,'0') AS Telephone_amt,
isnull(water_amt,'0') AS water_amt,
isnull(other_amt,'0') + isnull(other_house_amt,'0') AS other_amt,
isnull(total,'0') AS total,
isnull(property_taxes_amt,'0' ) AS property_taxes_amt,
isnull(insurance_amt,'0' ) AS insurance_amt,
isnull(other_house_amt,'0' ) AS other_house_amt,
isnull(gas_amt,'0' ) AS gas_amt,
isnull(elec_amt,'0' ) AS elec_amt,
isnull(otherfuel_amt,'0' ) AS otherfuel_amt
from Ext_App_Group_Other_Expenses APP_DEP (nolock)
WHERE APP_DEP.APP_ID=#APP_ID
SET #rent_amt_h = 'rent_amt'
SET #house_payment_amt_h = 'house_payment_amt'
SET #insurance_amt_h = 'insurance_amt'
SET #property_taxes_amt_h = 'property_taxes_amt'
SET #Other_house_amt_h = 'other_house_amt'
SET #gas_amt_u = 'gas_amt'
SET #elec_amt_u = 'elec_amt'
SET #otherfuel_amt_u = 'otherfuel_amt'
SET #Trash_Collection_amt_u = 'Trash_Collection_amt'
SET #Sewerage_amt_u = 'Sewerage_amt'
SET #Telephone_amt_u = 'Telephone_amt'
SET #water_amt_u = 'water_amt'
SET #other_amt_u = 'other_amt'
SET #total_u = 'total'
DECLARE #APPSHELTER_COUNT INT
if (rent_amt!=0 or house_payment_amt!=0 or insurance_amt != 0 or property_taxes_amt != 0 or gas_amt != 0 or elec_amt != 0 or otherfuel_amt != 0 or Trash_Collection_amt != 0 or Sewerage_amt != 0 or Telephone_amt != 0 or water_amt != 0 or other_house_amt != 0 or other_amt != 0 or total !=0)
begin
SET #APPSHELTER_COUNT = (select Count(APP_ID) FROM ext_app_group_other_expenses (nolock) WHERE APP_ID = #App_Id )
end
else
begin
SET #APPSHELTER_COUNT = 0
end
Actually, I am trying the check whether the values in these text boxes are null or not. If not I have to set the count!
2 things:
(1). I think this is erroneous - I don't see these variables declared anywhere....
if (rent_amt!=0 or house_payment_amt!=0 or insurance_amt != 0 or property_taxes_amt != 0 or gas_amt != 0 or elec_amt != 0 or otherfuel_amt != 0 or Trash_Collection_amt != 0 or Sewerage_amt != 0 or Telephone_amt != 0 or water_amt != 0 or other_house_amt != 0 or other_amt != 0 or total !=0)
(2). You defined a bunch of variables at the top #rent_amt_h, #house_payment_amt_h, etc as NUMERIC, but then at the bottom you are setting them to a string value. This will also throw an error - if you are trying to say that these are your column names, then this is not the way to do it - you aren't selecting it anywhere, either. Please clarify what you are hoping to achieve...
Please find below simplified code which seemed to meet your requirements. To test it change values from 0 to 1 in insert statement:
-- Dynamic SQL command string variable
DECLARE #cmdSQL VARCHAR(1000)
-- Column names
DECLARE #rent_amt_h VARCHAR(20) = 'App_Id'
DECLARE #house_payment_amt_h VARCHAR(20) = 'PrimaryName'
-- Count variable
DECLARE #APPSHELTER_COUNT INT
-- Make SQL command
SET #cmdSQL = '
Declare #APPSHELTER Table
(App_Id Numeric,PrimaryName Varchar(300))
insert into #APPSHELTER(App_Id,PrimaryName) values (0,0)
SELECT * FROM #APPSHELTER WHERE ' + #rent_amt_h + ' !=0 OR ' + #house_payment_amt_h + ' != 0'
-- Execute dynamic SQL
EXEC(#cmdsql)
IF ##ROWCOUNT != 0
SET #APPSHELTER_COUNT = 1
ELSE
SET #APPSHELTER_COUNT = 0
SELECT #APPSHELTER_COUNT
I have a column in my database table which has lows of text and lows of rows. In the text, a url needs to be changed for every row. In each row the url can exist more than once.
Can I use the replace function to change all the urls in the text field in my database table without affecting the rest of the text in that same column?
Thanks
Use REPLACE()
UPDATE table SET text = REPLACE(text, 'from', 'to')
Make precise from: like with http://url_from.com/ to http://url_to.com/
Based on article posted at https://web.archive.org/web/20150521050102/http://sqlserver2000.databases.aspfaq.com:80/how-do-i-handle-replace-within-an-ntext-column-in-sql-server.html
I needed to write a find and replace for CHAR(146) ` in a text field. Above article for fort nText and the same solution for text worked with nText with the following changes:
- VARCHAR(32) from nVARCHAR(32)
- use #lenOldString = DATALENGTH(#oldString) instead of SET #lenOldString = DATALENGTH(#oldString)/2.
DECLARE
#TextPointer BINARY(16),
#TextIndex INT,
#oldString VARCHAR(32),
#newString VARCHAR(32),
#lenOldString INT,
#currentDataID INT;
SET #oldString = '’';
SET #newString = '''';
IF CHARINDEX(#oldString, #newString) > 0
BEGIN
PRINT 'Quitting to avoid infinite loop.';
END
ELSE
BEGIN
--Need the for nText fields
--SET #lenOldString = DATALENGTH(#oldString)/2
--Use this for text fields
SET #lenOldString = DATALENGTH(#oldString)
DECLARE irows CURSOR
LOCAL FORWARD_ONLY STATIC READ_ONLY FOR
SELECT
DataID
FROM
dbo.tbData
WHERE
PATINDEX('%'+#oldString+'%', TextData) > 0;
OPEN irows;
FETCH NEXT FROM irows INTO #currentDataID;
WHILE (##FETCH_STATUS = 0)
BEGIN
SELECT
#TextPointer = TEXTPTR(TextData),
#TextIndex = PATINDEX('%'+#oldString+'%', TextData)
FROM
dbo.tbData
WHERE
DataID = #currentDataID;
SELECT #TextPointer, #TextIndex
WHILE
(
SELECT
PATINDEX('%'+#oldString+'%', TextData)
FROM
dbo.tbData
WHERE
DataID = #currentDataID
) > 0
BEGIN
SELECT
#TextIndex = PATINDEX('%'+#oldString+'%', TextData)-1
FROM
dbo.tbData
WHERE
DataID = #currentDataID;
UPDATETEXT dbo.tbData.TextData #TextPointer #TextIndex #lenOldString #newString;
END
FETCH NEXT FROM irows INTO #currentDataID;
END
CLOSE irows;
DEALLOCATE irows;
END