sql function to return table of names and values given a querystring - sql

Anyone have a t-sql function that takes a querystring from a url and returns a table of name/value pairs?
eg I have a value like this stored in my database:
foo=bar&baz=qux&x=y
and I want to produce a 2-column (key and val) table (with 3 rows in this example), like this:
name | value
-------------
foo | bar
baz | qux
x | y
UPDATE: there's a reason I need this in a t-sql function; I can't do it in application code. Perhaps I could use CLR code in the function, but I'd prefer not to.
UPDATE: by 'querystring' I mean the part of the url after the '?'. I don't mean that part of a query will be in the url; the querystring is just used as data.

create function dbo.fn_splitQuerystring(#querystring nvarchar(4000))
returns table
as
/*
* Splits a querystring-formatted string into a table of name-value pairs
* Example Usage:
select * from dbo.fn_splitQueryString('foo=bar&baz=qux&x=y&y&abc=')
*/
return (
select 'name' = SUBSTRING(s,1,case when charindex('=',s)=0 then LEN(s) else charindex('=',s)-1 end)
, 'value' = case when charindex('=',s)=0 then '' else SUBSTRING(s,charindex('=',s)+1,4000) end
from dbo.fn_split('&',#querystring)
)
go
Which utilises this general-purpose split function:
create function dbo.fn_split(#sep nchar(1), #s nvarchar(4000))
returns table
/*
* From https://stackoverflow.com/questions/314824/
* Splits a string into a table of values, with single-char delimiter.
* Example Usage:
select * from dbo.fn_split(',', '1,2,5,2,,dggsfdsg,456,df,1,2,5,2,,dggsfdsg,456,df,1,2,5,2,,')
*/
AS
RETURN (
WITH Pieces(pn, start, stop) AS (
SELECT 1, 1, CHARINDEX(#sep, #s)
UNION ALL
SELECT pn + 1, stop + 1, CHARINDEX(#sep, #s, stop + 1)
FROM Pieces
WHERE stop > 0
)
SELECT pn,
SUBSTRING(#s, start, CASE WHEN stop > 0 THEN stop-start ELSE 4000 END) AS s
FROM Pieces
)
go
Ultimately letting you do something like this:
select name, value
from dbo.fn_splitQuerystring('foo=bar&baz=something&x=y&y&abc=&=whatever')

I'm sure TSQL could be coerced to jump through this hoop for you, but why not parse the querystring in your application code where it most probably belongs?
Then you can look at this answer for what others have done to parse querystrings into name/value pairs.
Or this answer.
Or this.
Or this.

Please don't encode your query strings directly in URLs, for security reasons: anyone can easily substitute any old query to gain access to information they shouldn't have -- or worse, "DROP DATABASE;". Checking for suspicious "keywords" or things like quote characters is not a solution -- creative hackers will work around these measures, and you'll annoy everyone whose last name is "O'Reilly."
Exceptions: in-house-only servers or public https URLS. But even then, there's no reason why you can't build the SQL query on the client side and submit it from there.

Related

TSQL - Remove list of parameters from Text

I have a field in a SQL table which has text looks something like this:
'The Employee <PARAM1> was replaced with <PARAM2> and was given the new IP address <PARAM3> with limited access <PARAM4>. <PARAM2> loves the new role'
I want to remove all the <PARAMs> from the text and just show as below using TSQL
'The Employee was replaced with and was given the new IP address with limited access. loves the new role'
What is the best way to do that?
One way to solve this is to generate a common table expression that will hold the start position and length of each <PARAMn> in the string. To do that, you can use a single common table expression but I've done it with three so that the process is easy to understand.
Please note that I'm assuming the string only contains < and > as params separators - so there is no < or > chars in the content. If that's not the case, it's still solvable but the solution would need some changes.
You start with a numbers (tally) cte that starts with 1 and ends with the length of your string.
Then another cte to get the start position of each <PARAMn>.
A third cte is used to get the length of each <PARAMn> (since I'm assuming you are not limited to 10 parameters in a string, so you can have <PARAM12> or even <PARAM105>).
Then, create a query that will update the original string and remove the <PARAMn> one by one.
-- Test data:
DECLARE #str nvarchar(1000) = 'The Employee <PARAM1> was replaced with <PARAM2> and was given the new IP address <PARAM3> with limited access <PARAM4>. <PARAM2> loves the new role';
-- The numbers (Tally) cte:
WITH Tally AS
(
SELECT TOP(LEN(#Str)) ROW_NUMBER() OVER(ORDER BY ##SPID) As N
FROM sys.objects A CROSS JOIN sys.objects B
), -- The StartPosition cte contains the position of each < char in the string
StartPosition AS
(
SELECT DISTINCT
CHARINDEX('<', #Str, N) As Start
FROM Tally
WHERE CHARINDEX('<', #Str, N) > 0
), -- the Length cte contains both the start position and the length of each <PARAMn> in the string
Length As
(
SELECT Start,
CHARINDEX('>', #Str, Start) - Start + 1 As Length
FROM StartPosition
)
-- Use STUFF to remove `<PARAMn>`. Note the order by is critical to remove from end to start.
SELECT #Str = STUFF(#Str, Start, Length, '')
FROM Length
ORDER BY Start DESC
-- verify the results:
SELECT #Str
Result:
The Employee was replaced with and was given the new IP address with limited access . loves the new role
You might note that the results have places with double spaces where the <PARAMn> used to be - that can be solved by using the technique Gordon Linoff shows in this answer.

SQL Server - Select column that contains query string and split values into anothers 'columns'

I need to do a select in a column that contains a query string like:
user_id=300&company_id=201503&status=WAITING OPERATION&count=1
I want to perform a select and break each value in a new column, something like:
user_id | company_id | status | count
300 | 201503 | WAITING OPERATION | 1
How can i do it in SQL Server without use procs?
I've tried a function:
CREATE FUNCTION [xpto].[SplitGriswold]
(
#List NVARCHAR(MAX),
#Delim1 NCHAR(1),
#Delim2 NCHAR(1)
)
RETURNS TABLE
AS
RETURN
(
SELECT
Val1 = PARSENAME(Value,2),
Val2 = PARSENAME(Value,1)
FROM
(
SELECT REPLACE(Value, #Delim2, '&') FROM
(
SELECT LTRIM(RTRIM(SUBSTRING(#List, [Number],
CHARINDEX(#Delim1, #List + #Delim1, [Number]) - [Number])))
FROM (SELECT Number = ROW_NUMBER() OVER (ORDER BY name)
FROM sys.all_objects) AS x
WHERE Number <= LEN(#List)
AND SUBSTRING(#Delim1 + #List, [Number], LEN(#Delim1)) = #Delim1
) AS y(Value)
) AS z(Value)
);
GO
Execution:
select QueryString
from User.Log
CROSS APPLY notifier.SplitGriswold(REPLACE(QueryString, ' ', N'ŏ'), N'ŏ', '&') AS t;
But it returns me only one column with all inside:
QueryString
user_id=300&company_id=201503&status=WAITING OPERATION&count=1
Thanks in advance.
I've had to do this many times before, and you're in luck! Since you only have 3 delimiters per string, and that number is fixed, you can use SQL Server's PARSENAME function to do it. That's far less ugly than the best alternative (using the XML parsing stuff). Try this (untested) query (replace TABLE_NAME and COLUMN_NAME with the appropriate names):
SELECT
PARSENAME(REPLACE(COLUMN_NAME,'&','.'),1) AS 'User',
PARSENAME(REPLACE(COLUMN_NAME,'&','.'),2) AS 'Company_ID',
PARSENAME(REPLACE(COLUMN_NAME,'&','.'),3) AS 'Status',
PARSENAME(REPLACE(COLUMN_NAME,'&','.'),4) AS 'Count',
FROM TABLE_NAME
That'll get you the results in the form "user_id=300", which is far and away the hard part of what you want. I'll leave it to you to do the easy part (drop the stuff before the "=" sign).
NOTE: I can't remember if PARSENAME will freak out over the illegal name character (the "=" sign). If it does, simply nest another REPLACE in there to turn it into something else, like an underscore.
You need to use SQL SUBSTRING as part of your select statement. You would first need to build the first row, then use a UNION to return the second row.

Looking for a scalar function to find the last occurrence of a character in a string

Table FOO has a column FILEPATH of type VARCHAR(512). Its entries are absolute paths:
FILEPATH
------------------------------------------------------------
file://very/long/file/path/with/many/slashes/in/it/foo.xml
file://even/longer/file/path/with/more/slashes/in/it/baz.xml
file://something/completely/different/foo.xml
file://short/path/foobar.xml
There's ~50k records in this table and I want to know all distinct filenames, not the file paths:
foo.xml
baz.xml
foobar.xml
This looks easy, but I couldn't find a DB2 scalar function that allows me to search for the last occurrence of a character in a string. Am I overseeing something?
I could do this with a recursive query, but this appears to be overkill for such a simple task and (oh wonder) is extremely slow:
WITH PATHFRAGMENTS (POS, PATHFRAGMENT) AS (
SELECT
1,
FILEPATH
FROM FOO
UNION ALL
SELECT
POSITION('/', PATHFRAGMENT, OCTETS) AS POS,
SUBSTR(PATHFRAGMENT, POSITION('/', PATHFRAGMENT, OCTETS)+1) AS PATHFRAGMENT
FROM PATHFRAGMENTS
)
SELECT DISTINCT PATHFRAGMENT FROM PATHFRAGMENTS WHERE POS = 0
I think what you're looking for is the LOCATE_IN_STRING() scalar function. This is what Info Center has to say if you use a negative start value:
If the value of the integer is less than zero, the search begins at
LENGTH(source-string) + start + 1 and continues for each position to
the beginning of the string.
Combine that with the LENGTH() and RIGHT() scalar functions, and you can get what you want:
SELECT
RIGHT(
FILEPATH
,LENGTH(FILEPATH) - LOCATE_IN_STRING(FILEPATH,'/',-1)
)
FROM FOO
One way to do this is by taking advantage of the power of DB2s XQuery engine. The following worked for me (and fast):
SELECT DISTINCT XMLCAST(
XMLQuery('tokenize($P, ''/'')[last()]' PASSING FILEPATH AS "P")
AS VARCHAR(512) )
FROM FOO
Here I use tokenize to split the file path into a sequence of tokens and then select the last of these tokens. The rest is only conversion from SQL to XML types and back again.
I know that the problem from the OP was already solved but I decided to post the following anyway to hopefully help others like me that land here.
I came across this thread while searching for a solution to my similar problem which had the exact same requirement but was for a different kind of database that was also lacking the REVERSE function.
In my case this was for a OpenEdge (Progress) database, which has a slightly different syntax. This made the INSTR function available to me that most Oracle typed databases offer.
So I came up with the following code:
SELECT
SUBSTRING(
foo.filepath,
INSTR(foo.filepath, '/',1, LENGTH(foo.filepath) - LENGTH( REPLACE( foo.filepath, '/', '')))+1,
LENGTH(foo.filepath))
FROM foo
However, for my specific situation (being the OpenEdge (Progress) database) this did not result into the desired behaviour because replacing the character with an empty char gave the same length as the original string. This doesn't make much sense to me but I was able to bypass the problem with the code below:
SELECT
SUBSTRING(
foo.filepath,
INSTR(foo.filepath, '/',1, LENGTH( REPLACE( foo.filepath, '/', 'XX')) - LENGTH(foo.filepath))+1,
LENGTH(foo.filepath))
FROM foo
Now I understand that this code won't solve the problem for T-SQL because there is no alternative to the INSTR function that offers the Occurence property.
Just to be thorough I'll add the code needed to create this scalar function so it can be used the same way like I did in the above examples.
-- Drop the function if it already exists
IF OBJECT_ID('INSTR', 'FN') IS NOT NULL
DROP FUNCTION INSTR
GO
-- User-defined function to implement Oracle INSTR in SQL Server
CREATE FUNCTION INSTR (#str VARCHAR(8000), #substr VARCHAR(255), #start INT, #occurrence INT)
RETURNS INT
AS
BEGIN
DECLARE #found INT = #occurrence,
#pos INT = #start;
WHILE 1=1
BEGIN
-- Find the next occurrence
SET #pos = CHARINDEX(#substr, #str, #pos);
-- Nothing found
IF #pos IS NULL OR #pos = 0
RETURN #pos;
-- The required occurrence found
IF #found = 1
BREAK;
-- Prepare to find another one occurrence
SET #found = #found - 1;
SET #pos = #pos + 1;
END
RETURN #pos;
END
GO
To avoid the obvious, when the REVERSE function is available you do not need to create this scalar function and you can just get the required result like this:
SELECT
SUBSTRING(
foo.filepath,
LEN(foo.filepath) - CHARINDEX('\', REVERSE(foo.filepath))+2,
LEN(foo.filepath))
FROM foo
You could just do it in a single statement:
select distinct reverse(substring(reverse(FILEPATH), 1, charindex('/', reverse(FILEPATH))-1))
from filetable

Search a database for a sentence separated by a delimiter in SQL

I currently have knowledge base articles in a database, and the system separates the keywords relating to the article by a semicolon.
On the front facing form, I have a text box limited to 100 characters. How do I create a SQL query that looks at the words within the text box and starts matching those words to the keywords within the database?
I basically want to create a search a little like Google's.
I am currently using LIKE '%{token}%'. However, this is not good enough.
If you have an application layer that in-turn interacts with the database, then you should generate that portion of the query, searching for each word at the beginning, in the middle and at the end of the sentence column of each record, for each word in the search text.
(
sentence LIKE 'token;%'
OR sentence like '%;token;%'
OR sentence like '%;token'
)
E.g.
Search text = "search google";
Query:
(
sentence LIKE 'search;%'
OR sentence like '%;search;%'
OR sentence like '%;search'
)
AND
(
sentence LIKE 'google;%'
OR sentence like '%;google;%'
OR sentence like '%;google'
)
If I understood you correctly, I had a similar problem.
I solved it by stored function "split". The function splits the parameter by delimiters and returns a table with one column.
This allows you to join the returned table column with your article keywords.
Here is a bit more generic version of the code
FUNCTION [dbo].[Split] (#s varchar(512))
RETURNS table
AS
RETURN (
WITH Pieces(pn, start, stop) AS (
SELECT 1, 1, dbo.GetNextDelimiter(#s,0)
UNION ALL
SELECT pn + 1, stop + 1, dbo.GetNextDelimiter(#s, stop + 1)
FROM Pieces
WHERE stop > 0
),
Parts(pn,s,start,[end]) as
(SELECT pn,
SUBSTRING(#s, start, CASE WHEN (stop > 0) THEN stop-start ELSE 512 END) AS s, start, stop
FROM Pieces)
SELECT pn as PartNumber,s as StringPart,(start - 1) start,([end] - 1) [end] FROM Parts WHERE s != ''
)
The delimiter function :
FUNCTION [dbo].[GetNextDelimiter](#string nvarchar(max), #startFrom int)
RETURNS int
AS
BEGIN
-- Declare the return variable here
DECLARE #response int
SELECT #response = MIN(delimiterIndex)
FROM
(SELECT CHARINDEX(' ', #string, #startFrom)
UNION
SELECT CHARINDEX('-', #string, #startFrom)
UNION
SELECT CHARINDEX('''', #string, #startFrom)) as Delimiters(delimiterIndex)
WHERE delimiterIndex != 0
IF #response is NULL
SET #response = 0
RETURN #response
END
This would allow you to use not only semicolons, but any type and number of delimiters.
From here it's simple
Select * from Articles WHERE Token in Split(#Input)

SQL strip text and convert to integer

In my database (SQL 2005) I have a field which holds a comment but in the comment I have an id and I would like to strip out just the id, and IF possible convert it to an int:
activation successful of id 1010101
The line above is the exact structure of the data in the db field.
And no I don't want to do this in the code of the application, I actually don't want to touch it, just in case you were wondering ;-)
This should do the trick:
SELECT SUBSTRING(column, PATINDEX('%[0-9]%', column), 999)
FROM table
Based on your sample data, this that there is only one occurence of an integer in the string and that it is at the end.
I don't have a means to test it at the moment, but:
select convert(int, substring(fieldName, len('activation successful of id '), len(fieldName) - len('activation successful of id '))) from tableName
Would you be open to writing a bit of code? One option, create a CLR User Defined function, then use Regex. You can find more details here. This will handle complex strings.
If your above line is always formatted as 'activation successful of id #######', with your number at the end of the field, then:
declare #myColumn varchar(100)
set #myColumn = 'activation successful of id 1010102'
SELECT
#myColumn as [OriginalColumn]
, CONVERT(int, REVERSE(LEFT(REVERSE(#myColumn), CHARINDEX(' ', REVERSE(#myColumn))))) as [DesiredColumn]
Will give you:
OriginalColumn DesiredColumn
---------------------------------------- -------------
activation successful of id 1010102 1010102
(1 row(s) affected)
select cast(right(column_name,charindex(' ',reverse(column_name))) as int)
CAST(REVERSE(LEFT(REVERSE(#Test),CHARINDEX(' ',REVERSE(#Test))-1)) AS INTEGER)
-- Test table, you will probably use some query
DECLARE #testTable TABLE(comment VARCHAR(255))
INSERT INTO #testTable(comment)
VALUES ('activation successful of id 1010101')
-- Use Charindex to find "id " then isolate the numeric part
-- Finally check to make sure the number is numeric before converting
SELECT CASE WHEN ISNUMERIC(JUSTNUMBER)=1 THEN CAST(JUSTNUMBER AS INTEGER) ELSE -1 END
FROM (
select right(comment, len(comment) - charindex('id ', comment)-2) as justnumber
from #testtable) TT
I would also add that this approach is more set based and hence more efficient for a bunch of data values. But it is super easy to do it just for one value as a variable. Instead of using the column comment you can use a variable like #chvComment.
If the comment string is EXACTLY like that you can use replace.
select replace(comment_col, 'activation successful of id ', '') as id from ....
It almost certainly won't be though - what about unsuccessful Activations?
You might end up with nested replace statements
select replace(replace(comment_col, 'activation not successful of id ', ''), 'activation successful of id ', '') as id from ....
[sorry can't tell from this edit screen if that's entirely valid sql]
That starts to get messy; you might consider creating a function and putting the replace statements in that.
If this is a one off job, it won't really matter. You could also use a regex, but that's quite slow (and in any case mean you now have 2 problems).