I have PostgreSQL SQL that should look for a backslash in a column called source_username and if it finds the backslash, it should replace the current value of the source_username column with the same value without the characters before the backslash.
For example:
before source_username: domain\username
after source_username: username
with os_user as (
select source_username from itpserver.managed_incidents mi;
),
osUserWithoutDomain as (
select (
case when (select * from os_user) ilike '%\\%' and (select position('-' in (select * from os_user))>= 1) and (select length((select * from os_user)) != (select position('-' in (select * from os_user))) + 1)
then (
select substring(
(select * from os_user),(select position('\' in (select * from os_user)) + 1),(select length((select * from os_user)) - 1)
))
else ((select * from os_user))
end
)
)
UPDATE itpserver.managed_incidents SET source_username = replace(source_username, (select * from os_user), (select * from osUserWithoutDomain)),
description = replace(description , (select * from os_user), (select * from osUserWithoutDomain)),
additional_info = replace(additional_info , (select * from os_user), (select * from osUserWithoutDomain)),
typical_behavior = replace(typical_behavior , (select * from os_user), (select * from osUserWithoutDomain)),
raw_description = replace(raw_description , (select * from os_user), (select * from osUserWithoutDomain));
This SQL works fine when I have only one row in the table.
If I have multiple rows, I need to specify the row that I want to work with by adding where id = <id>
I wish to iterate all the relevant rows (all the rows that source_username contains backslash) and on each row to perform the SQL above.
I tried to do this with LOOP:
create or replace function fetcher()
returns void as $$
declare
emp record;
begin
for emp in select *
from itpserver.managed_incidents
order by id
limit 10
loop
raise notice '%', emp.id;
<my sql> where id = emp.id
end loop;
end;
$$language plpgsql;
select fetcher();
However, I get an error because I don't think it likes the 'with' statement.
Any idea how can I do it?
It's far simpler than that. You need to use the SUBSTR and STRPOS functions. Take a look at the results of this query.
https://dbfiddle.uk/9-yPKn6E
with os_user (source_username) as (
select 'domain\username'
union select 'mydomain\joe'
union select 'janet'
)
select u.source_username
, strpos(u.source_username, '\')
, substr(u.source_username, strpos(u.source_username, '\') + 1)
from os_user u
source_username
strpos
substr
domain\username
7
username
janet
0
janet
mydomain\joe
9
joe
What you need is:
UPDATE itpserver.managed_incidents
SET source_username = substr(source_username, strpos(source_username, '\') + 1)
, description = replace(description , source_username, substr(source_username, strpos(source_username, '\') + 1))
, additional_info = replace(additional_info , source_username, substr(source_username, strpos(source_username, '\') + 1))
, typical_behavior = replace(typical_behavior , source_username, substr(source_username, strpos(source_username, '\') + 1))
, raw_description = replace(raw_description , source_username, substr(source_username, strpos(source_username, '\') + 1));
This is based on lengthy experience with SQL Server and some quick document searches for Postgresql. The UPDATE statement may not work as I expect.
SQL by design/default works on complete data sets. It thus eliminates LOOPS entirely from the language - they are not needed. (Well not quite there are recursive queries). Your task is accomplished in a single update statement with a simple regular expression. See documentation String Functions:
update managed_incidents
set source_username = regexp_replace(source_username,'.*\\(.*)','\1');
Demo here.
Main Take away: Drop procedural logic terminology (for, loop, if then, ...) from your SQL vocabulary. (you choose alternatives with case.)
Related
I have a column 'zips' with type 'text' in the table parcels.
User can fill either a single zip code, OR multiple comma separated zips, OR a range of zips separated by a hyphon.
Examples of possible datas are.
'10001'
'10002,10010,10015'
'10001,"10010-10025"'
I need to match the records with a zipcode '10015'.
eg:
select *
from parcels
where "10015" = ANY(string_to_array(parcels.zips, ','))
The Above code is working for the comma separated zips, but I am not sure about how to deal with the ranges.
I am looking for something like
select *
from parcels
where (
loop though `string_to_array(parcels.zips, ',')` and if iterating
variable contains '-', then 'where 10015 BETWEEN 10010 AND 10025'.
ELSE if zip doesn't contains '-', Then '10015' = '10001(other elements in the array)'
)
and combine the loop conditions with OR
try this :
SELECT *
FROM parcels p
CROSS JOIN LATERAL regexp_split_to_table (p.zips, ',') AS z
WHERE CASE
WHEN strpos (z, '-') > 0
THEN '10015' BETWEEN split_part (z, '-', 1) AND split_part (z, '-', 2)
ELSE z = '10015'
END
You can unnest the elements of the column and use them in an EXIST condition that checks for ranges:
select *
from parcels p
where exists (select *
from (
select split_part(trim(both '"' from z.zip), '-', 1) as from_zip,
split_part(trim(both '"' from z.zip), '-', 2) as to_zip
from unnest(string_to_array(p.zip_codes, ',')) as z(zip)
) x
where (x.to_zip = '' and x.from_zip = '10015')
or (x.to_zip <> '' and '10015' between x.from_zip and coalesce(x.to_zip, '10015'))
);
I would put this into a function to make that easier:
create function contains_zip(p_codes text, p_zip_code text)
returns boolean
as
$$
select exists
(select *
from (
select split_part(trim(both '"' from z.zip), '-', 1) as from_zip,
split_part(trim(both '"' from z.zip), '-', 2) as to_zip
from unnest(string_to_array(p_codes, ',')) as z(zip)
) x
where (x.to_zip = '' and x.from_zip = p_zip_code)
or (x.to_zip <> '' and p_zip_code between x.from_zip and coalesce(x.to_zip, p_zip_code))
);
$$
language sql
immutable;
Then it is as easy as:
select *
from parcels p
where contains_zip(p.zip_codes, '10015');
Online example
try to get your data in the format you need them, with some CTEs:
with _data as (
select * from (values(1,'10001'),(2,'10002,10010,10015'), (3,'10001,"10010-10025"')) as _vals (i,x)
),
_data2 as (
select
i,
unnest(string_to_array(x,','))as x
from _data
),
_data3 as (
select
i,
x,
replace(split_part(x,'-',1),'"','') as x1,
replace(split_part(x,'-',2),'"','') as x2
from _data2
)
select * from _data3
where
case when x2 = '' then x1::int = 10015 end
or
case when x2 <> '' then 10015 between(x1::int) and (x2::int) end
I'm converting a stored procedure from MySql to SQL Server. The procedure has one input parameter nvarchar/varchar which is a comma-separated string, e.g.
'1,2,5,456,454,343,3464'
I need to write a query that will retrieve the relevant rows, in MySql I'm using FIND_IN_SET and I wonder what the equivalent is in SQL Server.
I also need to order the ids as in the string.
The original query is:
SELECT *
FROM table_name t
WHERE FIND_IN_SET(id,p_ids)
ORDER BY FIND_IN_SET(id,p_ids);
The equivalent is like for the where and then charindex() for the order by:
select *
from table_name t
where ','+p_ids+',' like '%,'+cast(id as varchar(255))+',%'
order by charindex(',' + cast(id as varchar(255)) + ',', ',' + p_ids + ',');
Well, you could use charindex() for both, but the like will work in most databases.
Note that I've added delimiters to the beginning and end of the string, so 464 will not accidentally match 3464.
You would need to write a FIND_IN_SET function as it does not exist. The closet mechanism I can think of to convert a delimited string into a joinable object would be a to create a table-valued function and use the result in a standard in statement. It would need to be similar to:
DECLARE #MyParam NVARCHAR(3000)
SET #MyParam='1,2,5,456,454,343,3464'
SELECT
*
FROM
MyTable
WHERE
MyTableID IN (SELECT ID FROM dbo.MySplitDelimitedString(#MyParam,','))
And you would need to create a MySplitDelimitedString type table-valued function that would split a string and return a TABLE (ID INT) object.
A set based solution that splits the id's into ints and join with the base table which will make use of index on the base table id. I assumed the id would be an int, otherwise just remove the cast.
declare #ids nvarchar(100) = N'1,2,5,456,454,343,3464';
with nums as ( -- Generate numbers
select top (len(#ids)) row_number() over (order by (select 0)) n
from sys.messages
)
, pos1 as ( -- Get comma positions
select c.ci
from nums n
cross apply (select charindex(',', #ids, n.n) as ci) c
group by c.ci
)
, pos2 as ( -- Distinct posistions plus start and end
select ci
from pos1
union select 0
union select len(#ids) + 1
)
, pos3 as ( -- add row number for join
select ci, row_number() over (order by ci) as r
from pos2
)
, ids as ( -- id's and row id for ordering
select cast(substring(#ids, p1.ci + 1, p2.ci - p1.ci - 1) as int) id, row_number() over (order by p1.ci) r
from pos3 p1
inner join pos3 p2 on p2.r = p1.r + 1
)
select *
from ids i
inner join table_name t on t.id = i.id
order by i.r;
You can also try this by using regex to get the input values from comma separated string :
select * from table_name where id in (
select regexp_substr(p_ids,'[^,]+', 1, level) from dual
connect by regexp_substr(p_ids, '[^,]+', 1, level) is not null );
I have a table that contains text field with placeholders. Something like this:
Row Notes
1. This is some notes ##placeholder130## this ##myPlaceholder##, #oneMore#. End.
2. Second row...just a ##test#.
(This table contains about 1-5k rows on average. Average number of placeholders in one row is 5-15).
Now, I have a lookup table that looks like this:
Name Value
placeholder130 Dog
myPlaceholder Cat
oneMore Cow
test Horse
(Lookup table will contain anywhere from 10k to 100k records)
I need to find the fastest way to join those placeholders from strings to a lookup table and replace with value. So, my result should look like this (1st row):
This is some notes Dog this Cat, Cow. End.
What I came up with was to split each row into multiple for each placeholder and then join it to lookup table and then concat records back to original row with new values, but it takes around 10-30 seconds on average.
You could try to split the string using a numbers table and rebuild it with for xml path.
select (
select coalesce(L.Value, T.Value)
from Numbers as N
cross apply (select substring(Notes.notes, N.Number, charindex('##', Notes.notes + '##', N.Number) - N.Number)) as T(Value)
left outer join Lookup as L
on L.Name = T.Value
where N.Number <= len(notes) and
substring('##' + notes, Number, 2) = '##'
order by N.Number
for xml path(''), type
).value('text()[1]', 'varchar(max)')
from Notes
SQL Fiddle
I borrowed the string splitting from this blog post by Aaron Bertrand
SQL Server is not very fast with string manipulation, so this is probably best done client-side. Have the client load the entire lookup table, and replace the notes as they arrived.
Having said that, it can of course be done in SQL. Here's a solution with a recursive CTE. It performs one lookup per recursion step:
; with Repl as
(
select row_number() over (order by l.name) rn
, Name
, Value
from Lookup l
)
, Recurse as
(
select Notes
, 0 as rn
from Notes
union all
select replace(Notes, '##' + l.name + '##', l.value)
, r.rn + 1
from Recurse r
join Repl l
on l.rn = r.rn + 1
)
select *
from Recurse
where rn =
(
select count(*)
from Lookup
)
option (maxrecursion 0)
Example at SQL Fiddle.
Another option is a while loop to keep replacing lookups until no more are found:
declare #notes table (notes varchar(max))
insert #notes
select Notes
from Notes
while 1=1
begin
update n
set Notes = replace(n.Notes, '##' + l.name + '##', l.value)
from #notes n
outer apply
(
select top 1 Name
, Value
from Lookup l
where n.Notes like '%##' + l.name + '##%'
) l
where l.name is not null
if ##rowcount = 0
break
end
select *
from #notes
Example at SQL Fiddle.
I second the comment that tsql is just not suited for this operation, but if you must do it in the db here is an example using a function to manage the multiple replace statements.
Since you have a relatively small number of tokens in each note (5-15) and a very large number of tokens (10k-100k) my function first extracts tokens from the input as potential tokens and uses that set to join to your lookup (dbo.Token below). It was far too much work to look for an occurrence of any of your tokens in each note.
I did a bit of perf testing using 50k tokens and 5k notes and this function runs really well, completing in <2 seconds (on my laptop). Please report back how this strategy performs for you.
note: In your example data the token format was not consistent (##_#, ##_##, #_#), I am guessing this was simply a typo and assume all tokens take the form of ##TokenName##.
--setup
if object_id('dbo.[Lookup]') is not null
drop table dbo.[Lookup];
go
if object_id('dbo.fn_ReplaceLookups') is not null
drop function dbo.fn_ReplaceLookups;
go
create table dbo.[Lookup] (LookupName varchar(100) primary key, LookupValue varchar(100));
insert into dbo.[Lookup]
select '##placeholder130##','Dog' union all
select '##myPlaceholder##','Cat' union all
select '##oneMore##','Cow' union all
select '##test##','Horse';
go
create function [dbo].[fn_ReplaceLookups](#input varchar(max))
returns varchar(max)
as
begin
declare #xml xml;
select #xml = cast(('<r><i>'+replace(#input,'##' ,'</i><i>')+'</i></r>') as xml);
--extract the potential tokens
declare #LookupsInString table (LookupName varchar(100) primary key);
insert into #LookupsInString
select distinct '##'+v+'##'
from ( select [v] = r.n.value('(./text())[1]', 'varchar(100)'),
[r] = row_number() over (order by n)
from #xml.nodes('r/i') r(n)
)d(v,r)
where r%2=0;
--tokenize the input
select #input = replace(#input, l.LookupName, l.LookupValue)
from dbo.[Lookup] l
join #LookupsInString lis on
l.LookupName = lis.LookupName;
return #input;
end
go
return
--usage
declare #Notes table ([Id] int primary key, notes varchar(100));
insert into #Notes
select 1, 'This is some notes ##placeholder130## this ##myPlaceholder##, ##oneMore##. End.' union all
select 2, 'Second row...just a ##test##.';
select *,
dbo.fn_ReplaceLookups(notes)
from #Notes;
Returns:
Tokenized
--------------------------------------------------------
This is some notes Dog this Cat, Cow. End.
Second row...just a Horse.
Try this
;WITH CTE (org, calc, [Notes], [level]) AS
(
SELECT [Notes], [Notes], CONVERT(varchar(MAX),[Notes]), 0 FROM PlaceholderTable
UNION ALL
SELECT CTE.org, CTE.[Notes],
CONVERT(varchar(MAX), REPLACE(CTE.[Notes],'##' + T.[Name] + '##', T.[Value])), CTE.[level] + 1
FROM CTE
INNER JOIN LookupTable T ON CTE.[Notes] LIKE '%##' + T.[Name] + '##%'
)
SELECT DISTINCT org, [Notes], level FROM CTE
WHERE [level] = (SELECT MAX(level) FROM CTE c WHERE CTE.org = c.org)
SQL FIDDLE DEMO
Check the below devioblog post for reference
devioblog post
To get speed, you can preprocess the note templates into a more efficient form. This will be a sequence of fragments, with each ending in a substitution. The substitution might be NULL for the last fragment.
Notes
Id FragSeq Text SubsId
1 1 'This is some notes ' 1
1 2 ' this ' 2
1 3 ', ' 3
1 4 '. End.' null
2 1 'Second row...just a ' 4
2 2 '.' null
Subs
Id Name Value
1 'placeholder130' 'Dog'
2 'myPlaceholder' 'Cat'
3 'oneMore' 'Cow'
4 'test' 'Horse'
Now we can do the substitutions with a simple join.
SELECT Notes.Text + COALESCE(Subs.Value, '')
FROM Notes LEFT JOIN Subs
ON SubsId = Subs.Id WHERE Notes.Id = ?
ORDER BY FragSeq
This produces a list of fragments with substitutions complete. I am not an MSQL user, but in most dialects of SQL you can concatenate these fragments in a variable quite easily:
DECLARE #Note VARCHAR(8000)
SELECT #Note = COALESCE(#Note, '') + Notes.Text + COALSCE(Subs.Value, '')
FROM Notes LEFT JOIN Subs
ON SubsId = Subs.Id WHERE Notes.Id = ?
ORDER BY FragSeq
Pre-processing a note template into fragments will be straightforward using the string splitting techniques of other posts.
Unfortunately I'm not at a location where I can test this, but it ought to work fine.
I really don't know how it will perform with 10k+ of lookups.
how does the old dynamic SQL performs?
DECLARE #sqlCommand NVARCHAR(MAX)
SELECT #sqlCommand = N'PlaceholderTable.[Notes]'
SELECT #sqlCommand = 'REPLACE( ' + #sqlCommand +
', ''##' + LookupTable.[Name] + '##'', ''' +
LookupTable.[Value] + ''')'
FROM LookupTable
SELECT #sqlCommand = 'SELECT *, ' + #sqlCommand + ' FROM PlaceholderTable'
EXECUTE sp_executesql #sqlCommand
Fiddle demo
And now for some recursive CTE.
If your indexes are correctly set up, this one should be very fast or very slow. SQL Server always surprises me with performance extremes when it comes to the r-CTE...
;WITH T AS (
SELECT
Row,
StartIdx = 1, -- 1 as first starting index
EndIdx = CAST(patindex('%##%', Notes) as int), -- first ending index
Result = substring(Notes, 1, patindex('%##%', Notes) - 1)
-- (first) temp result bounded by indexes
FROM PlaceholderTable -- **this is your source table**
UNION ALL
SELECT
pt.Row,
StartIdx = newstartidx, -- starting index (calculated in calc1)
EndIdx = EndIdx + CAST(newendidx as int) + 1, -- ending index (calculated in calc4 + total offset)
Result = Result + CAST(ISNULL(newtokensub, newtoken) as nvarchar(max))
-- temp result taken from subquery or original
FROM
T
JOIN PlaceholderTable pt -- **this is your source table**
ON pt.Row = T.Row
CROSS APPLY(
SELECT newstartidx = EndIdx + 2 -- new starting index moved by 2 from last end ('##')
) calc1
CROSS APPLY(
SELECT newtxt = substring(pt.Notes, newstartidx, len(pt.Notes))
-- current piece of txt we work on
) calc2
CROSS APPLY(
SELECT patidx = patindex('%##%', newtxt) -- current index of '##'
) calc3
CROSS APPLY(
SELECT newendidx = CASE
WHEN patidx = 0 THEN len(newtxt) + 1
ELSE patidx END -- if last piece of txt, end with its length
) calc4
CROSS APPLY(
SELECT newtoken = substring(pt.Notes, newstartidx, newendidx - 1)
-- get the new token
) calc5
OUTER APPLY(
SELECT newtokensub = Value
FROM LookupTable
WHERE Name = newtoken -- substitute the token if you can find it in **your lookup table**
) calc6
WHERE newstartidx + len(newtxt) - 1 <= len(pt.Notes)
-- do this while {new starting index} + {length of txt we work on} exceeds total length
)
,lastProcessed AS (
SELECT
Row,
Result,
rn = row_number() over(partition by Row order by StartIdx desc)
FROM T
) -- enumerate all (including intermediate) results
SELECT *
FROM lastProcessed
WHERE rn = 1 -- filter out intermediate results (display only last ones)
I need a query like below,
select * from table1 where usr_id = user-id + '#' (i.e. if user-id is
1234, then value to be validated is '1234#')
Please let me know how to do this in SQL?
It seems you want to concatenate two strings. The way to do it depend on your DBMS:
-- SQL Server / Microsoft Access
SELECT * FROM table1 WHERE usr_id = user-id + '#'
-- Oracle
SELECT * FROM table1 WHERE usr_id = user-id || '#'
-- MySQL
SELECT * FROM table1 WHERE usr_id = CONCAT(user-id, '#')
Reference : SQL CONCATENATE (appending strings to one another)
Try this
SELECT * FROM table1 WHERE usr_id = CONVERT(VARCHAR(100), #ID) + '#'
OR
SELECT * FROM table1 WHERE REPLACE(usr_id , '#', '') = #ID
I am trying to create a delimitted string from the results of a query in DB2 on the iSeries (AS/400). I've done this in T-SQL, but can't find a way to do it here.
Here is my code in T-SQL. I'm looking for an equivelant in DB2.
DECLARE #a VARCHAR(1000)
SELECT #a = COALESCE(#a + ', ' + [Description], [Description])
FROM AP.Checkbooks
SELECT #a
If the descriptions in my table look like this:
Desc 1
Desc 2
Desc 3
Then it will return this:
Desc 1, Desc 2, Desc 3
Essentially you're looking for the equivalent of MySQL's GROUP_CONCAT aggregate function in DB2. According to one thread I found, you can mimic this behaviour by going through the XMLAGG function:
create table t1 (num int, color varchar(10));
insert into t1 values (1,'red'), (1,'black'), (2,'red'), (2,'yellow'), (2,'green');
select num,
substr( xmlserialize( xmlagg( xmltext( concat( ', ', color ) ) ) as varchar( 1024 ) ), 3 )
from t1
group by num;
This would return
1 red,black
2 red,yellow,green
(or should, if I'm reading things correctly)
You can do this using common table expressions (CTEs) and recursion.
with
cte1 as
(select description, row_number() over() as row_nbr from checkbooks),
cte2 (list, cnt, cnt_max) AS
(SELECT VARCHAR('', 32000), 0, count(description) FROM cte1
UNION ALL
SELECT
-- No comma before the first description
case when cte2.list = '' THEN RTRIM(CHAR(cte1.description))
else cte2.list || ', ' || RTRIM(CHAR(cte1.description)) end,
cte2.cnt + 1,
cte2.cnt_max
FROM cte1,cte2
WHERE cte1.row_nbr = cte2.cnt + 1 AND cte2.cnt < cte2.cnt_max ),
cte3 as
(select list from cte2
where cte2.cnt = cte2.cnt_max fetch first 1 row only)
select list from cte3;
I'm trying to do this in OLEDB and from what I understand you can't do this because you can't do anything fancy in SQL for OLEDB like declare variables or create a table. So I guess there is no way.
If you are running DB2 9.7 or higher, you can use LISTAGG function. Have a look here:
http://pic.dhe.ibm.com/infocenter/db2luw/v9r7/index.jsp?topic=%2Fcom.ibm.db2.luw.sql.ref.doc%2Fdoc%2Fr0058709.html