Bulk update with associative array in PostgreSQL - sql

Let's say I have an associative array (defined in another language) like so:
apply = {
'qwer': ['tju', 'snf', 'rjtj', 'sadgg']
'asdf': ['rtj', 'sfm', 'rtjt', 'adjdj']
...
'zxcv': ['qwr', 'trj', '3w4u', '3tt3']
}
And I have a table like so:
CREATE TABLE apples (
id integer,
name varchar(10),
key varchar(10),
value varchar(10)
);
I want to apply an update where if apples.value is in one of the lists of the apply variable, then set the apples.key to the key of the array. If apples.value was tju then set apples.key to qwer.
My current approach looks like this (mixing PostgreSQL with any procedural language):
for key in apply.keys:
UPDATE apples SET key=$key
FROM (SELECT unnest(array($apply[key])) AS value) AS update_table
WHERE value=update_table.value
I want to do this in a single statement.

As proof of concept for the given example, with the string formatted exactly as you display:
Demonstrating a prepared statement, like your client probably uses.
PREPARE my_update AS
UPDATE apples a
SET key = upd.key
FROM (
SELECT trim (split_part(key_val, ': ', 1), ' ''') AS key
, string_to_array(translate(split_part(key_val, ': ', 2), '[]''', ''), ', ') AS val_arr
FROM unnest(string_to_array(trim($1, E'{}\n'), E'\n')) key_val
) upd
WHERE a.value = ANY(upd.val_arr);
EXECUTE in the same session any number of times:
EXECUTE my_update($assoc_arr${
'qwer': ['tju', 'snf', 'rjtj', 'sadgg']
'asdf': ['rtj', 'sfm', 'rtjt', 'adjdj']
'zxcv': ['qwr', 'trj', '3w4u', '3tt3']
}$assoc_arr$);
SQL Fiddle.
Related:
Insert text with single quotes in PostgreSQL
Split given string and prepare case statement
But I would rather process the type in its original language and pass key and val_arr separately.

Related

How determine if range list contains specified integer

Product type table contains product types. Some ids may missing :
create table artliik (liiginrlki char(3) primary key);
insert into artliik values('1');
insert into artliik values('3');
insert into artliik values('4');
...
insert into artliik values('999');
Property table contais comma separated list of types.
create table strings ( id char(100) primary key, kirjeldLku chr(200) );
insert into strings values ('item1', '1,4-5' );
insert into strings values ('item2', '1,2,3,6-9,23-44,45' );
Type can specified as single integer, e.q 1,2,3 or as range like 6-9 or 23-44
List can contain both of them.
How to all properties for given type.
Query
select id
from artliik
join strings on ','||trim(strings.kirjeldLku)||',' like '%,'||trim(artliik.liiginrlki)||',%'
returns date for single integer list only.
How to change join so that type ranges in list like 6-9 are also returned?
Eq. f list contains 6-9, Type 6,7,8 and 9 shoud included in report.
Postgres 13 is used.
I would suggest a helper function similar to unnest that honors ranges.
Corrected function
create or replace function unnest_ranges(s text)
returns setof text language sql immutable as
$$
with t(x) as (select unnest(string_to_array(s, ',')))
select generate_series
(
split_part(x, '-', 1)::int,
case when x ~ '-' then split_part(x, '-', 2)::int else x::int end,
1
)::text
from t;
$$;
Then you can 'normalize' table strings and join.
select *
from artliik a
join (select id, unnest_ranges(kirjeldLku) from strings) as t(id, v)
on a.liiginrlki = v;
The use of a function definition is of course optional. I prefer it because the function is generic and reusable.
dbfiddle.uk demo will only works on pg14, since only pg14 have multirange data type. But customizeable icu collation works in pg13.
Collation doc: https://www.postgresql.org/docs/current/collation.html
Idea: create a multirange text data type that will sort numeric value based on their numerical value. like 'A-21' < 'A-123'.
CREATE COLLATION testcoll_numeric (
provider = icu,
locale = '#colNumeric=yes'
);
CREATE TYPE textrange AS RANGE (
subtype = text,
multirange_type_name = mulitrange_of_text,
COLLATION = testcoll_numeric
);
So
SELECT
mulitrange_of_text (textrange ('1'::text, '11'::text)) #> '9'::text AS contain_9;
should return true.
artliik table structure remain the same, but strings table need to change a bit.
CREATE temp TABLE strings (
id text PRIMARY KEY,
kirjeldLku mulitrange_of_text
);
then query it:
SELECT DISTINCT
strings.id
FROM
artliik,
strings
WHERE
strings.kirjeldLku #> liiginrlki::text
ORDER BY
1;

Checking if field contains multiple string in sql server

I am working on a sql database which will provide with data some grid. The grid will enable filtering, sorting and paging but also there is a strict requirement that users can enter free text to a text input above the grid for example
'Engine 1001 Requi' and that the result will contain only rows which in some columns contain all the pieces of the text. So one column may contain Engine, other column may contain 1001 and some other will contain Requi.
I created a technical column (let's call it myTechnicalColumn) in the table (let's call it myTable) which will be updated each time someone inserts or updates a row and it will contain all the values of all the columns combined and separated with space.
Now to use it with entity framework I decided to use a table valued function which accepts one parameter #searchQuery and it will handle it like this:
CREATE FUNCTION myFunctionName(#searchText NVARCHAR(MAX))
RETURNS #Result TABLE
( ... here come columns )
AS
BEGIN
DECLARE #searchToken TokenType
INSERT INTO #searchToken(token) SELECT value FROM STRING_SPLIT(#searchText,' ')
DECLARE #searchTextLength INT
SET #searchTextLength = (SELECT COUNT(*) FROM #searchToken)
INSERT INTO #Result
SELECT
... here come columns
FROM myTable
WHERE (SELECT COUNT(*) FROM #searchToken WHERE CHARINDEX(token, myTechnicalColumn) > 0) = #searchTextLength
RETURN;
END
Of course the solution works fine but it's kinda slow. Any hints how to improve its efficiency?
You can use an inline Table Valued Function, which should be quite a lot faster.
This would be a direct translation of your current code
CREATE FUNCTION myFunctionName(#searchText NVARCHAR(MAX))
RETURNS TABLE
AS RETURN
(
WITH searchText AS (
SELECT value token
FROM STRING_SPLIT(#searchText,' ') s(token)
)
SELECT
... here come columns
FROM myTable t
WHERE (
SELECT COUNT(*)
FROM searchText
WHERE CHARINDEX(s.token, t.myTechnicalColumn) > 0
) = (SELECT COUNT(*) FROM searchText)
);
GO
You are using a form of query called Relational Division Without Remainder and there are other ways to cut this cake:
CREATE FUNCTION myFunctionName(#searchText NVARCHAR(MAX))
RETURNS TABLE
AS RETURN
(
WITH searchText AS (
SELECT value token
FROM STRING_SPLIT(#searchText,' ') s(token)
)
SELECT
... here come columns
FROM myTable t
WHERE NOT EXISTS (
SELECT 1
FROM searchText
WHERE CHARINDEX(s.token, t.myTechnicalColumn) = 0
)
);
GO
This may be faster or slower depending on a number of factors, you need to test.
Since there is no data to test, i am not sure if the following will solve your issue:
-- Replace the last INSERT portion
INSERT INTO #Result
SELECT
... here come columns
FROM myTable T
JOIN #searchToken S ON CHARINDEX(S.token, T.myTechnicalColumn) > 0

Inserting space between the string (SQL Server)

I have a table called Manufacturers which contains a column Name with a data type of Varchar(100) that has 5761 entries such as XYZ(XYZ Corp).
I wish to insert a space between the Z and the (.
I have seen the STUFF and LEFT commands but can't seem to figure out if they apply to my scenario.
You can try the REPLACE (Transact-SQL)
function as shown below.
update <yourTableName>
set <yourColumName> = replace(<yourColumName>, '(', ' ( ')
where <put the conditions here>
Here is an implementation to you.
create table test (Name varchar(20))
insert into test values ('XYZ(XYZ Corp)')
--selecting before update
select * from test
--updating the record
update test
set Name = replace(Name, '(', ' (')
where name like '%(%' --Here you can add the conditions as you want to restrict the number of rows to be updated based on the available data or the patterns.
--selecting after update
select * from test
Live Demo
I would recommend:
update Manufacturers
set name = replace(name, '(', ' (')
where name like '%[^ ](%';
This will only update rows where there is not already a space before the open paren.
Note: If you have multiple open parens, it will update all of them. If that is an issue, ask a new question with appropriate sample data.

How to replace duplicate words in a column with just one word in SQL Server

I have a few million strings that relate to file paths in my database;
due to a third party program these paths have become nested like below:
C:\files\thirdparty\thirdparty\thirdparty\thirdparty\thirdparty\thirdparty\unique_bit_here\
I want update the entries so that thirdparty\thirdparty\etc becomes \thirdparty.
I have tried this code:
UPDATE table
SET Field = REPLACE(Field, 'tables\thirdparty\%thirdparty\%\', 'tables\thirdparty\')
WHILE EXISTS (SELECT * FROM table WHERE Field LIKE '%\thirdparty\thirdparty\%')
BEGIN
UPDATE table SET Field = REPLACE(Field, '\thirdparty\thirdparty\', '\thirdparty\')
END
So do you want something like this?
SELECT SUBSTRING('tables\thirdparty\%thirdparty\%\',0,CHARINDEX('\','tables\thirdparty\%thirdparty\%\',0)) + '\thirdparty\'
OR
UPDATE table
SET Field = REPLACE(Field, Field, (SELECT SUBSTRING(Field,0,CHARINDEX('\',Field,0)) + '\thirdparty\'))
You can avoid using a loop with the following technique:
Update TABLE
SET Field = left(Field,charindex('*',replace(Field, 'thirdparty\', '*'))-1)+'thirdparty\'+right(Field,charindex('*',reverse(replace(Field, 'thirdparty\', '*')))-1)
Its too late, but I just guess if I want replace a single word in repeating multiple time same word as he want. This will replace all with append a single time '\thirdparty'...
Check this.
Declare #table table(Field varchar(max))
insert into #table values('C:\files\thirdparty\thirdparty\thirdparty\thirdparty\thirdparty\thirdparty\unique_bit_here\')
,('C:\files\thirdparty\thirdparty\thirdparty\thirdparty\thirdparty\thirdparty\unique_bit_here\1')
,('C:\files\thirdparty\thirdparty\thirdparty\thirdparty\thirdparty\thirdparty\unique_bit_here\2')
,('C:\files\thirdparty\thirdparty\thirdparty\thirdparty\thirdparty\thirdparty\unique_bit_here\3')
,('C:\files\thirdparty\thirdparty\thirdparty\thirdparty\thirdparty\thirdparty\unique_bit_here\4')
UPDATE #table
SET Field = SUBSTRING (Field, 1, CHARINDEX('\thirdparty', Field ) ) + 'thirdparty\'
--replace (Field , 'thirdparty\' ,'')
+ reverse( SUBSTRING ( REVERSE(Field), 1, CHARINDEX(reverse('\thirdparty'), REVERSE(Field) )-2 ) )
--REPLACE(Field, 'tables\thirdparty\%thirdparty\%\', 'tables\thirdparty\')
select * from #table

Execute table valued function from row values

Given a table as below where fn contains the name of an existing table valued functions and param contains the param to be passed to the function
fn | param
----------------
'fn_one' | 1001
'fn_two' | 1001
'fn_one' | 1002
'fn_two' | 1002
Is there a way to get a resulting table like this by using set-based operations?
The resulting table would contain 0-* lines for each line from the first table.
param | resultval
---------------------------
1001 | 'fn_one_result_a'
1001 | 'fn_one_result_b'
1001 | 'fn_two_result_one'
1002 | 'fn_two_result_one'
I thought I could do something like (pseudo)
select t1.param, t2.resultval
from table1 t1
cross join exec sp_executesql('select * from '+t1.fn+'('+t1.param+')') t2
but that gives a syntax error at exec sp_executesql.
Currently we're using cursors to loop through the first table and insert into a second table with exec sp_executesql. While this does the job correctly, it is also the heaviest part of a frequently used stored procedure and I'm trying to optimize it. Changes to the data model would probably imply changes to most of the core of the application and that would cost more then just throwing hardware at sql server.
I believe that this should do what you need, using dynamic SQL to generate a single statement that can give you your results and then using that with EXEC to put them into your table. The FOR XML trick is a common one for concatenating VARCHAR values together from multiple rows. It has to be written with the AS [text()] for it to work.
--=========================================================
-- Set up
--=========================================================
CREATE TABLE dbo.TestTableFunctions (function_name VARCHAR(50) NOT NULL, parameter VARCHAR(20) NOT NULL)
INSERT INTO dbo.TestTableFunctions (function_name, parameter)
VALUES ('fn_one', '1001'), ('fn_two', '1001'), ('fn_one', '1002'), ('fn_two', '1002')
CREATE TABLE dbo.TestTableFunctionsResults (function_name VARCHAR(50) NOT NULL, parameter VARCHAR(20) NOT NULL, result VARCHAR(200) NOT NULL)
GO
CREATE FUNCTION dbo.fn_one
(
#parameter VARCHAR(20)
)
RETURNS TABLE
AS
RETURN
SELECT 'fn_one_' + #parameter AS result
GO
CREATE FUNCTION dbo.fn_two
(
#parameter VARCHAR(20)
)
RETURNS TABLE
AS
RETURN
SELECT 'fn_two_' + #parameter AS result
GO
--=========================================================
-- The important stuff
--=========================================================
DECLARE #sql VARCHAR(MAX)
SELECT #sql =
(
SELECT 'SELECT ''' + T1.function_name + ''', ''' + T1.parameter + ''', F.result FROM ' + T1.function_name + '(' + T1.parameter + ') F UNION ALL ' AS [text()]
FROM
TestTableFunctions T1
FOR XML PATH ('')
)
SELECT #sql = SUBSTRING(#sql, 1, LEN(#sql) - 10)
INSERT INTO dbo.TestTableFunctionsResults
EXEC(#sql)
SELECT * FROM dbo.TestTableFunctionsResults
--=========================================================
-- Clean up
--=========================================================
DROP TABLE dbo.TestTableFunctions
DROP TABLE dbo.TestTableFunctionsResults
DROP FUNCTION dbo.fn_one
DROP FUNCTION dbo.fn_two
GO
The first SELECT statement (ignoring the setup) builds a string which has the syntax to run all of the functions in your table, returning the results all UNIONed together. That makes it possible to run the string with EXEC, which means that you can then INSERT those results into your table.
A couple of quick notes though... First, the functions must all return identical result set structures - the same number of columns with the same data types (technically, they might be able to be different data types if SQL Server can always do implicit conversions on them, but it's really not worth the risk). Second, if someone were able to update your functions table they could use SQL injection to wreak havoc on your system. You'll need that to be tightly controlled and I wouldn't let users just enter in function names, etc.
You cannot access objects by referencing their names in a SQL statement. One method would be to use a case statement:
select t1.*,
(case when fn = 'fn_one' then dbo.fn_one(t1.param)
when fn = 'fn_two' then dbo.fn_two(t1.param)
end) as resultval
from table1 t1 ;
Interestingly, you could encapsulate the case as another function, and then do:
select t1.*, dbo.fn_generic(t1.fn, t1.param) as resultval
from table1 t1 ;
However, in SQL Server, you cannot use dynamic SQL in a user-defined function (defined in T-SQL), so you would still need to use case or similar logic.
Either of these methods is likely to be much faster than a cursor, because they do not require issuing multiple queries.