Related
I need to select a value within a comma delimited string using only SQL. Is this possible?
Data
A B C
1 Luigi Apple,Banana,Pineapple,,Citrus
I need to select specifically the 2nd item in column C, in this case banana. I need help. I cannot create new SQL functions, I can only use SQL. This is the as400 so the SQL is somewhat old tech.
Update..
With help from #Sandeep we were able to come up with
SELECT xmlcast(xmlquery('$x/Names/Name[2]' passing xmlparse(document CONCAT(CONCAT('<?xml version="1.0" encoding="UTF-8" ?><Names><Name>',REPLACE(ODWDATA,',','</Name><Name>')),'</Name></Names>')) as "x") as varchar(1000)) FROM ACL00
I'm getting this error
Keyword PASSING not expected. Valid tokens: ) ,.
New update. Problem solved by using UDF of Oracle's INSTR
I'm assuming db2 which I don't use, so the following syntax may not be bang on but the approach works.
In Oracle I'd use INSTR() and SUBSTR(), Google suggests LOCATE() and SUBSTR() for db2
Use LOCATE to get the position of the first comma, and use that value in SUBSTR to grab the end of YourColumn starting after the first comma
SUBSTR(YourColumn, LOCATE(YourColumn, ',') + 1)
You started with "Apple,Banana,Pineapple,,Citrus", you should now have "Banana,Pineapple,,Citrus", so we use LOCATE and SUBSTR again on the string returned above.
SUBSTR(SUBSTR(YourColumn, LOCATE(YourColumn, ',') + 1), 1, LOCATE(SUBSTR(YourColumn, LOCATE(YourColumn, ',') + 1), ',') - 1)
First SUBSTR is getting the right hand side of the string so we only need a start position parameter, second SUBSTR is grabbing the left side of the string so we need two, the start position and the length to return.
If you want 2nd item only than you can use substring function:
DECLARE #TABLE TABLE
(
A INT,
B VARCHAR(100),
C VARCHAR(100)
)
DECLARE #NTH INT = 3
INSERT INTO #TABLE VALUES (1,'Luigi','Apple,Banana,Pineapple,,Citrus')
SELECT REPLACE(REPLACE(CAST(CAST('<Name>'+ REPLACE(C,',','</Name><Name>') +'</Name>' AS XML).query('/Name[sql:variable("#NTH")]') AS VARCHAR(1000)),'<Name>',''),'</Name>','') FROM #TABLE
I am answering my own question now. It is impossible to do this with the built in functions within AS400
You have to create an UDF of Oracle's INSTR
Enter this within STRSQL it will create a new function called INSTRB
CREATE FUNCTION INSTRB (C1 VarChar(4000), C2 VarChar(4000), N integer, M integer)
RETURNS Integer
SPECIFIC INSTRBOracleBase
LANGUAGE SQL
CONTAINS SQL
NO EXTERNAL ACTION
DETERMINISTIC
BEGIN ATOMIC
DECLARE Pos, R, C2L Integer;
SET C2L = LENGTH(C2);
IF N > 0 THEN
SET (Pos, R) = (N, 0);
WHILE R < M AND Pos > 0 DO
SET Pos = LOCATE(C2,C1,Pos);
IF Pos > 0 THEN
SET (Pos, R) = (Pos + 1, R + 1);
END IF;
END WHILE;
RETURN (Pos - 1)*(1-SIGN(M-R));
ELSE
SET (Pos, R) = (LENGTH(C1)+N, 0);
WHILE R < M AND Pos > 0 DO
IF SUBSTR(C1,Pos,C2L) = C2 THEN
SET R = R + 1;
END IF;
SET Pos = Pos - 1;
END WHILE;
RETURN (Pos + 1)*(1-SIGN(M-R));
END IF;
END
Then to select the nth delimited value within a comma delimited string... in this case the 14th
use this query utilizing the new function
SELECT SUBSTRING(C,INSTRB(C,',',1,13)+1,INSTRB(C,',',1,14)-INSTRB(C,',',1,13)-1) FROM TABLE
A much prettier solution IMO would be to encapsulate a Recursive Common Table Expression (recursive CTE aka RCTE) of the data from the column C to generate a result TABLE [i.e. a User Defined Table Function (a Table UDF aka UDTF)] then use a Scalar Subselect to choose which effective record\row number.
select
a
, b
, ( select S.token_vc
from table( split_tokens(c) ) as S
where S.token_nbr = 2
) as "2nd Item of column C"
from The_Table /* in OP described with columns a,b,c but no DDL */
Yet prettier would be to make the result of that same RCTE a scalar value, so as to allow being invoked simply as a Scalar UDF with the effective row number [as another argument] defining specifically which element to select.
select
a
, b
, split_tokens(c, 2) as "2nd Item of column C"
from The_Table /* in OP described with columns a,b,c but no DDL */
The latter could be more efficient, limiting the row-data produced by the RCTE, to only the desired numbered token and those preceding numbered tokens. I can not comment on the efficiency with regard to impacts on CPU and storage as contrasted with any of the other answers offered, but my own experience with the temporary-storage implementation and the overall quickness of the RCTE results has been positive especially when other row selection limits the number of derived-table results that must be produced for the overall query request.
The UDF [and\or UDTF and the RCTE that implements them] is left as an exercise for the reader; mostly, because I do not have a system on a release that has support for recursive table expressions. If asked [e.g. in a comment to this answer], I could provide untested code source.
I have found the locate_in_string function to work very well in this case.
select substr(
c,
locate_in_string(c, ',')+1,
locate_in_string(c, ',', locate_in_string(c, ',')+1) - locate_in_string(c, ',')-1
) as fruit2
from ACL00 for read only with ur;
=# select row(0, 1) ;
row
-------
(0,1)
(1 row)
How to get 0 within the same query? I figured the below sort of working but is there any simple way?
=# select json_agg(row(0, 1))->0->'f1' ;
?column?
----------
0
(1 row)
No luck with array-like syntax [0].
Thanks!
Your row type is anonymous and therefore you cannot access its elements easily. What you can do is create a TYPE and then cast your anonymous row to that type and access the elements defined in the type:
CREATE TYPE my_row AS (
x integer,
y integer
);
SELECT (row(0,1)::my_row).x;
Like Craig Ringer commented in your question, you should avoid producing anonymous rows to begin with, if you can help it, and type whatever data you use in your data model and queries.
If you just want the first element from any row, convert the row to JSON and select f1...
SELECT row_to_json(row(0,1))->'f1'
Or, if you are always going to have two integers or a strict structure, you can create a temporary table (or type) and a function that selects the first column.
CREATE TABLE tmptable(f1 int, f2 int);
CREATE FUNCTION gettmpf1(tmptable) RETURNS int AS 'SELECT $1.f1' LANGUAGE SQL;
SELECT gettmpf1(ROW(0,1));
Resources:
https://www.postgresql.org/docs/9.2/static/functions-json.html
https://www.postgresql.org/docs/9.2/static/sql-expressions.html
The json solution is very elegant. Just for fun, this is a solution using regexp (much uglier):
WITH r AS (SELECT row('quotes, "commas",
and a line break".',null,null,'"fourth,field"')::text AS r)
--WITH r AS (SELECT row('',null,null,'')::text AS r)
--WITH r AS (SELECT row(0,1)::text AS r)
SELECT CASE WHEN r.r ~ '^\("",' THEN ''
WHEN r.r ~ '^\("' THEN regexp_replace(regexp_replace(regexp_replace(right(r.r, -2), '""', '\"', 'g'), '([^\\])",.*', '\1'), '\\"', '"', 'g')
ELSE (regexp_matches(right(r.r, -1), '^[^,]*'))[1] END
FROM r
When converting a row to text, PostgreSQL uses quoted CSV formatting. I couldn't find any tools for importing quoted CSV into an array, so the above is a crude text manipulation via mostly regular expressions. Maybe someone will find this useful!
With Postgresql 13+, you can just reference individual elements in the row with .fN notation. For your example:
select (row(0, 1)).f1; --> returns 0.
See https://www.postgresql.org/docs/13/sql-expressions.html#SQL-SYNTAX-ROW-CONSTRUCTORS
I'm using PostgreSQL 9.3.
The table partner.partner_statistic contains the following columns:
id reg_count
serial integer
I wrote the function convert(integer):
CREATE FUNCTION convert(d integer) RETURNS integer AS $$
BEGIN
--Do something and return integer result
END
$$ LANGUAGE plpgsql;
And now I need to write a function returned array of integers as follows:
CREATE FUNCTION res() RETURNS integer[] AS $$
<< outerblock >>
DECLARE
arr integer[]; --That array of integers I need to fill in depends on the result of query
r partner.partner_statistic%rowtype;
table_name varchar DEFAULT 'partner.partner_statistic';
BEGIN
FOR r IN
SELECT * FROM partner.partner_statistic offset 0 limit 100
LOOP
--
-- I need to add convert(r[reg_count]) to arr where r[id] = 0 (mod 5)
--
-- How can I do that?
END LOOP;
RETURN;
END;
$$ LANGUAGE plpgsql;
You don't need (and shouldn't use) PL/PgSQL loops for this. Just use an aggregate. I'm kind of guessing about what you mean by "where r[id] = 0 (mod 5) but I'm assuming you mean "where id is evenly divisible by 5". (Note that this is NOT the same thing as "every fifth row" because generated IDs have gaps).
Something like:
SELECT array_agg(r.reg_count)
FROM partner.partner_statistic
WHERE id % 5 = 0
LIMIT 100
probably meets your needs.
If you want to return the value, use RETURN QUERY SELECT ... or preferably use a simple sql language function.
If you want a dynamic table name, use:
RETURN QUERY EXECUTE format('
SELECT array_agg(r.reg_count)
FROM %I
WHERE id % 5 = 0
LIMIT 100', table_name::regclass);
I'm trying to implement an exponential moving average (EMA) on postgres, but as I check documentation and think about it the more I try the more confused I am.
The formula for EMA(x) is:
EMA(x1) = x1
EMA(xn) = α * xn + (1 - α) * EMA(xn-1)
It seems to be perfect for an aggregator, keeping the result of the last calculated element is exactly what has to be done here. However an aggregator produces one single result (as reduce, or fold) and here we need a list (a column) of results (as map). I have been checking how procedures and functions work, but AFAIK they produce one single output, not a column. I have seen plenty of procedures and functions, but I can't really figure out how does this interact with relational algebra, especially when doing something like this, an EMA.
I did not have luck searching the Internets so far. But the definition for an EMA is quite simple, I hope it is possible to translate this definition into something that works in postgres and is simple and efficient, because moving to NoSQL is going to be excessive in my context.
Thank you.
PD: here you can see an example:
https://docs.google.com/spreadsheet/ccc?key=0AvfclSzBscS6dDJCNWlrT3NYdDJxbkh3cGJ2S2V0cVE
You can define your own aggregate function and then use it with a window specification to get the aggregate output at each stage rather than a single value.
So an aggregate is a piece of state, and a transform function to modify that state for each row, and optionally a finalising function to convert the state to an output value. For a simple case like this, just a transform function should be sufficient.
create function ema_func(numeric, numeric) returns numeric
language plpgsql as $$
declare
alpha numeric := 0.5;
begin
-- uncomment the following line to see what the parameters mean
-- raise info 'ema_func: % %', $1, $2;
return case
when $1 is null then $2
else alpha * $2 + (1 - alpha) * $1
end;
end
$$;
create aggregate ema(basetype = numeric, sfunc = ema_func, stype = numeric);
which gives me:
steve#steve#[local] =# select x, ema(x, 0.1) over(w), ema(x, 0.2) over(w) from data window w as (order by n asc) limit 5;
x | ema | ema
-----------+---------------+---------------
44.988564 | 44.988564 | 44.988564
39.5634 | 44.4460476 | 43.9035312
38.605724 | 43.86201524 | 42.84396976
38.209646 | 43.296778316 | 41.917105008
44.541264 | 43.4212268844 | 42.4419368064
These numbers seem to match up to the spreadsheet you added to the question.
Also, you can define the function to pass alpha as a parameter from the statement:
create or replace function ema_func(state numeric, inval numeric, alpha numeric)
returns numeric
language plpgsql as $$
begin
return case
when state is null then inval
else alpha * inval + (1-alpha) * state
end;
end
$$;
create aggregate ema(numeric, numeric) (sfunc = ema_func, stype = numeric);
select x, ema(x, 0.5 /* alpha */) over (order by n asc) from data
Also, this function is actually so simple that it doesn't need to be in plpgsql at all, but can be just a sql function, although you can't refer to parameters by name in one of those:
create or replace function ema_func(state numeric, inval numeric, alpha numeric)
returns numeric
language sql as $$
select case
when $1 is null then $2
else $3 * $2 + (1-$3) * $1
end
$$;
This type of query can be solved with a recursive CTE - try:
with recursive cte as (
select n, x ema from my_table where n = 1
union all
select m.n, alpha * m.x + (1 - alpha) * cte.ema
from cte
join my_table m on cte.n = m.n - 1
cross join (select ? alpha) a)
select * from cte;
--$1 Stock code
--$2 exponential;
create or replace function fn_ema(text,numeric)
returns numeric as
$body$
declare
alpha numeric := 0.5;
var_r record;
result numeric:=0;
n int;
p1 numeric;
begin
alpha=2/(1+$2);
n=0;
for var_r in(select *
from stock_old_invest
where code=$1 order by stock_time desc)
loop
if n>0 then
result=result+(1-alpha)^n*var_r.price_now;
else
p1=var_r.price_now;
end if;
n=n+1;
end loop;
result=alpha*(result+p1);
return result;
end
$body$
language plpgsql volatile
cost 100;
alter function fn_ema(text,numeric)
owner to postgres;
I have a field (column in Oracle) called X that has values like "a1b2c3", "abc", "1ab", "123", "156"
how do I write an sql query that returns me only the X that hold pure numerical values = no letters? from the example above would be „123“ and „156“
select X
from myTable
where ...??
You can use the REGEXP_LIKE function as:
SELECT X
FROM myTable
WHERE REGEXP_LIKE(X, '^[[:digit:]]+$');
Sample run:
SQL> SELECT X FROM SO;
X
--------------------
12c
123
abc
a12
SQL> SELECT X FROM SO WHERE REGEXP_LIKE(X, '^[[:digit:]]+$');
X
--------------------
123
SQL>
If the only characters to consider are letters then you can do:
select X from myTable where upper(X) = lower(X)
But of course that won't filter out other characters, just letters.
Since Oracle 12c (at least) there has been a built-in function to check whether a character value is numeric: VALIDATE_CONVERSION
select X from myTable where validate_conversion(X as number) = 1
If you use Oracle 10 or higher you can use regexp functions as codaddict suggested. In earlier versions translate function will help you:
select * from tablename where translate(x, '.1234567890', '.') is null;
More info about Oracle translate function can be found here or in official documentation "SQL Reference"
UPD: If you have signs or spaces in your numbers you can add "+-" characters to the second parameter of translate function.
What about 1.1E10, +1, -0, etc? Parsing all possible numbers is trickier than many people think. If you want to include as many numbers are possible you should use the to_number function in a PL/SQL function. From http://www.oracle-developer.net/content/utilities/is_number.sql:
CREATE OR REPLACE FUNCTION is_number (str_in IN VARCHAR2) RETURN NUMBER IS
n NUMBER;
BEGIN
n := TO_NUMBER(str_in);
RETURN 1;
EXCEPTION
WHEN VALUE_ERROR THEN
RETURN 0;
END;
/
The complete list of the regexp_like and other regexp functions in Oracle 11.1:
http://66.221.222.85/reference/regexp.html
In your example:
SELECT X
FROM test
WHERE REGEXP_LIKE(X, '^[[:digit:]]$');
You can use following command -
LENGTH(TRIM(TRANSLATE(string1, '+-.0123456789', '')))
This will return NULL if your string1 is Numeric
your query would be -
select * from tablename
where LENGTH(TRIM(TRANSLATE(X, '+-.0123456789', ''))) is null