I'm writing some functions for mapping a Postgres database. The functions are used to create dynamic queries for a javascript API.
I've come across a particular query that truncates the output of a column and I cannot determine why.
The width of the column in the following query seems to be truncated to 63 characters:
SELECT
CASE
WHEN t.column_name ~ '_[uid]*$' AND t.has_fn THEN
format(
'format(''%s/%%s'', %s) AS %s',
array_to_string(ARRAY['',t.fk_schema,t.fk_name],'/'),
t.column_name,
regexp_replace(t.column_name, '_[uid]*$', '_link')
)
ELSE t.column_name
END AS column
FROM core.map_type() t; -- core.map_type() is a set returning function
This query is used to create a select list for another query, but the string produced by format() is truncated to 63 characters.
However, if I add || '' to the ELSE branch of the CASE statement, the problem goes away:
SELECT
CASE
WHEN t.column_name ~ '_[uid]*$' AND t.has_fn THEN
format(
'format(''%s/%%s'', %s) AS %s',
array_to_string(ARRAY['',t.fk_schema,t.fk_name],'/'),
t.column_name,
regexp_replace(t.column_name, '_[uid]*$', '_link')
)
ELSE t.column_name || '' -- add empty string here
END AS column
FROM core.map_type() t; -- core.map_type() is a set returning function
Having a column truncated like this is a worrisome problem. The fix here is a total hack and does not feel like a real solution.
Why is this happening? How can it be fixed?
In this case, the value of t.column_name in the ELSE branch of the CASE statement is of type name. name is a type used internally by Postgres for naming things. It has a fixed length of 64 bytes.
The column is truncated to suit the length of the ELSE branch.
Casting t.column_name to text will fix the problem:
SELECT
CASE
WHEN t.column_name ~ '_[uid]*$' AND t.has_fn THEN
format(
'format(''%s/%%s'', %s) AS %s',
array_to_string(ARRAY['',t.fk_schema,t.fk_name],'/'),
t.column_name,
regexp_replace(t.column_name, '_[uid]*$', '_link')
)
ELSE t.column_name::text
END AS column
FROM core.map_type() t;
Related
SET v_ArrayLength= ARRAY_COUNT(ARRAY_SPLIT(p_SeasonsStr, ','));
FOR v_ArrayPos in 1 .. v_ArrayLength LOOP
IF (v_ArrayPos<>v_ArrayLength) THEN
SET v_SeasonsStrs= v_SeasonsStrs||GET_VALUE_VARCHAR(ARRAY_SPLIT(p_SeasonsStr, ','),v_ArrayPos )||'_'||v_ArrayPos||',';
ELSE
SET v_SeasonsStrs= v_SeasonsStrs||GET_VALUE_VARCHAR(ARRAY_SPLIT(p_SeasonsStr, ','),v_ArrayPos )||'_'||v_ArrayPos;
END IF;
END LOOP;
how i can MIGRATE THIS CODE FROM NETEZZA TO DB2?
It would be good especially for those who are not familiar with Netezza functions, if you provided some data sample in p_SeasonsStr and the result expected in v_SeasonsStr.
The code above probably can be amended just to a single select statement in Db2 for LUW:
select listagg(tok||'_'||seq, ',') within group (order by seq)
into v_SeasonsStrs
from xmltable('for $id in tokenize($s, ",") return <i>{string($id)}</i>'
passing
-- 'str1,str2,str3'
p_SeasonsStr
as "s"
columns
seq for ordinality
, tok varchar(50) path '.'
) t;
If you comment out the rows with a variable and a parameter and uncomment the commented out row, and run the statement you got, the result is: str1_1,str2_2,str3_3
I want to convert null values to ' '. But when I use this code I got null values as 'NULL':
SELECT NVL(column_a, ' ') FROM table_a
If your GUI displays "NULL" for those values, it is the GUI setting, not Oracle value. Set it to something else (e.g. nothing, in your case).
I had a SQL procedure that increments through each row and and pads some trailing zeros on values depending on the length of the value after a decimal point. Trying to carry this over to a PSQL environment I realized there was a lot of syntax differences between SQL and PSQL. I managed to make the conversion over time but I am still getting a syntax error and cant figure out why. Can someone help me figure out why this wont run? I am currently running it in PGadmin if that makes any difference.
DO $$
DECLARE
counter integer;
before decimal;
after decimal;
BEGIN
counter := 1;
WHILE counter <> 2 LOOP
before = (select code from table where ID = counter);
after = (SELECT SUBSTRING(code, CHARINDEX('.', code) + 1, LEN(code)) as Afterward from table where ID = counter);
IF before = after
THEN
update table set code = before + '.0000' where ID = counter;
ELSE
IF length(after) = 1 THEN
update table set code = before + '000' where ID = counter;
ELSE IF length(after) = 2 THEN
update table set code = before + '00' where ID = counter;
ELSE IF length(after) = 3 THEN
update table set code = before + '0' where ID = counter;
ELSE
select before;
END IF;
END IF;
counter := counter + 1;
END LOOP
END $$;
Some examples of the input/output of the intended result:
Input 55.5 > Output 55.5000
Input 55 > Output 55.0000
Thanks for your help,
Justin
There is no need for a function or even an update on the table to format values when displaying them.
Assuming the values are in fact numbers stored in a decimal or float column, all you need to do is to apply the to_char() function when retrieving them:
select to_char(code, 'FM999999990.0000')
from data;
This will output 55.5000 or 55.0000
The drawback of the to_char() function is that you need to anticipate the maximum number of digits of that can occur. If you have not enough 9 in the format mask, the output will be something like #.###. But as too many digits in the format mask don't hurt, I usually throw a lot into the format mask.
For more information on formatting functions, please see the manual: https://www.postgresql.org/docs/current/static/functions-formatting.html#FUNCTIONS-FORMATTING-NUMERIC-TABLE
If you insist on storing formatted data, you can use to_char() to update the table:
update the_table
set code = to_char(code::numeric, 'FM999999990.0000');
Casting the value to a number will of course fail if there a non-numeric values in the column.
But again: I strong recommend to store numbers as numbers, not as strings.
If you want to compare this to a user input, it's better to convert the user input to a proper number and compare that to the (number) values stored in the database.
The string matching that you are after doesn't actually require a function either. Using substring() with a regex will do that:
update the_table
set code = code || case length(coalesce(substring(code from '\.[0-9]*$'), ''))
when 4 then '0'
when 3 then '00'
when 2 then '000'
when 1 then '0000'
when 0 then '.0000'
else ''
end
where length(coalesce(substring(code from '\.[0-9]*$'), '')) < 5;
substring(code from '\.[0-9]*$') extracts everything the . followed by numbers that is at the end of the string. So for 55.0 it returns .0 for 55.50 it returns .50 if there is no . in the value, then it returns null that's why the coalesce is needed.
The length of that substring tells us how many digits are present. Depending on that we can then append the necessary number of zeros. The case can be shortened so that not all possible length have to be listed (but it's not simpler):
update the_table
set code = code || case length(coalesce(substring(code from '\.[0-9]*$'), ''))
when 0 then '.0000'
else lpad('0', 5- length(coalesce(substring(code from '\.[0-9]*$'), '')), '0')
end
where length(coalesce(substring(code from '\.[0-9]*$'), '')) < 5;
Another option is to use the position of the . inside the string to calculate the number of 0 that need to be added:
update the_table
set code =
code || case
when strpos(code, '.') = 0 then '0000'
else rpad('0', 4 - (length(code) - strpos(code, '.')), '0')
end
where length(code) - strpos(code, '.') < 4;
Regular expressions are quite expensive not using them will make this faster. The above will however only work if there is always at most one . in the value.
But if you can be sure that every value can be cast to a number, the to_char() method with a cast is definitely the most robust one.
To only process rows where the code columns contains correct numbers, you can use a where clause in the SQL statement:
where code ~ '^[0-9]+(\.[0-9][0-9]?)?$'
To change the column type to numeric:
alter table t alter column code type numeric
Using Postgresql 8.4, how can I right-pad a string with blanks without truncating it when it's too long?
The problem is that rpad truncates the string when it is actually longer than number of characters to pad. Example:
SELECT rpad('foo', 5); ==> 'foo ' -- fine
SELECT rpad('foo', 2); ==> 'fo' -- not good, I want 'foo' instead.
The shortest solution I found doesn't involve rpad at all:
SELECT 'foo' || repeat(' ', 5-length('foo')); ==> 'foo ' -- fine
SELECT 'foo' || repeat(' ', 2-length('foo')); ==> 'foo' -- fine, too
but this looks ugly IMHO. Note that I don't actually select the string 'foo' of course, instead I select from a column:
SELECT colname || repeat(' ', 30-length(colname)) FROM mytable WHERE ...
Is there a more elegant solution?
found a slightly more elegant solution:
SELECT greatest(colname,rpad(colname, 2));
eg:
SELECT greatest('foo',rpad('foo', 5)); -- 'foo '
SELECT greatest('foo',rpad('foo', 2)); -- 'foo'
.
To explain how it works:
rpad('foo',5) = 'foo ' which is > 'foo' (greatest works with strings as well as numbers)
rpad('foo',2) = 'fo' which is < 'foo', so 'foo' is selected by greatest function.
if you want left-padded words you cant use greatest because it compares left-to-right (eg 'oo' with 'foo') and in some cases this will be greater or smaller depending on the string. I suppose you could reverse the string and use the rpad and reverse it back, or just use the original solution which works in both cases.
If you don't want to write that repeat business all the time, just write your own function for it. Something like this:
create or replace function rpad_upto(text, int) returns text as $$
begin
if length($1) >= $2 then
return $1;
end if;
return rpad($1, $2);
end;
$$ language plpgsql;
or this:
create or replace function rpad_upto(text, int) returns text as $$
select $1 || repeat(' ', $2 - length($1));
$$ language sql;
Then you can say things like:
select rpad_upto(colname, 30) from mytable ...
You might want to consider what you want rpad_upto(null, n) to produce while you're at it. Both versions of rpad_upto above will return NULL if $1 is NULL but you can tweak them to return something else without much difficulty.
how about this
select case when length(col) < x then rpad(col, x)
else col
end
from table
Assuming efficiency is not your biggest concern here:
select regexp_replace(format('%5s', 'foo'), '(\s*)(\S*)', '\2\1')
format() left-pads the string to the desired width
then regexp_replace moves any leading spaces to the end.
I guess that would fail if you have leading spaces in the strings and you want to preserve them. Also note that format() doesn't return null on null params.
PostgreSQL statement below is to right pad three place values and alter column data type to text for column 'columnname.' I used pycharm IDE to help construct statement. The statement will pad with 000.
I have been looking for a while to solve the same issue, except for left pad and I thought I would share.
alter table 'schema.tablename' alter column 'columnname' type text using rpad('columnname'::text,3,'0')
In a SQL query on Oracle 10g, I need to determine whether a string is numeric or not. How can I do this?
You can use REGEXP_LIKE:
SELECT 1 FROM DUAL
WHERE REGEXP_LIKE('23.9', '^\d+(\.\d+)?$', '')
You ca try this:
SELECT LENGTH(TRIM(TRANSLATE(string1, ' +-.0123456789', ' '))) FROM DUAL
where string1 is what you're evaluating. It will return null if numeric. Look here for further clarification
I don't have access to a 10G instance for testing, but this works in 9i:
CREATE OR REPLACE FUNCTION is_numeric (p_val VARCHAR2)
RETURN NUMBER
IS
v_val NUMBER;
BEGIN
BEGIN
IF p_val IS NULL OR TRIM (p_val) = ''
THEN
RETURN 0;
END IF;
SELECT TO_NUMBER (p_val)
INTO v_val
FROM DUAL;
RETURN 1;
EXCEPTION
WHEN OTHERS
THEN
RETURN 0;
END;
END;
SELECT is_numeric ('333.5') is_numeric
FROM DUAL;
I have assumed you want nulls/empties treated as FALSE.
As pointed out by Tom Kyte in http://asktom.oracle.com/pls/apex/f?p=100:11:0::::P11_QUESTION_ID:7466996200346537833, if you're using the built-in TO_NUMBER in a user defined function, you may need a bit of extra trickery to make it work.
FUNCTION is_number(x IN VARCHAR2)
RETURN NUMBER
IS
PROCEDURE check_number (y IN NUMBER)
IS
BEGIN
NULL;
END;
BEGIN
PRAGMA INLINE(check_number, 'No');
check_number(TO_NUMBER(x);
RETURN 1;
EXCEPTION
WHEN INVALID_NUMBER
THEN RETURN 0;
END is_number;
The problem is that the optimizing compiler may recognize that the result of the TO_NUMBER is not used anywhere and optimize it away.
Says Tom (his example was about dates rather then numbers):
the disabling of function inlining will make it do the call to
check_date HAS to be made as a function call - making it so that the
DATE has to be pushed onto the call stack. There is no chance for the
optimizing compiler to remove the call to to_date in this case. If the
call to to_date needed for the call to check_date fails for any
reason, we know that the string input was not convertible by that date
format.
Here is a method to determine numeric that can be part of a simple query, without creating a function. Accounts for embedded spaces, +- not the first character, or a second decimal point.
var v_test varchar2(20);
EXEC :v_test := ' -24.9 ';
select
(case when trim(:v_test) is null then 'N' ELSE -- only banks, or null
(case when instr(trim(:v_test),'+',2,1) > 0 then 'N' ELSE -- + sign not first char
(case when instr(trim(:v_test),'-',2,1) > 0 then 'N' ELSE -- - sign not first char
(case when instr(trim(:v_test),' ',1,1) > 0 then 'N' ELSE -- internal spaces
(case when instr(trim(:v_test),'.',1,2) > 0 then 'N' ELSE -- second decimal point
(case when LENGTH(TRIM(TRANSLATE(:v_test, ' +-.0123456789',' '))) is not null then 'N' ELSE -- only valid numeric charcters.
'Y'
END)END)END)END)END)END) as is_numeric
from dual;
I found that the solution
LENGTH(TRIM(TRANSLATE(string1, ' +-.0123456789', ' '))) is null
allows embedded blanks ... it accepts "123 45 6789" which for my purpose is not a number.
Another level of trim/translate corrects this. The following will detect a string field containing consecutive digits with leading or trailing blanks such that to_number(trim(string1)) will not fail
LENGTH(TRIM(TRANSLATE(translate(trim(string1),' ','X'), '0123456789', ' '))) is null
For integers you can use the below. The first translate changes spaces to be a character and the second changes numbers to be spaces. The Trim will then return null if only numbers exist.
TRIM(TRANSLATE(TRANSLATE(TRIM('1 2 3d 4'), ' ','#'),'0123456789',' ')) is null