How to remove the blank spaces between two lines in a column - sql

I have a column that returns the following output:
PERSON_NUMBER FEEDBACK
13636 -Very attentive during our sessions
-Very interative session, questions
were asked and answered well.
Nice turn over
Debates were good
I want the output of FEEDBACK Column to look like:
PERSON_NUMBER FEEDBACK
13636 -Very attentive during our sessions
-Very interative session, questions
were asked and answered well.
Nice turn over
Debates were good
I.e. the extra blank spaces between the lines should be removed.

You could do a regex replacement on (?:\r?\n){2,} and replace with just a single CR?LF:
SELECT PERSON_NUMBER,
REGEXP_REPLACE(FEEDBACK,
'(' || chr(13) || '?' || chr(10) || '){2,}',
chr(13) || chr(10)) AS FEEDBACK
FROM yourTable;

If you need to remove all the lines that look empty (e.g. that may also contain spaces, tabs or other non-printable characters), then you may use negation of the [:graph:] class in the regex to find such rows and replace them with empty row and then replace all newline repetitions with single one.
Below is the code:
with a as (
select q'[-Very attentive during our sessions
-Very interative session, questions
were asked and answered well.]' || chr(13) || chr(10) || q'[
]' || chr(10) || q'[
]' || chr(10) || chr(10) || q'[Nice turn over]' as q
from dual
)
select
regexp_replace(
regexp_replace(
q,
/*
Replace LF followed by any non-printable sequence that ends with newline
with single newline
*/
chr(10) || '[^[:graph:]]*(' || chr(13) || '?' || chr(10) || ')',
chr(10) || '\1'
),
/*Then replace newline repetitions*/
'(' || chr(13) || '?' || chr(10) || ')+',
'\1'
) as q
from a
q
-Very attentive during our sessions-Very interative session, questions  were asked and answered well.Nice turn over
db<>fiddle here

Related

Oracle SQL RTRIM and replacing CHR 13,10,160,9 still leaving space at the end [duplicate]

This question already has answers here:
Oracle regexp_replace - removing trailing spaces
(3 answers)
Closed 10 months ago.
The original query returned result.
It had what looks like two spaces after the period.
I tried RTRIM and it removes ONE of the spaces result.
I then tried to replace the CHR characters with no luck.
rtrim(replace(COLUMN,CHR(13)||CHR(10)||CHR(160)||CHR(9),'')) as FORMATTED_COLUMN
What other things can I try?
To make your current approach work, you would have to use REGEXP_REPLACE with an alternation:
REGEXP_REPLACE(COLUMN, '[' || CHR(13) || '|' || CHR(10) || '|' || CHR(160) || '|' || CHR(9) || ']+$', '')
If you wanted to use plain REPLACE, then you would have to chain function calls:
RTRIM(REPLACE(REPLACE(REPLACE(REPLACE(COLUMN, CHR(13), ''), CHR(10), ''), CHR(160), ''), CHR(9), ''))
But then the replacements would happen everywhere, and not just at the end of the string.

Format Column Headers during Pipe Delimited Concatenation (Oracle SQL)

Overview: I am tasked with providing a data extract from an Oracle database as a pipe-delimited output text file. I will be using SQLPlus to do this on the server where the data lives. Ordinarily, this task is not beyond my experience, but this time, the business desires column headers to be present.
Consider the following five columns that I need to output:
SELECT
a.USER_NAME || '|'
|| a.LAST_NAME || '|'
|| a.FIRST_NAME || '|'
|| b.PRODUCT_PURCHASED || '|'
|| c.DATEPURCHASED
FROM ...
WHERE ... ;
This SQL works fine, where the output looks like:
omnusruthius|ruthius|omnus|stackoverflow_prod|19-APR-16
However, the business wants it to look like:
USER_NM|LAST|FIRST|PROD|EFFECTIVE_DATE
omnusruthius|ruthius|omnus|stackoverflow_prod|19-APR-16
Problem: So the objective here is essentially to output the first row with custom-named column headers (aliases), as shown above. So my first approach was to try something like:
SELECT
a.USER_NAME AS USER_NM || '|'
|| a.LAST_NAME AS LAST || '|'
|| a.FIRST_NAME AS FIRST || '|'
|| b.PRODUCT_PURCHASED AS PROD || '|'
|| c.DATEPURCHASED AS EFFECTIVE_DATE
FROM ...
WHERE ...
Unfortunately, I receive:
ORA-00923: FROM keyword not found where expected
I'm not sure how that would help anyway, as the original SQL output without aliases does not show column headers in the first row anyway. Remember, this is through the command line (SQLPlus), not Toad or some other RDMS.
So then I tried:
SELECT
'USER_NM', 'LAST', 'FIRST', 'PROD', 'EFFECTIVE_DATE' FROM DUAL
UNION ALL
SELECT
a.USER_NAME || '|'
|| a.LAST_NAME || '|'
|| a.FIRST_NAME || '|'
|| b.PRODUCT_PURCHASED || '|'
|| c.DATEPURCHASED
FROM ...
WHERE ...
Which gives the following error:
ORA-01789: query block has incorrect number of result columns
I feel so close to the solution, what am I missing here? Any help will be appreciated!
Edit: Just a note to a future reader, both answers here will help you solve this problem, but upon further tweaking, I realize we've all been overthinking the solution. I'm not going to propose a new solution as the change is trivial, but consider doing the following instead:
SELECT 'USER_NM|LAST|FIRST|PROD|EFFECTIVE_DATE' FROM DUAL;
SELECT a.USER_NAME AS USER_NM || '|'
|| a.LAST_NAME AS LAST || '|'
|| a.FIRST_NAME AS FIRST || '|'
|| b.PRODUCT_PURCHASED AS PROD || '|'
|| c.DATEPURCHASED AS EFFECTIVE_DATE
FROM ...
WHERE ...
ORDER BY ... ;
The key here is the use of the semicolons in SQL*Plus. That first SELECT statement is completely independent from the second; no UNION is necessary as the output of the first query is automatically displayed immediately before the output of the second query. Both can have their own rules, which is especially convenient if your latter query is much more complicated. I can confirm the above query is working, and I'm surprised it took me that long to make that realization...
When concatenating make sure the header is a single string. Because you are concatenating the values in columns to be on one row.
If comma separation is used, as you have it in the question, the result block should also have 5 columns,which is not the case.
SELECT
'USER_NM|LAST|FIRST|PROD|EFFECTIVE_DATE' FROM DUAL
UNION ALL
SELECT
a.USER_NAME || '|'
|| a.LAST_NAME || '|'
|| a.FIRST_NAME || '|'
|| b.PRODUCT_PURCHASED || '|'
|| c.DATEPURCHASED
FROM ...
WHERE ...
Edit: The columns can also be sorted.
SELECT 'USER_NM|LAST|FIRST|PROD|EFFECTIVE_DATE' FROM DUAL
UNION ALL
SELECT * FROM (
SELECT
a.USER_NAME || '|'
|| a.LAST_NAME || '|'
|| a.FIRST_NAME || '|'
|| b.PRODUCT_PURCHASED || '|'
|| c.DATEPURCHASED
FROM ...
WHERE ...
ORDER BY DATEPURCHASED) --add any other columns needed
Since you tagged the question with SQL*Plus, you can use the PROMPT command to generate the header, which avoids complications ordering the results with a union:
PROMPT USER_NM|LAST|FIRST|PROD|EFFECTIVE_DATE
SELECT
a.USER_NAME AS USER_NM || '|'
|| a.LAST_NAME AS LAST || '|'
|| a.FIRST_NAME AS FIRST || '|'
|| b.PRODUCT_PURCHASED AS PROD || '|'
|| c.DATEPURCHASED AS EFFECTIVE_DATE
FROM ...
WHERE ...
ORDER BY ...
That takes the headings out of the SQL and into the client realm, where it arguably belongs. This also works in SQL Developer, and other clients may be able to do something similar. It won't work if you run the query on its own from another client, or over JDBC, or whatever; but then whatever is running the query can (and maybe should) provide the header in that case too.
If you aren't already, you could also consider doing SET HEADING OFF or SET PAGESIZE 0 to suppress the column headings from the query itself (though from what you said you're already doing that); and possibly SET EMBED OFF, though I don't think that's needed unless you do a separate query to generate the header line.

Oracle regexp_replace string cleanse in SQL query returning full dataset, not just mismatching rows

I'm using Oracle REGEXP_REPLACE to remove all non standard ascii (special) characters from a CLOB freetext field a client uses to store data, some users copy and paste from Excel which causes issues when reading the text from the CLOB. I've managed to get rid of all special characters and maintain the layout for paragraphs etc.
The issue I'm having is when I compare the cleansed field to the original, after converting to char string of course, is it is returning rows that have no difference between them, I've pulled out both values and there is no difference in most cases.
It's returning the whole dataset instead of just those that have been cleansed.I've run out of ideas
FYI a big part of maintaining the layout means I need to keep the carriage return CHR(13)
Below is the query
select *
from (
select incident, entity,
trim(to_char(textarea)) textarea,
trim(to_char(regexp_replace(textarea,'[^' || CHR(13) || CHR(32) || '-' || CHR(126) || ']',''))) regexp_text
from response
) tab
where tab.regexp_text <> tab.textarea
I've tried a number of different combinations with trimming whitespace and substring smaller strings etc. with no luck
Your pattern is: '[^' || CHR(13) || CHR(32) || '-' || CHR(126) || ']'.
The caret (^) is replacing everything that is not a special character with an empty string. With Oracle's default settings, I would expect this to return the empty string, which is treated as a NULL -- but that would mean nothing is returned.
In any case, try removing the caret:
trim(to_char(regexp_replace(textarea, '[CHR(13) || CHR(32) || '-' || CHR(126) || ']',' '))) as regexp_text
WITH data ( value ) AS (
SELECT 'éklzéiuhkn' FROM DUAL
)
SELECT REGEXP_REPLACE( value, '[^\r -~]', '' )
FROM data;
Outputs:
REGEXP_R
--------
klziuhkn
So your query should be something like:
SELECT *
FROM (
SELECT incident,
entity,
TRIM( textarea ) AS textarea,
TRIM( REGEXP_REPLACE( textarea, '[^\r -~]', '' ) ) AS regexp_text
FROM response
)
WHERE textarea <> regexp_text;
I finally figured it out, the issue was in formatting the comparisson strings in the where clause to include all Ascii characters and match against the regexp I planned to use, it was the only way to eliminate bogus and invisible encoded characters.
select incident, entity,
regexp_replace(textarea,'[^' || CHR(13) || ' -}‘-~À-ü]',''))) regexp_text
regexp_replace(textarea,'[^' || CHR(13) || CHR(32) || '-' || CHR(125) || CHR(145) || '-' || CHR(152) || CHR(192) || '-' || CHR(252) || ']','') regexp_text2
from response
where to_char(regexp_replace(textarea,'[^' || CHR(163) || CHR(1) || '-' || CHR(125) || CHR(145) || '-' || CHR(152) || CHR(192) || '-' || CHR(252) || ']','')) <> to_char(regexp_replace(textarea,'[^' || CHR(1) || '-' || CHR(255) || ']',''))
I included both lines to show the simple regexp of '[^' || CHR(13) || ' -}‘-~À-ü]' as well as the CHR(x) version.
I also needed to include latin characters in the end.
For some reason using only CHR() when specifing ascii characters works 100% of the time, I guess it might have something to do with the environments NLS_LANG setting.
This should work for all those looking to exclude weird encoded characters from their strings

How to search new line char in oracle table?

I have an issue with a file that's created from data (in format of bytes) returned from the database.
the problem is that there is a few newlines in the file.
i would like to know if there is a way to write a where clause to check if some record has a newline character?
Using the CHR function to look for the ASCII value:
select *
from your_table
where instr(your_text_col, chr(10)) > 0;
If you want to search for carriage returns, that would be chr(13).
You can write something like:
SELECT id
FROM table_name
WHERE field_name LIKE '%'||CHR(10)||'%'
;
(|| is the concatenation operator; CHR(10) is the tenth ASCII character, i.e. a newline.)
Depending on the platform, a newline will generally either be a CHR(10) (Unix) or a CHR(13) followed by a CHR(10) (Windows). There are other options for other more esoteric platforms, but 99.999% of the time, it will be one of these two.
You can search the data in a column looking for one or both characters
SELECT instr( column_name, CHR(10) ) position_of_first_lf,
instr( column_name, CHR(13) || CHR(10) ) position_of_first_cr_lf
FROM table_name
WHERE instr( column_name, CHR(10) ) > 0
This worked the best for me.
select * from label_master where regexp_like (text, chr(10));

Trim Whitespaces (New Line and Tab space) in a String in Oracle

I need to trim New Line (Chr(13) and Chr(10) and Tab space from the beginning and end of a String) in an Oracle query. I learnt that there is no easy way to trim multiple characters in Oracle. "trim" function trims only single character. It would be a performance degradation if i call trim function recursivelly in a loop using a function. I heard regexp_replace can match the whitespaces and remove them.
Can you guide of a reliable way to use regexp_replace to trim multiple tabspaces or new lines or combinations of them in beginning and end of a String. If there is any other way, Please guide me.
If you have Oracle 10g, REGEXP_REPLACE is pretty flexible.
Using the following string as a test:
chr(9) || 'Q qwer' || chr(9) || chr(10) ||
chr(13) || 'qwerqwer qwerty' || chr(9) ||
chr(10) || chr(13)
The [[:space:]] will remove all whitespace, and the ([[:cntrl:]])|(^\t) regexp will remove non-printing characters and tabs.
select
tester,
regexp_replace(tester, '(^[[:space:]]+)|([[:space:]]+$)',null)
regexp_tester_1,
regexp_replace(tester, '(^[[:cntrl:]^\t]+)|([[:cntrl:]^\t]+$)',null)
regexp_tester_2
from
(
select
chr(9) || 'Q qwer' || chr(9) || chr(10) ||
chr(13) || 'qwerqwer qwerty' || chr(9) ||
chr(10) || chr(13) tester
from
dual
)
Returning:
REGEXP_TESTER_1: "Qqwerqwerqwerqwerty"
REGEXP_TESTER_2: "Q qwerqwerqwer qwerty"
Hope this is of some use.
This how I would implement it:
REGEXP_REPLACE(text,'(^[[:space:]]*|[[:space:]]*$)')
How about the quick and dirty translate function?
This will remove all occurrences of each character in string1:
SELECT translate(
translate(
translate(string1, CHR(10), '')
, CHR(13), '')
, CHR(09), '') as massaged
FROM BLAH;
Regexp_replace is an option, but you may see a performance hit depending on how complex your expression is.
You could use both LTRIM and RTRIM.
select rtrim(ltrim('abcdab','ab'),'ab') from dual;
If you want to trim CHR(13) only when it comes with a CHR(10) it gets more complicated. Firstly, translated the combined string to a single character. Then LTRIM/RTRIM that character, then replace the single character back to the combined string.
select replace(rtrim(ltrim(replace('abccccabcccaab','ab','#'),'#'),'#'),'#','ab') from dual;
TRANSLATE (column_name, 'd'||CHR(10)||CHR(13), 'd')
The 'd' is a dummy character, because translate does not work if the 3rd parameter is null.
For what version of Oracle? 10g+ supports regexes - see this thread on the OTN Discussion forum for how to use REGEXP_REPLACE to change non-printable characters into ''.
I know this is not a strict answer for this question, but I've been working in several scenarios where you need to transform text data following these rules:
No spaces or ctrl chars at the beginning of the string
No spaces or ctrl chars at the end of the string
Multiple ocurrencies of spaces or ctrl chars will be replaced to a single space
Code below follow the rules detailed above:
WITH test_view AS (
SELECT CHR(9) || 'Q qwer' || CHR(9) || CHR(10) ||
CHR(13) || ' qwerqwer qwerty ' || CHR(9) ||
CHR(10) || CHR(13) str
FROM DUAL
) SELECT
str original
,TRIM(REGEXP_REPLACE(str, '([[:space:]]{2,}|[[:cntrl:]])', ' ')) fixed
FROM test_view;
ORIGINAL FIXED
---------------------- ----------------------
Q qwer Q qwer qwerqwer qwerty
qwerqwer qwerty
1 row selected.
If at all anyone is looking to convert data in 1 variable that lies in 2 or 3 different lines like below
'Data1
Data2'
And you want to display data as 'Data1 Data2' then use below
select TRANSLATE ('Data1
Data2', ''||CHR(10), ' ') from dual;
it took me hrs to get the right output. Thanks to me I just saved you 1 or 2 hrs :)
In cases where the Oracle solution seems overly convoluted, I create a java class with static methods and then install it as a package in Oracle. This might not be as performant, but you will eventually find other cases (date conversion to milliseconds for example) where you will find the java fallback helpful.
Below code can be used to Remove New Line and Table Space in text column
Select replace(replace(TEXT,char(10),''),char(13),'')
Try the code below.
It will work if you enter multiple lines in a single column.
create table products (prod_id number , prod_desc varchar2(50));
insert into products values(1,'test first
test second
test third');
select replace(replace(prod_desc,chr(10),' '),chr(13),' ') from products where prod_id=2;
Output :test first test second test third
TRIM(BOTH chr(13)||chr(10)||' ' FROM str)
Instead of using regexp_replace multiple time use (\s) as given below;
SELECT regexp_replace('TEXT','(\s)','')
FROM dual;
Fowloing code remove newline from both side of string:
select ltrim(rtrim('asbda'||CHR(10)||CHR(13) ,''||CHR(10)||CHR(13)),''||CHR(10)||CHR(13)) from dual
but in most cases this one is just enought :
select rtrim('asbda'||CHR(10)||CHR(13) ,''||CHR(10)||CHR(13))) from dual
UPDATE My_Table
SET Mycolumn1 =
TRIM (
TRANSLATE (Mycolumn1,
CHR (10) || CHR (11) || CHR (13),
' '))
WHERE ( INSTR (Mucolumn1, CHR (13)) > 0
OR INSTR (Mucolumn1, CHR (10)) > 0
OR INSTR (Mucolumn1, CHR (11)) > 0);