Best way to check for "empty or null value" - sql

What is best way to check if value is null or empty string in Postgres sql statements?
Value can be long expression so it is preferable that it is written only once in check.
Currently I'm using:
coalesce( trim(stringexpression),'')=''
But it looks a bit ugly.
stringexpression may be char(n) column or expression containing char(n) columns with trailing spaces.
What is best way?

The expression stringexpression = '' yields:
TRUE   .. for '' (or for any string consisting of only spaces with the data type char(n))
NULL   .. for NULL
FALSE .. for anything else
So to check for: "stringexpression is either NULL or empty":
(stringexpression = '') IS NOT FALSE
Or the reverse approach (may be easier to read):
(stringexpression <> '') IS NOT TRUE
Works for any character type including char(n). The manual about comparison operators.
Or use your original expression without trim(), which is costly noise for char(n) (see below), or incorrect for other character types: strings consisting of only spaces would pass as empty string.
coalesce(stringexpression, '') = ''
But the expressions at the top are faster.
Asserting the opposite is even simpler: "stringexpression is neither NULL nor empty":
stringexpression <> ''
About char(n)
This is about the data type char(n), short for: character(n). (char / character are short for char(1) / character(1).) Its use is discouraged in Postgres:
In most situations text or character varying should be used instead.
Do not confuse char(n) with other, useful, character types varchar(n), varchar, text or "char" (with double-quotes).
In char(n) an empty string is not different from any other string consisting of only spaces. All of these are folded to n spaces in char(n) per definition of the type. It follows logically that the above expressions work for char(n) as well - just as much as these (which wouldn't work for other character types):
coalesce(stringexpression, ' ') = ' '
coalesce(stringexpression, '') = ' '
Demo
Empty string equals any string of spaces when cast to char(n):
SELECT ''::char(5) = ''::char(5) AS eq1
, ''::char(5) = ' '::char(5) AS eq2
, ''::char(5) = ' '::char(5) AS eq3;
Result:
eq1 | eq2 | eq3
----+-----+----
t | t | t
Test for "null or empty string" with char(n):
SELECT stringexpression
, stringexpression = '' AS base_test
, (stringexpression = '') IS NOT FALSE AS test1
, (stringexpression <> '') IS NOT TRUE AS test2
, coalesce(stringexpression, '') = '' AS coalesce1
, coalesce(stringexpression, ' ') = ' ' AS coalesce2
, coalesce(stringexpression, '') = ' ' AS coalesce3
FROM (
VALUES
('foo'::char(5))
, ('')
, (' ') -- not different from '' in char(n)
, (NULL)
) sub(stringexpression);
Result:
stringexpression | base_test | test1 | test2 | coalesce1 | coalesce2 | coalesce3
------------------+-----------+-------+-------+-----------+-----------+-----------
foo | f | f | f | f | f | f
| t | t | t | t | t | t
| t | t | t | t | t | t
null | null | t | t | t | t | t
Test for "null or empty string" with text:
SELECT stringexpression
, stringexpression = '' AS base_test
, (stringexpression = '') IS NOT FALSE AS test1
, (stringexpression <> '') IS NOT TRUE AS test2
, coalesce(stringexpression, '') = '' AS coalesce1
, coalesce(stringexpression, ' ') = ' ' AS coalesce2
, coalesce(stringexpression, '') = ' ' AS coalesce3
FROM (
VALUES
('foo'::text)
, ('')
, (' ') -- different from '' in a sane character types
, (NULL)
) sub(stringexpression);
Result:
stringexpression | base_test | test1 | test2 | coalesce1 | coalesce2 | coalesce3
------------------+-----------+-------+-------+-----------+-----------+-----------
foo | f | f | f | f | f | f
| t | t | t | t | f | f
| f | f | f | f | f | f
null | null | t | t | t | t | f
db<>fiddle here
Old sqlfiddle
Related:
Any downsides of using data type "text" for storing strings?

To check for null and empty:
coalesce(string, '') = ''
To check for null, empty and spaces (trim the string)
coalesce(TRIM(string), '') = ''

Checking for the length of the string also works and is compact:
where length(stringexpression) > 0;

A lot of the answers are the shortest way, not the necessarily the best way if the column has lots of nulls. Breaking the checks up allows the optimizer to evaluate the check faster as it doesn't have to do work on the other condition.
(stringexpression IS NOT NULL AND trim(stringexpression) != '')
The string comparison doesn't need to be evaluated since the first condition is false.

another way is
nullif(trim(stringExpression),'') is not null

If there may be empty trailing spaces, probably there isn't better solution. COALESCE is just for problems like yours.

Something that I saw people using is stringexpression > ''. This may be not the fastest one, but happens to be one of the shortest.
Tried it on MS SQL as well as on PostgreSQL.

In ran into a kind of similar case, were I had to do this . My Table definition look like :
id(bigint)|name (character varying)|results(character varying)
1 | "Peters"| [{"jk1":"jv1"},{"jk1":"jv2"}]
2 | "Russel"| null
To filter out the results column with null or empty in it , what worked was :
SELECT * FROM tablename where results NOT IN ('null','{}');
This returned all rows which are not null on results.
I'm not sure how to fix this query to return the same all rows which are not null on results.
SELECT * FROM tablename where results is not null;
--- hmm what am I missing,casting ? any inputs?

found this post looking for a solution to 'don't show me data that is '' (blank or single space char) or null'. in my case, we only want to show the user records with these values populated. i hope this response helps another looking for the same. the answers above didn't work in my case.
our app is running rails with postgres. looking at how rails builds the query for .where.not(company_website: [nil, '']) in our app, which works just fine, i can see the resulting sql statement in console.
WHERE NOT ((contacts.company_website = '' OR contacts.company_website IS NULL))
i added this bit and it works as intended.

I like answer by yglodt, but calculating exact length may be expensive for big sets and big strings, so I go with:
coalesce(trim('a') > '','f')

My preffered way to compare nullable fields is:
NULLIF(nullablefield, :ParameterValue) IS NULL AND NULLIF(:ParameterValue, nullablefield) IS NULL . This is cumbersome but is of universal use while Coalesce is impossible in some cases.
The second and inverse use of NULLIF is because "NULLIF(nullablefield, :ParameterValue) IS NULL" will always return "true" if the first parameter is null.

If database having large number of records then null check can take more time
you can use null check in different ways like :
1) where columnname is null
2) where not exists()
3) WHERE (case when columnname is null then true end)

Related

How can I query for blank columns? [duplicate]

What is best way to check if value is null or empty string in Postgres sql statements?
Value can be long expression so it is preferable that it is written only once in check.
Currently I'm using:
coalesce( trim(stringexpression),'')=''
But it looks a bit ugly.
stringexpression may be char(n) column or expression containing char(n) columns with trailing spaces.
What is best way?
The expression stringexpression = '' yields:
TRUE   .. for '' (or for any string consisting of only spaces with the data type char(n))
NULL   .. for NULL
FALSE .. for anything else
So to check for: "stringexpression is either NULL or empty":
(stringexpression = '') IS NOT FALSE
Or the reverse approach (may be easier to read):
(stringexpression <> '') IS NOT TRUE
Works for any character type including char(n). The manual about comparison operators.
Or use your original expression without trim(), which is costly noise for char(n) (see below), or incorrect for other character types: strings consisting of only spaces would pass as empty string.
coalesce(stringexpression, '') = ''
But the expressions at the top are faster.
Asserting the opposite is even simpler: "stringexpression is neither NULL nor empty":
stringexpression <> ''
About char(n)
This is about the data type char(n), short for: character(n). (char / character are short for char(1) / character(1).) Its use is discouraged in Postgres:
In most situations text or character varying should be used instead.
Do not confuse char(n) with other, useful, character types varchar(n), varchar, text or "char" (with double-quotes).
In char(n) an empty string is not different from any other string consisting of only spaces. All of these are folded to n spaces in char(n) per definition of the type. It follows logically that the above expressions work for char(n) as well - just as much as these (which wouldn't work for other character types):
coalesce(stringexpression, ' ') = ' '
coalesce(stringexpression, '') = ' '
Demo
Empty string equals any string of spaces when cast to char(n):
SELECT ''::char(5) = ''::char(5) AS eq1
, ''::char(5) = ' '::char(5) AS eq2
, ''::char(5) = ' '::char(5) AS eq3;
Result:
eq1 | eq2 | eq3
----+-----+----
t | t | t
Test for "null or empty string" with char(n):
SELECT stringexpression
, stringexpression = '' AS base_test
, (stringexpression = '') IS NOT FALSE AS test1
, (stringexpression <> '') IS NOT TRUE AS test2
, coalesce(stringexpression, '') = '' AS coalesce1
, coalesce(stringexpression, ' ') = ' ' AS coalesce2
, coalesce(stringexpression, '') = ' ' AS coalesce3
FROM (
VALUES
('foo'::char(5))
, ('')
, (' ') -- not different from '' in char(n)
, (NULL)
) sub(stringexpression);
Result:
stringexpression | base_test | test1 | test2 | coalesce1 | coalesce2 | coalesce3
------------------+-----------+-------+-------+-----------+-----------+-----------
foo | f | f | f | f | f | f
| t | t | t | t | t | t
| t | t | t | t | t | t
null | null | t | t | t | t | t
Test for "null or empty string" with text:
SELECT stringexpression
, stringexpression = '' AS base_test
, (stringexpression = '') IS NOT FALSE AS test1
, (stringexpression <> '') IS NOT TRUE AS test2
, coalesce(stringexpression, '') = '' AS coalesce1
, coalesce(stringexpression, ' ') = ' ' AS coalesce2
, coalesce(stringexpression, '') = ' ' AS coalesce3
FROM (
VALUES
('foo'::text)
, ('')
, (' ') -- different from '' in a sane character types
, (NULL)
) sub(stringexpression);
Result:
stringexpression | base_test | test1 | test2 | coalesce1 | coalesce2 | coalesce3
------------------+-----------+-------+-------+-----------+-----------+-----------
foo | f | f | f | f | f | f
| t | t | t | t | f | f
| f | f | f | f | f | f
null | null | t | t | t | t | f
db<>fiddle here
Old sqlfiddle
Related:
Any downsides of using data type "text" for storing strings?
To check for null and empty:
coalesce(string, '') = ''
To check for null, empty and spaces (trim the string)
coalesce(TRIM(string), '') = ''
Checking for the length of the string also works and is compact:
where length(stringexpression) > 0;
A lot of the answers are the shortest way, not the necessarily the best way if the column has lots of nulls. Breaking the checks up allows the optimizer to evaluate the check faster as it doesn't have to do work on the other condition.
(stringexpression IS NOT NULL AND trim(stringexpression) != '')
The string comparison doesn't need to be evaluated since the first condition is false.
another way is
nullif(trim(stringExpression),'') is not null
If there may be empty trailing spaces, probably there isn't better solution. COALESCE is just for problems like yours.
Something that I saw people using is stringexpression > ''. This may be not the fastest one, but happens to be one of the shortest.
Tried it on MS SQL as well as on PostgreSQL.
In ran into a kind of similar case, were I had to do this . My Table definition look like :
id(bigint)|name (character varying)|results(character varying)
1 | "Peters"| [{"jk1":"jv1"},{"jk1":"jv2"}]
2 | "Russel"| null
To filter out the results column with null or empty in it , what worked was :
SELECT * FROM tablename where results NOT IN ('null','{}');
This returned all rows which are not null on results.
I'm not sure how to fix this query to return the same all rows which are not null on results.
SELECT * FROM tablename where results is not null;
--- hmm what am I missing,casting ? any inputs?
found this post looking for a solution to 'don't show me data that is '' (blank or single space char) or null'. in my case, we only want to show the user records with these values populated. i hope this response helps another looking for the same. the answers above didn't work in my case.
our app is running rails with postgres. looking at how rails builds the query for .where.not(company_website: [nil, '']) in our app, which works just fine, i can see the resulting sql statement in console.
WHERE NOT ((contacts.company_website = '' OR contacts.company_website IS NULL))
i added this bit and it works as intended.
I like answer by yglodt, but calculating exact length may be expensive for big sets and big strings, so I go with:
coalesce(trim('a') > '','f')
My preffered way to compare nullable fields is:
NULLIF(nullablefield, :ParameterValue) IS NULL AND NULLIF(:ParameterValue, nullablefield) IS NULL . This is cumbersome but is of universal use while Coalesce is impossible in some cases.
The second and inverse use of NULLIF is because "NULLIF(nullablefield, :ParameterValue) IS NULL" will always return "true" if the first parameter is null.
If database having large number of records then null check can take more time
you can use null check in different ways like :
1) where columnname is null
2) where not exists()
3) WHERE (case when columnname is null then true end)

Update/Replace with a substring - triple backslashes

I have a column full of duplicate filepaths:
\\C:\298788\DOC1\SUB1\\\C:\298788\DOC1\SUB1\FILE.txt
\\C:\298788\DOC1\SUB1\\\C:\298788\DOC1\SUB1\FILE.txt
\\C:\298788\DOC1\SUB1\\\C:\298788\DOC1\SUB1\FILE.txt
I need only the second part of the string ie. C:\298788\DOC1\SUB1\FILE.txt
How do I replace up to the triple backslashes with nothing. I have tried:
UPDATE [TABLE].[dbo].[ColumnName]
SET [ColumnName] = REPLACE([ColumnName], '%\\\', '');
It says all rows updated however nothing has changed. Assuming its something to do with the backslashes.
Using SQL SERVER 2012.
Using stuff()
select col = stuff(col,1,charindex('\\\',col,2)+2,'')
from tbl
rextester demo: http://rextester.com/QRKWP8606
returns:
+------------------------------+
| col |
+------------------------------+
| C:\298788\DOC1\SUB1\FILE.txt |
| C:\298788\DOC1\SUB1\FILE.txt |
| C:\298788\DOC1\SUB1\FILE.txt |
+------------------------------+
as an update:
update tbl
set col = stuff(col,1,charindex('\\\',col,2)+2,'')
where charindex('\\\',col,2)>0

add leading zeros to a varchar column

Select len(productid),productid,*
FROM dbo.produts
where Place = 'KA'
and len(productid) <> 10
order by len(productid)
this query filters the data that needs to be updated
-- i need to update the data in 'productid' that is not in the correct format
--(the data should be 10 characters, in 4-2-4 format with leading 0s where they are needed) (productid column is a varchar (256) column)
basically i need to add leading zeros to a varchar column that has a where condition
|productid| |Correct productid value| |
---------------------------------------
|234-55-43||000234-55-43|
|76-7-89||0000076-7-89|
what are the possible solutions for updating these records?
This is very simple, actually:
SELECT productid,
RIGHT('000000000'+productid,10) CorrectProductid
FROM dbo.YourTable
;
If you wanted the corrected format to be in the 4-2-4 format you mentioned:
Using parsename() to parse out the pieces of the string, and right() to add the extra '0's.
select
col
, Corrected = right('0000'+parsename(replace(col,'-','.'),3),4)
+ '-' + right('00'+parsename(replace(col,'-','.'),2),2)
+ '-' + right('0000'+parsename(replace(col,'-','.'),1),4)
from t
where col not like '____-__-____'
rextester demo: http://rextester.com/OXM28014
returns:
+-----------+--------------+
| col | Corrected |
+-----------+--------------+
| 234-55-43 | 0234-55-0043 |
| 76-7-89 | 0076-07-0089 |
+-----------+--------------+
As an update:
update t
set col = right('0000'+parsename(replace(col,'-','.'),3),4)
+ '-' + right('00'+parsename(replace(col,'-','.'),2),2)
+ '-' + right('0000'+parsename(replace(col,'-','.'),1),4)
where col not like '____-__-____'

How to remove duplicates within a string from the table in SQL server 2016

I got a table which contains a column of strings. Those strings are separated by a ;. Now I want to remove the duplicates after the string is splitted. For example:
-----------
| w;w;e;e |
-----------
| q;r;r;q |
-----------
| b;n;n;b |
-----------
The result should be:
-------
| w;e |
-------
| q;r |
-------
| b;n |
-------
Also it should not be a Select function but a (not 100% sure) delete function. So the values in the original table won't be duplicated anymore.
For an update statement, this will de-duplicate your column:
update t
set col = stuff((
select distinct
';'+s.Value
from string_split(t.col,';') as s
for xml path (''), type).value('.','varchar(1024)')
,1,1,'');
In sql server 2016, you can use string_split() along with the stuff() with select ... for xml path ('') method of string concatenation to concatenate only distinct values.
select
t.id
, t.col
, dedup = stuff((
select distinct
';'+s.Value
from string_split(t.col,';') as s
for xml path (''), type).value('.','varchar(1024)')
,1,1,'')
from t
dbfiddle demo: here
rextester demo: http://rextester.com/MAME55141; this demo uses a CSV Splitter function by Jeff Moden in the absence of string_split().
returns:
+----+---------+-------+
| id | col | dedup |
+----+---------+-------+
| 1 | w;w;e;e | e;w |
| 2 | q;r;r;q | q;r |
| 3 | b;n;n;b | b;n |
+----+---------+-------+
splitting strings reference:
Tally OH! An Improved SQL 8K “CSV Splitter” Function - Jeff Moden
Splitting Strings : A Follow-Up - Aaron Bertrand
Split strings the right way – or the next best way - Aaron Bertrand
string_split() in SQL Server 2016 : Follow-Up #1 - Aaron Bertrand
If "e", "r", and "w" are the only values in the string, then the simplest way is to reconstruct the string:
select stuff( (case when string like '%e%' then ';e' else '' end) +
(case when string like '%r%' then ';r' else '' end) +
(case when string like '%w%' then ';w' else '' end),
1, 1, ''
)
I suspect the values might be limited, because these look like file permissions (read/write/execute). Otherwise, you will need to parse the string into separate rows (using XML, a UDF, or a recursive CTE) and recombine the values.
You should learn a lesson here. Don't store lists in strings. These values should either be flags (if I am right about there only being a handful of values). Or, they should be on separate rows of a another table.

show value from text..if delimited by ; show only first value

I have table with values. It is ntext because of ; delimited. Values can be empty, 1 number and numbers delimited by semicolon (as shown)
+-----------+
| room |
+-----------+
| 64 |
+-----------+
| 60008 |
+-----------+
| |
+-----------+
| 127;50047 |
+-----------+
I have this code. Substring is looking for ; and show first value. It is working only where the values are delimited. So how can I change it, that it will show first value when ; and single value also. So from table bellow I will get 64,60008, ,127.
SELECT
T0.U_Scid as 'id',
T3.U_Boarding as 'start',
T3.U_Boarding as 'end',
SUBSTRING(T5.U_Partner, 0, CHARINDEX(';', T5.U_Partner)) AS 'room_id',
CASE WHEN datalength(T5.U_Partner)=0 THEN '9999' ELSE T5.U_Partner END AS 'room_id' ,
CASE WHEN datalength(T5.U_Partner) > 4 THEN T5.U_Partner ELSE '9999' END AS 'partners_id' ,
This is just bonus question. CASE are looking for length of value, if the value is longer than 4 ( 600008 ) write to room_id 9999 and save 600008 to partners_id. If it is empty write 9999 to room_id.
How to make it works together?.so getting value from T_Partner..save it into temporary table T1.TempRoom ( I suppose ).. so T1.TempRoom (is filled with numbers like 64, ,60008,127) then CASE is checking T1.TempRoom for values and save it into room_id and partners_id.
Am I right?
Here is a simple method:
SUBSTRING(T5.U_Partner, 1, CHARINDEX(';', T5.U_Partner + ';')) AS room_id,
That is, concatenate the semicolon to the argument for CHARINDEX(). That will prevent any error occurring.
In addition, indexing for SUBSTRING() starts at 1, not 0.
And, don't use single quotes for column names. Only use them for string and date literals.
EDIT:
You can always use the verbose form:
(CASE WHEN T5.U_Partner LIKE '%;%'
THEN SUBSTRING(T5.U_Partner, 1, CHARINDEX(';', T5.U_Partner + ';'))
ELSE T5.U_Partner
END) AS room_id