How do I return my records grouped by NULL and NOT NULL? - sql

I have a table that has a processed_timestamp column -- if a record has been processed then that field contains the datetime it was processed, otherwise it is null.
I want to write a query that returns two rows:
NULL xx -- count of records with null timestamps
NOT NULL yy -- count of records with non-null timestamps
Is that possible?
Update: The table is quite large, so efficiency is important. I could just run two queries to calculate each total separately, but I want to avoid hitting the table twice if I can avoid it.

In MySQL you could do something like
SELECT
IF(ISNULL(processed_timestamp), 'NULL', 'NOT NULL') as myfield,
COUNT(*)
FROM mytable
GROUP BY myfield

In T-SQL (MS SQL Server), this works:
SELECT
CASE WHEN Field IS NULL THEN 'NULL' ELSE 'NOT NULL' END FieldContent,
COUNT(*) FieldCount
FROM
TheTable
GROUP BY
CASE WHEN Field IS NULL THEN 'NULL' ELSE 'NOT NULL' END

Oracle:
group by nvl2(field, 'NOT NULL', 'NULL')

Try the following, it's vendor-neutral:
select
'null ' as type,
count(*) as quant
from tbl
where tmstmp is null
union all
select
'not null' as type,
count(*) as quant
from tbl
where tmstmp is not null
After having our local DB2 guru look at this, he concurs: none of the solutions presented to date (including this one) can avoid a full table scan (of the table if timestamp is not indexed, or of the indexotherwise). They all scan every record in the table exactly once.
All the CASE/IF/NVL2() solutions do a null-to-string conversion for each row, introducing unnecessary load on the DBMS. This solution does not have that problem.

Stewart,
Maybe consider this solution. It is (also!) vendor non-specific.
SELECT count([processed_timestamp]) AS notnullrows,
count(*) - count([processed_timestamp]) AS nullrows
FROM table
As for efficiency, this avoids 2x index seeks/table scans/whatever by including the results on one row. If you absolutely require 2 rows in the result, two passes over the set may be unavoidable because of unioning aggregates.
Hope this helps

If it's oracle then you can do:
select decode(field,NULL,'NULL','NOT NULL'), count(*)
from table
group by decode(field,NULL,'NULL','NOT NULL');
I'm sure that other DBs allow for similar trick.

Another MySQL method is to use the CASE operator, which can be generalised to more alternatives than IF():
SELECT CASE WHEN processed_timestamp IS NULL THEN 'NULL'
ELSE 'NOT NULL' END AS a,
COUNT(*) AS n
FROM logs
GROUP BY a

SQL Server (starting with 2012):
SELECT IIF(ISDATE(processed_timestamp) = 0, 'NULL', 'NON NULL'), COUNT(*)
FROM MyTable
GROUP BY ISDATE(processed_timestamp);

Another way in T-sql (sql-server)
select count(case when t.timestamps is null
then 1
else null end) NULLROWS,
count(case when t.timestamps is not null
then 1
else null end) NOTNULLROWS
from myTable t

If your database has an efficient COUNT(*) function for a table, you could COUNT whichever is the smaller number, and subtract.

In Oracle
SELECT COUNT(*), COUNT(TIME_STAMP_COLUMN)
FROM TABLE;
count(*) returns the count of all rows
count(column_name) returns the number of rows which are not NULL, so
SELECT COUNT(*) - COUNT(TIME_STAMP_COLUMN) NUL_COUNT,
COUNT(TIME_STAMP_COLUMN) NON_NUL_COUNT
FROM TABLE
ought to do the job.
If the column is indexed, you might end up with some sort of range scan and avoid actually reading the table.

I personally like Pax's solution, but if you absolutely require only one row returned (as I had recently), In MS SQL Server 2005/2008 you can "stack" the two queries using a CTE
with NullRows (countOf)
AS
(
SELECT count(*)
FORM table
WHERE [processed_timestamp] IS NOT NULL
)
SELECT count(*) AS nulls, countOf
FROM table, NullRows
WHERE [processed_timestamp] IS NULL
GROUP BY countOf
Hope this helps

[T-SQL]:
select [case], count(*) tally
from (
select
case when [processed_timestamp] is null then 'null'
else 'not null'
end [case]
from myTable
) a
And you can add into the case statement whatever other values you'd like to form a partition, e.g. today, yesterday, between noon and 2pm, after 6pm on a Thursday.

Select Sum(Case When processed_timestamp IS NULL
Then 1
Else 0
End) not_processed_count,
Sum(Case When processed_timestamp Is Not NULL
Then 1
Else 0
End) processed_count,
Count(1) total
From table
Edit: didn't read carefully, this one returns a single row.

Related

Why Using COALESCE or CASE keep returning null

I have the following SQL Query :
(SELECT ROUND(SUM(NBTOSUM)/1000000,1) FROM MyTable t2 WHERE t2.ELEMNAME IN ('A','B','C'))
Which works fine.
But Where there is no 'A','B','C' the result of the select is (null)
So to handle it, I did the following :
(SELECT COALESCE(ROUND(SUM(NBTOSUM)/1000000,1),0) FROM MyTable t2 WHERE t2.ELEMNAME IN ('A','B','C'))
And also try :
(SELECT
CASE
WHEN SUM(NBTOSUM)/1000000 IS NULL THEN 0
ELSE ROUND(SUM(NBTOSUM)/1000000,1)
END
FROM MyTable t2 WHERE t2.ELEMNAME IN ('A','B','C'))
But both keep returning null
What am I doing wrong ?
Move the WHERE restrictions to the CASE expression as well:
SELECT ROUND(SUM(CASE WHEN t2.ELEMNAME IN ('A','B','C')
THEN NBTOSUM ELSE 0 END) / 1000000, 1)
FROM MyTable t2;
Note that this trick solves the null problem and also avoids the need for an ugly COALESCE() call.
Your code should work as the SUM aggregation function will generate a single row of output regardless of whether the number of input rows is zero or non-zero. If there are no input rows or the values are all NULL then the output of the SUM will be NULL and then COALESCE would work.
Since you claim it does not then that suggests that there is something else going on in your query that you have not shared in the question.
You have braces around your statement suggesting that you are using it as part of a larger statement. If so, you can try moving the COALESCE to the outer query:
SELECT COALESCE(
(
SELECT ROUND(SUM(NBTOSUM)/1000000,1)
FROM MyTable
WHERE ELEMNAME IN ('A','B','C')
),
0
)
FROM your_outer_query;
That might fix the problem if you are somehow correlating to an outer query but your question makes no mention of that.
fiddle

Why does this not return 0

I have a query like:
select nvl(nvl(sum(a.quantity),0)-nvl(cc.quantityCor,0),0)
from RCV_TRANSACTIONS a
LEFT JOIN (select c.shipment_line_id,c.oe_order_line_id,nvl(sum(c.quantity),0) quantityCor
from RCV_TRANSACTIONS c
where c.TRANSACTION_TYPE='CORRECT'
group by c.shipment_line_id,c.oe_order_line_id) cc on (a.shipment_line_id=cc.shipment_line_id and a.shipment_line_id=7085740)
where a.transaction_type='DELIVER'
and a.shipment_line_id=7085740
group by nvl(cc.quantityCor,0);
The query runs OK, but returns no value. I want it to return 0 if there is no quantity found. Where have I gone wrong?
An aggregation query with a GROUP BY returns no rows if all rows are filtered out.
An aggregation query with no GROUP BY always returns one row, even if all rows are filtered out.
So, just remove the GROUP BY. And change the SELECT to:
select coalesce(sum(a.quantity), 0) - coalesce(max(cc.quantityCor), 0)
I may be wrong, but it seems you merely want to subtract CORRECT quantity from DELIVER quantity for shipment 7085740. You don't need a complicated query for that. Especially your GROUP BY clauses make no sense if that is what you are after.
One way to write this query would be:
select
sum(case when transaction_type = 'DELIVER' then quantity else 0 end) -
sum(case when transaction_type = 'CORRECT' then quantity else 0 end) as diff
from rcv_transactions
where shipment_line_id = 7085740;
I had a query like this and was trying to return 'X' when the item is not valid.
SELECT case when segment1 is not null then segment1 else 'X' end
--INTO v_orgValidItem
FROM mtl_system_items_b
WHERE segment1='1676001000'--'Jul-00'--l_item
and organization_id=168;
..but it was returning NULL.
Changed to use aggregation with no group by and now it returns 'X' when the item is not valid.
SELECT case when max(segment1) is not null then max(segment1) else 'X' end valid
--INTO v_orgValidItem
FROM mtl_system_items_b
WHERE segment1='1676001000'--'Jul-00'--l_item
and organization_id=168;--l_ship_to_organization_id_pb;
Here is another example, proving the order of operations really matters.
When there is no match for this quote number, this query returns NULL:
SELECT MAX(NVL(QUOTE_VENDOR_QUOTE_NUMBER,0))
FROM PO_HEADERS_ALL
WHERE QUOTE_VENDOR_QUOTE_NUMBER='foo.bar';
..reversing the order of MAX and NVL makes all the difference. This query returns the NULL value condition:
SELECT NVL(MAX(QUOTE_VENDOR_QUOTE_NUMBER),0)
FROM PO_HEADERS_ALL
WHERE QUOTE_VENDOR_QUOTE_NUMBER='foo.bar';

How to nest a CTE (Common Table Expression)

I have the below query
With max_cm1 as (select * from tableA)
Select * ,
CASE WHEN TO_CHAR(CCP2.END_DATE,'MM/DD/YYYY') <> '09/09/9000' THEN 'CLOSED'
WHEN MAX_CM1.MAX_ROLE_CM IS NOT NULL AND HIST.PCMUID IS NOT NULL THEN 'ASSIGNED'
ELSE 'UNASSIGNED'
END STATUS
from max_cm1
Now I need to filter on the case statement. How can I do this?
You can use an alias eg: m.
With max_cm1 as (select * from tableA)
Select m.* ,
CASE WHEN TO_CHAR(CCP2.END_DATE,'MM/DD/YYYY') <> '09/09/9000' THEN 'CLOSED'
WHEN MAX_CM1.MAX_ROLE_CM IS NOT NULL AND HIST.PCMUID IS NOT NULL THEN 'ASSIGNED'
ELSE 'UNASSIGNED'
END STATUS
from max_cm1 m;
In your case, you don't need a CTE unless you are joining it with other table with some expressions in CTE. Directly you can fetch from table A with the same method if you are only interested in select '*'.
Your question is unclear. Also, the query as given is somewhat confusing, as it qualifies some columns with table names (CCP2 and HIST) that don't appear elsewhere in the query. Further, as written there seems to be no purpose to the CTE at all.
I'm assuming that what you want is to include the given CASE expression in the result set, but also use it within the WHERE clause to filter the results (e.g. WHERE CASE ... END = 'CLOSED'. The simple way to do this is to repeat the CASE expression; but of course duplicating logic is never a good choice. So the better way, which I think is the point of your question, is to include that derived column in a CTE so you can then refer to it by name in the WHERE clause.
It also looks like you are probably running into the issue of trying to select all columns (*) plus a derived column. The way around this is to qualify the * with the table name, or an alias as indicated in one of the other answers.
Putting this all together, I believe you want something like the following. I'm keeping the column expressions (e.g. HIST.PCMUID) as you wrote them although as written they make no sense. I'm guessing that tableA really represents some join of multiple tables.
WITH max_cm1 AS (
SELECT tableA.* ,
CASE WHEN TO_CHAR(CCP2.END_DATE,'MM/DD/YYYY') <> '09/09/9000' THEN 'CLOSED'
WHEN MAX_CM1.MAX_ROLE_CM IS NOT NULL AND HIST.PCMUID IS NOT NULL THEN
'ASSIGNED'
ELSE 'UNASSIGNED'
END STATUS
FROM tableA
)
SELECT *
FROM max_cm1
WHERE status = 'CLOSED'

How to summarize SQL table to return value conditionally

I have a table with several rows, and several columns. It looks like this:
Name Description
X PASS
X PASS
X FAIL
I want it to return only one row. If all of them are PASS, return PASS.
If one or more of them are FAIL, then return FAIL.
What's the best way to go about achieving this in SQL Server 2008?
EDIT: The values in the name column will always be the same.
Depending on the database indexes, and assuming you want one row returned per unique name, I would look at the performance of
select
name,
min([description]) as description
from
tableA
group by
name
compared to the other solutions
SELECT TOP 1 CASE Description WHEN 'FAIL' THEN 'FAIL' ELSE 'PASS' END
FROM DaTable
ORDER BY Description
OP: Is it possible that the table is empty? In that case this query won't return any rows, obviously.
EDIT
According to aquinas's comment I created a modified query without ordering:
SELECT CASE COUNT(Description) WHEN 0 THEN 'FAIL' ELSE 'PASS' END
FROM DaTable
WHERE Description = 'FAIL'
This query will return PASS if DaTable is empty.
This is the simplest solution you will find:
SELECT MIN(Description) FROM tbl
If there's at least one FAIL, then our result column will contain FAIL, otherwise, it will contain PASS.
You can use EXISTS to get the existance of a row containing "FAIL".
You could also try something like:
SELECT TOP 1 COALESCE(tFail.Description,t.Description)
FROM myTable AS t
LEFT JOIN myTable AS tFail ON tFail.Name = t.Name AND tFail.Description = 'FAIL'
WHERE t.Name = 'x'
Here is the query:
--DROP TABLE result
CREATE TABLE result(Name varchar(10),Description varchar(20))
--select * from result
INSERT INTO result
VALUES('X','PASS'),('X','PASS'),('X','FAIL')
;WITH CTE(descp,cnt) as (SELECT [description],COUNT(*) as cnt FROM result group by [description])
SELECT CASE WHEN COUNT(*) > 1 then 'FAIL' when COUNT(*)=1 then MAX(descp) else 'PASS' END FROM CTE

Counting null and non-null values in a single query

I have a table
create table us
(
a number
);
Now I have data like:
a
1
2
3
4
null
null
null
8
9
Now I need a single query to count null and not null values in column a
This works for Oracle and SQL Server (you might be able to get it to work on another RDBMS):
select sum(case when a is null then 1 else 0 end) count_nulls
, count(a) count_not_nulls
from us;
Or:
select count(*) - count(a), count(a) from us;
If I understood correctly you want to count all NULL and all NOT NULL in a column...
If that is correct:
SELECT count(*) FROM us WHERE a IS NULL
UNION ALL
SELECT count(*) FROM us WHERE a IS NOT NULL
Edited to have the full query, after reading the comments :]
SELECT COUNT(*), 'null_tally' AS narrative
FROM us
WHERE a IS NULL
UNION
SELECT COUNT(*), 'not_null_tally' AS narrative
FROM us
WHERE a IS NOT NULL;
Here is a quick and dirty version that works on Oracle :
select sum(case a when null then 1 else 0) "Null values",
sum(case a when null then 0 else 1) "Non-null values"
from us
for non nulls
select count(a)
from us
for nulls
select count(*)
from us
minus
select count(a)
from us
Hence
SELECT COUNT(A) NOT_NULLS
FROM US
UNION
SELECT COUNT(*) - COUNT(A) NULLS
FROM US
ought to do the job
Better in that the column titles come out correct.
SELECT COUNT(A) NOT_NULL, COUNT(*) - COUNT(A) NULLS
FROM US
In some testing on my system, it costs a full table scan.
As i understood your query, You just run this script and get Total Null,Total NotNull rows,
select count(*) - count(a) as 'Null', count(a) as 'Not Null' from us;
usually i use this trick
select sum(case when a is null then 0 else 1 end) as count_notnull,
sum(case when a is null then 1 else 0 end) as count_null
from tab
group by a
Just to provide yet another alternative, Postgres 9.4+ allows applying a FILTER to aggregates:
SELECT
COUNT(*) FILTER (WHERE a IS NULL) count_nulls,
COUNT(*) FILTER (WHERE a IS NOT NULL) count_not_nulls
FROM us;
SQLFiddle: http://sqlfiddle.com/#!17/80a24/5
This is little tricky. Assume the table has just one column, then the Count(1) and Count(*) will give different values.
set nocount on
declare #table1 table (empid int)
insert #table1 values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(NULL),(11),(12),(NULL),(13),(14);
select * from #table1
select COUNT(1) as "COUNT(1)" from #table1
select COUNT(empid) "Count(empid)" from #table1
Query Results
As you can see in the image, The first result shows the table has 16 rows. out of which two rows are NULL. So when we use Count(*) the query engine counts the number of rows, So we got count result as 16. But in case of Count(empid) it counted the non-NULL-values in the column empid. So we got the result as 14.
so whenever we are using COUNT(Column) make sure we take care of NULL values as shown below.
select COUNT(isnull(empid,1)) from #table1
will count both NULL and Non-NULL values.
Note: Same thing applies even when the table is made up of more than one column. Count(1) will give total number of rows irrespective of NULL/Non-NULL values. Only when the column values are counted using Count(Column) we need to take care of NULL values.
I had a similar issue: to count all distinct values, counting null values as 1, too. A simple count doesn't work in this case, as it does not take null values into account.
Here's a snippet that works on SQL and does not involve selection of new values.
Basically, once performed the distinct, also return the row number in a new column (n) using the row_number() function, then perform a count on that column:
SELECT COUNT(n)
FROM (
SELECT *, row_number() OVER (ORDER BY [MyColumn] ASC) n
FROM (
SELECT DISTINCT [MyColumn]
FROM [MyTable]
) items
) distinctItems
Try this..
SELECT CASE
WHEN a IS NULL THEN 'Null'
ELSE 'Not Null'
END a,
Count(1)
FROM us
GROUP BY CASE
WHEN a IS NULL THEN 'Null'
ELSE 'Not Null'
END
Here are two solutions:
Select count(columnname) as countofNotNulls, count(isnull(columnname,1))-count(columnname) AS Countofnulls from table name
OR
Select count(columnname) as countofNotNulls, count(*)-count(columnname) AS Countofnulls from table name
Try
SELECT
SUM(ISNULL(a)) AS all_null,
SUM(!ISNULL(a)) AS all_not_null
FROM us;
Simple!
If you're using MS Sql Server...
SELECT COUNT(0) AS 'Null_ColumnA_Records',
(
SELECT COUNT(0)
FROM your_table
WHERE ColumnA IS NOT NULL
) AS 'NOT_Null_ColumnA_Records'
FROM your_table
WHERE ColumnA IS NULL;
I don't recomend you doing this... but here you have it (in the same table as result)
use ISNULL embedded function.
All the answers are either wrong or extremely out of date.
The simple and correct way of doing this query is using COUNT_IF function.
SELECT
COUNT_IF(a IS NULL) AS nulls,
COUNT_IF(a IS NOT NULL) AS not_nulls
FROM
us
SELECT SUM(NULLs) AS 'NULLS', SUM(NOTNULLs) AS 'NOTNULLs' FROM
(select count(*) AS 'NULLs', 0 as 'NOTNULLs' FROM us WHERE a is null
UNION select 0 as 'NULLs', count(*) AS 'NOTNULLs' FROM us WHERE a is not null) AS x
It's fugly, but it will return a single record with 2 cols indicating the count of nulls vs non nulls.
This works in T-SQL. If you're just counting the number of something and you want to include the nulls, use COALESCE instead of case.
IF OBJECT_ID('tempdb..#us') IS NOT NULL
DROP TABLE #us
CREATE TABLE #us
(
a INT NULL
);
INSERT INTO #us VALUES (1),(2),(3),(4),(NULL),(NULL),(NULL),(8),(9)
SELECT * FROM #us
SELECT CASE WHEN a IS NULL THEN 'NULL' ELSE 'NON-NULL' END AS 'NULL?',
COUNT(CASE WHEN a IS NULL THEN 'NULL' ELSE 'NON-NULL' END) AS 'Count'
FROM #us
GROUP BY CASE WHEN a IS NULL THEN 'NULL' ELSE 'NON-NULL' END
SELECT COALESCE(CAST(a AS NVARCHAR),'NULL') AS a,
COUNT(COALESCE(CAST(a AS NVARCHAR),'NULL')) AS 'Count'
FROM #us
GROUP BY COALESCE(CAST(a AS NVARCHAR),'NULL')
Building off of Alberto, I added the rollup.
SELECT [Narrative] = CASE
WHEN [Narrative] IS NULL THEN 'count_total' ELSE [Narrative] END
,[Count]=SUM([Count]) FROM (SELECT COUNT(*) [Count], 'count_nulls' AS [Narrative]
FROM [CrmDW].[CRM].[User]
WHERE [EmployeeID] IS NULL
UNION
SELECT COUNT(*), 'count_not_nulls ' AS narrative
FROM [CrmDW].[CRM].[User]
WHERE [EmployeeID] IS NOT NULL) S
GROUP BY [Narrative] WITH CUBE;
SELECT
ALL_VALUES
,COUNT(ALL_VALUES)
FROM(
SELECT
NVL2(A,'NOT NULL','NULL') AS ALL_VALUES
,NVL(A,0)
FROM US
)
GROUP BY ALL_VALUES
select count(isnull(NullableColumn,-1))
if its mysql, you can try something like this.
select
(select count(*) from TABLENAME WHERE a = 'null') as total_null,
(select count(*) from TABLENAME WHERE a != 'null') as total_not_null
FROM TABLENAME
Just in case you wanted it in a single record:
select
(select count(*) from tbl where colName is null) Nulls,
(select count(*) from tbl where colName is not null) NonNulls
;-)
for counting not null values
select count(*) from us where a is not null;
for counting null values
select count(*) from us where a is null;
I created the table in postgres 10 and both of the following worked:
select count(*) from us
and
select count(a is null) from us
In my case I wanted the "null distribution" amongst multiple columns:
SELECT
(CASE WHEN a IS NULL THEN 'NULL' ELSE 'NOT-NULL' END) AS a_null,
(CASE WHEN b IS NULL THEN 'NULL' ELSE 'NOT-NULL' END) AS b_null,
(CASE WHEN c IS NULL THEN 'NULL' ELSE 'NOT-NULL' END) AS c_null,
...
count(*)
FROM us
GROUP BY 1, 2, 3,...
ORDER BY 1, 2, 3,...
As per the '...' it is easily extendable to more columns, as many as needed
Number of elements where a is null:
select count(a) from us where a is null;
Number of elements where a is not null:
select count(a) from us where a is not null;