I have a table with high number of value columns which looks like:
CREATE TABLE tbl ("timestamp" timestamp PRIMARY KEY, "a" real, "b" real, "c" real)
The table may contain a dynamic number of value columns (like a,b,c).
I need to delete rows with where all values are null, but timestamp is not null.
I am unable to build a select query which will return null rows only.
Doing something like:
select *
from tbl
where tbl is null
will not work, because timestamp is not null
I tried to take the following working example:
select *
from tbl
where not (a,b,c) is null
and add a subselect to it:
select *
from tbl
where not (
SELECT string_agg(column_name, ',')
FROM information_schema.columns
WHERE table_name = 'tbl' and column_name != 'timestamp'
) is null
But it is not working.
You can convert the row to a JSONB object, remove the "timestamp" column and then check for an empty value:
select *
from tbl
where jsonb_strip_nulls(to_jsonb(tbl) - 'timestamp') = '{}'::jsonb;
This can directly be used to delete the rows:
delete from tbl
where jsonb_strip_nulls(to_jsonb(tbl) - 'timestamp') = '{}'::jsonb);
Related
I have a table, let's call it A. I want to delete rows with IDs 1 and 2 from that table. For that, I created a table variable #B, containing values 1 and 2 but that column I will name PK.
Now I do this:
DELETE FROM A
WHERE ID IN (
SELECT ID
FROM #B
)
Notice my (deliberate) programming error. In the sub-select, I have used a wrong column name. Accidentally it is the same name used in table A.
This should result in an 'invalid column name' error, right? Except it does not. It executes. Not only that, all data from table A gets deleted. As if there is no more predicate.
I have created a full demo script:
-- What happened to my data???
IF OBJECT_ID('tempdb..#JustATable') IS NOT NULL
DROP TABLE #JustATable
CREATE TABLE #JustATable (
PK INT NOT NULL PRIMARY KEY IDENTITY(1, 1),
ID INT NOT NULL,
NOTE VARCHAR(100) NOT NULL
)
INSERT INTO #JustATable (ID, NOTE)
SELECT database_id, DB_NAME(database_id)
FROM sys.databases;
SELECT NULL [inserted all the rows from sys.databases into the temptable], *
FROM #JustATable;
DECLARE #JustATableVariable TABLE (
PK INT NOT NULL PRIMARY KEY IDENTITY(1, 1),
ID_2 INT NOT NULL,
NOTE VARCHAR(100) NOT NULL
)
INSERT INTO #JustATableVariable (ID_2, NOTE)
SELECT database_id, DB_NAME(database_id)
FROM sys.databases
WHERE database_id = 2;
SELECT NULL [this is my table variable data], *
FROM #JustATableVariable;
DELETE FROM #JustATable
WHERE ID IN (
SELECT ID_2
FROM #JustATableVariable
);
SELECT NULL [I have just removed tempdb from the temptable], *
FROM #JustATable;
DELETE FROM #JustATable
WHERE ID IN (
SELECT ID /* this is the wrong column name but the same name as used in the temptable column */
FROM #JustATableVariable
);
SELECT NULL [wait...where is my data?], *
FROM #JustATable;
Can someone explain to me what is going on here? Has anyone seen this behavior before? Could this be a bug?
In the subquery ... (select id from #b) the column id is not fully qualified. So according to SQL specs, the RDBMS will first see if id column exists in table #b. If it does not, it will search "upwards" until it finds the id column in table a. The query is effectively identical to:
delete from a where id in (select a.id from #b)
Syntactically correct, semantically wrong.
I have that query:
DECLARE #test AS varchar =
(select * from users where Usr_ID in
(select Doc_Shortstringcolumn1 from Documents where Doc_ID = 11931))
And I've got an error "Only one expression can be specified in the select list when the subquery is not introduced with EXISTS." The result of select statement are three numbers, which i need to split into three rows.
(select * from users where Usr_ID in
(select Doc_Shortstringcolumn1 from Documents where Doc_ID = 11931))
you query returns multiple rows as a result variable can not contain multiple rows value.
if your query just return one value then it will return correct result
but if you change your query like below then it will works
DECLARE #test AS varchar =
(select top 1 Doc_Shortstringcolumn1 from users where Usr_ID in
(select Doc_Shortstringcolumn1 from Documents where Doc_ID = 11931)
)
Looking at your Query, It seems that you are trying to store a table into a variable which is of Varchar data type which you cannot do in SQL Server.
There are 2 possible solutions
1.
You May select only the Required filed instead of the * ad in that case if there are more than one row is returned, then only the first one will be stored in the variable and all other values will be ignored.
If this is ok with you then you may go ahead with this approach
DECLARE #test AS varchar =
select
#test = YourColumnName
from users where Usr_ID in
(
select Doc_Shortstringcolumn1 from Documents where Doc_ID = 11931
)
2
The Second approach is to use a table variable or a Temporary table to store the values so that you can store all the values and retrieve the same when needed.
Using Temp Table
select
*
into #temp
from users where Usr_ID in
(
select Doc_Shortstringcolumn1 from Documents where Doc_ID = 11931
)
using Table Variable
DECLARE #test AS TABLE
(
Column1 VARCHAR(50),
Column2 VARCHAR(50),
Column3 VARCHAR(50)
)
INSERT INTO #Test
(
Column1,
Column2,
Column3
)
select
YourColumnName1,
YourColumnName2,
YourColumnName3
from users where Usr_ID in
(
select Doc_Shortstringcolumn1 from Documents where Doc_ID = 11931
)
I have a table with 20 columns .How do i know if any of the column contains null values. And in case if there are nulls ,how to get count of them.
Use jsonb functions:
create table my_table(id int primary key, val numeric, str text, day date);
insert into my_table values
(1, 10, 'a', '2018-01-01'),
(2, 20, 'b', null),
(3, 30, null, null),
(4, null, null, null);
select key as column, count(*) as null_values
from my_table t
cross join jsonb_each_text(to_jsonb(t))
where value is null
group by key;
column | null_values
--------+-------------
val | 1
str | 2
day | 3
(3 rows)
Working example in rextester.
This Query should Create a query to do that:
SELECT 'SELECT ' || string_agg('count(' || quote_ident(attname) || ') as '||attname||'_not_null_count, count(case when ' || quote_ident(attname) || ' is null then 1 end) as '||attname||'_null_count', ', ')
|| ' FROM ' || attrelid::regclass
FROM pg_attribute
WHERE attrelid = 'myTable'::regclass --> Change myTable to your table name
AND attnum >= 1 -- exclude tableoid & friends (neg. attnum)
AND attisdropped is FALSE -- exclude deleted columns
GROUP BY attrelid;
You can after that transpose columns to rows on excel.
count(nmuloc) only counts rows where the column nmuloc IS NOT NULL. count(*) counts all rows, regardless of anything being NULL or not. So the difference of them is the count of rows where nmuloc IS NULL.
SELECT count(*) - count(nmuloc1) count_of_nulls_in_nmuloc1,
...
count(*) - count(nmuloc20) count_of_nulls_in_nmuloc20
FROM elbat;
You can see that in all_tab_cols, once the table is analyzed or gathered stats on that table.
select COLUMN_NAME, NUM_NULLS from all_tab_cols where table_name = 'tablename'
You can have sql query to get that details like below -
select 'col1Name', count(col1Name) from table where col1Name is null
union
select 'col2Name', count(col2Name) from table where col2Name is null
union ...
select 'col20Name', count(col20Name) from table where col20Name is null
If it is oracle, then you can write some dynamic SQL in stored procedure as well.
When I create a temp table using a select into in SQL Server, is there a way to specify that a column should be nullable? I have a multi-step process where I'm making a temp table by selecting a lot of columns (which is why I'm not doing a create table #tmp (...)). After I make that temp table, I'm updating some columns and some of those updates might null out a field.
I know I could do an alter table alter column statement to achieve what I want, but I'm curious about whether there's a way to specify this in the select itself. I know you can inline cast your columns to get the desired datatype, but I can't see how you specify nullability.
Nullability is inherited from the source column.
You can lose or gain nullability with an expression:
Example (constant literals appear to be problematic - need a good NOOP function which can return NULL):
CREATE TABLE SO5465245_IN
(
a INT NOT NULL
,b INT NULL
) ;
GO
SELECT COALESCE(a, NULL) AS a
,ISNULL(b, 0) AS b
,COALESCE(10, NULL) AS c1
,COALESCE(ABS(10), NULL) AS c2
,CASE WHEN COALESCE(10, NULL) IS NOT NULL THEN COALESCE(10, NULL) ELSE NULL END AS c3
INTO SO5465245_OUT
FROM SO5465245_IN ;
GO
SELECT TABLE_NAME
,COLUMN_NAME
,IS_NULLABLE
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME LIKE 'SO5465245%'
ORDER BY TABLE_NAME
,ORDINAL_POSITION ;
GO
DROP TABLE SO5465245_IN ;
GO
DROP TABLE SO5465245_OUT ;
GO
This soulution I've recently come up with and though I should share:
select top 0
B.*
into
TargetTable
from
SourceTable as A
left join SourceTable as B on 1 = 0
This effectively creates a duplicated structure of SourceTable in TargetTable with all columns nullable (at least in sql2008).
CONVERT will make your columns nullable, and works for literals/constants too. Tested in SQL Server 2005/2008.
SELECT
SomeText = CONVERT(varchar(10), 'literal'),
SomeNumber = CONVERT(int, 0)
INTO SO5465245
INSERT SO5465245 VALUES (null, null)
SELECT TABLE_NAME, COLUMN_NAME, DATA_TYPE, IS_NULLABLE
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'SO5465245'
ORDER BY TABLE_NAME, ORDINAL_POSITION
DROP TABLE SO5465245
If you want to inherit nullablity for the destination column irrespective of the source table columns you can follow this query.
SELECT COLUMN1, COLUMN2, COLUMN3 INTO DestinationTable from SourceTable
if this was your query where COLUMN1,COLUMN2,COLUMN3 were not nullable in SourceTable then change the query as
SELECT NULL COLUMN1, NULL COLUMN2, NULL COLUMN3 INTO DestinationTable from SourceTable
so, this will allow you to insert null values in to the Destination table.
I recently had the same issue - I wanted to use "select into", wanted all columns in the target table to be nullable & a repeatable approach where I didn't have to know the names of the fields in the source table.
select *
into dbo.I_Data
from
(select 1[Z_1]) A
full join (select null[Z_2], * from dbo.S_Data) B on A.Z_1 = B.Z_2
where
dbo.S_Data is the source data table and
[Z_1] & [Z_2] are two dummy columns used for the join
Then to clean up:
(a) Remove the row of nulls
delete dbo.I_Data where [Z_1] = 1
(b) Remove the dummy fields:
alter table dbo.I_Data
drop column [Z_1], [Z_2]
Regards.
Lets say I have a table with following columns:
Id,TableName,Columnname
Normally If I would like to do a select with ordering on TableName Primary and ColumnName secondary I would do
Select * From MyTable Order By TableName,ColumnName
Now say I would like a special case of ordering depending on TableName
Select * From MyTable Order By Case
When TableName = 'Foo' Then '1'
When TableName = 'Bar' Then '2'
When TableName = 'Test' Then '3'
ELSE 1000 END;
How do I combine the second alternative with the first alternative, that is special ordering on first TableName and then order the Columnnames with that TableName?
I'm running Microsoft SQL
Thanks in advance.
Just add column in order list, like any other ORDER BY :)
Select * From MyTable Order By Case
When TableName = 'Foo' Then '1'
When TableName = 'Bar' Then '2'
When TableName = 'Test' Then '3'
ELSE 1000 END, ColumnName;
The case method is fine. I would often use this short-cut:
Select *
From MyTable
Order By charindex(TableName, ',Test,Bar,Foo,') desc
This is a wee bit arcane. charindex() returns the position of the name in the string. So, this form will not work for all string values, but it can be generalized:
Order By charindex(',' + TableName + ',', ',Test,Bar,Foo,') desc
The reversal of the list is to handle missing values. These are assigned a value of 0, so by putting the values in reverse order, the missing ones come last. Note that if I know all the values, then I would write this as:
Order By charindex(TableName, ',Foo,Bar,Test,') asc
As alternative, you could
1) create a new table
CREATE TABLE_ORDER (TABLE_NAME VARCHAR(100) NOT NULL, ORDER_VAL INT);
ALTER TABLE TABLE_ORDER ADD CONSTRAINT TABLE_ORDER_PK PRIMARY KEY(TABLE_NAME);
2) insert into it appropriate values:
INSERT INTO TABLE_ORDER (TABLE_NAME, ORDER_VAL) VALUES ('Foo', 1);
INSERT INTO TABLE_ORDER (TABLE_NAME, ORDER_VAL) VALUES ('Bar', 2);
INSERT INTO TABLE_ORDER (TABLE_NAME, ORDER_VAL) VALUES ('Test', 3);
3) Use the following query
SELECT A.*
FROM MYTABLE A
LEFT JOIN TABLE_ORDER B ON A.TABLENAME= B.TABLE_NAME
ORDER BY B.ORDER_VAL, A.COLUMNNAME