I am reading a flat file in hive and i have null values coming in file like below
a|b|null|null|d
and when I create table on top of this with below datatypes
a char(1),b char(1),c char(1),varchar2(2),char(1)
and the value in table coming like this
a,b,n,nu,d
The oneway I can do this is to make the datatype as varchar2(4) and add check at null.
But is there any other way i can do this.
SerDe treats 'null' strings as normal values, no difference between value 'a' and 'null'.
Try to add 'serialization.null.format'='null' property to your table definition:
ALTER TABLE mytable SET tblproperties('serialization.null.format'='null');
Another approach is to use STRING data type and case statements is select:
select case when col = 'null' then null end as col
...
Related
I am using Oracle DB. At the database level, when you set a column value to either NULL or '' (empty string), the fetched value is NULL in both cases. Is it possible to store '' (empty string) as a non NULL value in the database?
I execute this
UPDATE contacts SET last_name = '' WHERE id = '1001';
commit;
SELECT last_name, ID FROM contacts WHERE id ='1001';
LAST_NAME ID
------------ ------
null 1001
Is it possible to store the last_name as a non-NULL empty string ('')?
The only way to do this in oracle is with some kind of auxiliary flag field, that when set is supposed to represent the fact that the value should be an empty string.
As far as i know Oracle does not distinguish between '' and NULL, see here.
Oracle has a well know behavior that it silently converts "" to NULL on INSERT and UPDATE statements.
You have to deal with this in your code to prevent this behavior by converting NULL to "" when you read the columns back in and just do not use null in your program to begin with.
A long time since I used Oracle, but I believe we used to use a single space ' ' to represent an empty string, then trim it after reading.
If you use a VARCHAR2 data type then NULL and '' are identical and you cannot distinguish between them; so, as mentioned in other answers, you would either need to:
Have an additional column that contains a flag that distinguishes between non-NULL and NULL values so that if then flag states it is non-NULL and it contains a NULL then you know it is an empty string; or
Use an alternate representation, such as a single space character, for an empty string. This would then mean that you cannot store a string with that alternate representation; however, if trailing white-space was syntactically invalid for the strings you are storing then using a single space character to represent an empty string would be fine.
If you are using a CLOB data type then you CAN store an empty string using the EMPTY_CLOB() function:
CREATE TABLE table_name (value CLOB);
INSERT INTO table_name (value) VALUES (NULL);
INSERT INTO table_name (value) VALUES (EMPTY_CLOB());
INSERT INTO table_name (value) VALUES ('A');
Then:
SELECT value, LENGTH(value) FROM table_name;
Outputs:
VALUE
LENGTH(VALUE)
null
null
0
A
1
db<>fiddle here
Hive converts Null to empty String in String Column. What is reason for that ?
As per our requirement, we need to see Null in string column instead of empty String otherwise Is Null not working for those columns.
So to solve this problem we set below property on Table:
TBLPROPERTIES('serialization.null.format'='')
But Still we see empty string instead of NULL and even SHOW TBLPROPERTIES also not showing this property in result, so i am not sure if this property is set or not.
I tried to set this property in DDL itself and even i tried
Alter Table <TableName> Set TBLPROPERTIES ('serialization.null.format' = '');
If needed create another table and store the values of this into that in this format. I mentioned -1 as example you can use anything of your choice.
in your select query
select
case when <col> is null then -1 else <col> end as <col>
from
table
How you are moving data?
If you are using sqoop then you can try passing below arguments
--input-null-string '\\N'
--input-null-non-string '\\N'
I'm creating an empty column in my table, when I use '' as columnname, it created empty column but String type. How can I create empty column but Integer/decimal type?
Null as columnname doesn't work!
Thank you!
Adding an INT column with NULLs inside a SELECT query:
SELECT
CAST(NULL AS INT) AS columnname
check this snippet which creates a new empty column of INT datatype
ALTER TABLE schema.tablename ADD column_a INT NULL ;
You can create empty column of integer type and null is not a string value:
SELECT
hat,
shoe,
boat,
0 as placeholder for number
FROM table
And '' as placeholder for strings.
And null as placeholder for null value.
Is it possible to copy data from column A to column B for all records in a table in SQL?
How about this
UPDATE table SET columnB = columnA;
This will update every row.
UPDATE table_name SET
destination_column_name=orig_column_name
WHERE condition_if_necessary
This will update all the rows in that columns if safe mode is not enabled.
UPDATE table SET columnB = columnA;
If safe mode is enabled then you will need to use a where clause.
I use primary key as greater than 0 basically all will be updated
UPDATE table SET columnB = columnA where table.column>0;
If you want to copy a column to another column with a different data type in PostgresSQL, you must cast/convert to the data type first, otherwise it will return
Query 1 ERROR: ERROR: column "test_date" is of type timestamp without
time zone but expression is of type character varying LINE 1: update
table_name set test_date = date_string_col
^ HINT: You will need to rewrite or cast the expression.
An example of converting varchar to timestamp:
update table_name set timestamp_col = date_string_col::TIMESTAMP;
An example of converting varchar to int:
update table_name set int_column = string_col::INTEGER;
but any column type(except file or the similar) can be copied to string(character varying) without cast the type.
I am using Oracle DB. At the database level, when you set a column value to either NULL or '' (empty string), the fetched value is NULL in both cases. Is it possible to store '' (empty string) as a non NULL value in the database?
I execute this
UPDATE contacts SET last_name = '' WHERE id = '1001';
commit;
SELECT last_name, ID FROM contacts WHERE id ='1001';
LAST_NAME ID
------------ ------
null 1001
Is it possible to store the last_name as a non-NULL empty string ('')?
The only way to do this in oracle is with some kind of auxiliary flag field, that when set is supposed to represent the fact that the value should be an empty string.
As far as i know Oracle does not distinguish between '' and NULL, see here.
Oracle has a well know behavior that it silently converts "" to NULL on INSERT and UPDATE statements.
You have to deal with this in your code to prevent this behavior by converting NULL to "" when you read the columns back in and just do not use null in your program to begin with.
A long time since I used Oracle, but I believe we used to use a single space ' ' to represent an empty string, then trim it after reading.
If you use a VARCHAR2 data type then NULL and '' are identical and you cannot distinguish between them; so, as mentioned in other answers, you would either need to:
Have an additional column that contains a flag that distinguishes between non-NULL and NULL values so that if then flag states it is non-NULL and it contains a NULL then you know it is an empty string; or
Use an alternate representation, such as a single space character, for an empty string. This would then mean that you cannot store a string with that alternate representation; however, if trailing white-space was syntactically invalid for the strings you are storing then using a single space character to represent an empty string would be fine.
If you are using a CLOB data type then you CAN store an empty string using the EMPTY_CLOB() function:
CREATE TABLE table_name (value CLOB);
INSERT INTO table_name (value) VALUES (NULL);
INSERT INTO table_name (value) VALUES (EMPTY_CLOB());
INSERT INTO table_name (value) VALUES ('A');
Then:
SELECT value, LENGTH(value) FROM table_name;
Outputs:
VALUE
LENGTH(VALUE)
null
null
0
A
1
db<>fiddle here