we created a snowflake table as shown and loaded data into into from flat file using copy into command
create or replace table temp.T_ERROR
(
ID NUMBER(38,0) Primary Key,
ERROR varchar(4) collate 'en-rtrim' NOT NULL,
)
so we saw rtrimmed values in ERROR then we tried to transfer these results into another snowflake table with no collate option set on this table
create or replace table DATA.T_ERROR_1
(
ID NUMBER(38,0) Primary Key,
ERROR varchar(4) NOT NULL,
)
ISSUE:
the DATA.T_ERROR_1 table is not getting the trimmed values from temp table instead its getting un-trimmed values which are in the original flat file
is there any other ways i can do this transfer which doesnt involve me writing rtrim ltrim or trim or every column
None of your tables have trimmed data. It only appears so in certain circumstances.
The table temp.T_ERROR has trimmed comparison semantics, and that means it compares & sorts as if it was trimmed. But assignment is not comparison, and the original value in temp.T_ERROR (which is space padded) is copied into the new table DATA.T_ERROR_1. You can check that the spaces are there with eg
SELECT '"'||ERROR||'"', LENGTH(ERROR) FROM temp.T_ERROR
As Mike Walton says elsewhere, if you have a chance to trim the spaces before the data enters Snowflake, your problems may be solved. If however you are porting an application that relies heavily on CHAR behaviour, you might be better off defining COLLATE for all the CHAR columns.
I've tested on other databases (ie Oracle) and it behaves the same way as Snowflake: if you copy from CHAR to VARCHAR you get the padding spaces, too.
Related
I am using SQL Server 2019 (v15.0.2080.9) and I've got a simple task but somehow I am not able to solve it...
I have a little database with one table containing a first name and a last name column
CREATE TABLE [dbo].[person]
(
[first_name] [nchar](200) NULL,
[last_name] [nchar](200) NULL,
[display_name] [nchar](400) NULL
) ON [PRIMARY]
GO
and I want to store the combination of first name with an extra whitespace in between in the third column (yes I really have to do that...).
So I thought I might use the CONCAT function
UPDATE [dbo].[person]
SET display_name = CONCAT(first_name, ' ', last_name)
But my display_name column is only showing me the first name... so what's wrong with my SQL?
Kind regards
Sven
Your method should work and does work. The issue, though is that the data types are nchar() instead of nvarchar(). That means they are padded with spaces and the last name is starting at position 201 in the string.
Just fix the data type.
In addition, I would suggest that you use a computed column:
alter table person add display_name as (concat(first_name, ' ', last_name));
This ensures that the display_name is always up-to-date -- even when the names change.
Here is a db<>fiddle.
As a note: char() and nchar() are almost never appropriate. The one exception is when you have fixed length strings, such as state or country abbreviations or account codes. In almost all cases, you want varchar() or nvarchar().
How to change column type from varchar2(10) to char(10) without losing existing data using oracle developer?
ALTER TABLE TBL_NAME
MODIFY (CRTE CHAR(10));
Will it impact existing data?
It should not impact existing data.
See this SQL Fiddle.
I would reconsider changing varchar2 for char. Char will take up more space on HDD as char will always take N bytes. And if your strings are not execatly 10 characters, all of them char will be pain for searches.
There is really no reason to do this. As this "Ask Tom" answer, explains, the two are stored equivalently.
If you want to ensure that the value has exactly 10 characters, then use a check constraint:
ALTER TABLE TBL_NAME ADD CONSTRAINT CHECK (LEN(CRTE) = 10);
The difference is padding the string in result sets, and that is often better handled on an ad-hoc basis. Trailing spaces can be quite tricky to deal with.
If you really want to change the column type, you can use:
alter table t modify ( x char(10) );
This should be safe with existing data, because you are not reducing the length of the column.
I have a table with a column configured to hold nvarchar data type.
I am trying to add a row using
INSERT INTO TABLE_NAME VALUES (value1, value2...)
Sql-server gets stuck on a 180 character string that I am trying to assign to the nvarchar data type column returning:
Error: The identifier that starts with [part of string] is too long.
Maximum length is 128.
I don't understand why this is happening since nvarchar(max) should hold 2GByte of storage as I read here: What is the maximum characters for the NVARCHAR(MAX)?
Any ideas of what I've got wrong here?
UPDATE:
The table was created with this:
CREATE TABLE MED_DATA (
MED_DATA_ID INT
,ORDER_ID INT
,GUID NVARCHAR
,INPUT_TXT NVARCHAR
,STATUS_CDE CHAR
,CRTE_DTM DATETIME
,MOD_AT_DTM DATETIME
,CHG_IN_REC_IND CHAR
,PRIMARY KEY (MED_DATA_ID)
)
And my actual INSERT statement is as follows:
INSERT INTO MED_DATA
VALUES (
5
,12
,"8fd9924"
,"{'firstName':'Foo','lastName':'Bar','guid':'8fd9924','weightChanged':false,'gender':'Male','heightFeet':9,'heightInches':9,'weightPounds':999}"
,"PENDING"
,"2017-09-02 00:00:00.000"
,"2017-09-02 00:00:00.000"
,NULL
)
By default, double quotes in T-SQL do not delimit a string. They delimit an identifier. So you cannot use double quotes here. You could change the default but shouldn't.
If this is being directly written in a query window, use single quotes for strings and then double up quotes within the string to escape them:
INSERT INTO MED_DATA VALUES (5, 12, '8fd9924', '{''firstName'':''Foo'',''lastName'':''Bar'',''guid'':''8fd9924'',''weightChanged'':false,''gender'':''Male'',''heightFeet'':9,''heightInches'':9,''weightPounds'':999}', 'PENDING', '2017-09-02T00:00:00.000', '2017-09-02T00:00:00.000', NULL)
But if, instead, you're passing this string across from another program, it's time to learn how to use parameterized queries. That'll also allow you to pass the dates across as dates and not rely on string parsing to reconstruct them correctly.
Also, as noted, you need to fix your table definitions because they've currently nvarchar which means the same as nvarchar(1).
Are you aware of what an Identifier is? Here is a hint - it is a NAME. SQL Server is not complaining about your data, it is complaining about a field or table name. SOmehow your SQL must be totally borked so that part of the text is parsed as name of a field or table. And yes, those are limited to 128 characters.
This is clear in the error message:
Error: The identifier
clearly states it is an identifier issue.
How do I alter a sql varchar column to a decimal column when there are nulls in the data?
I thought:
ALTER TABLE table1
ALTER COLUMN data decimal(19,6)
But I just get an error, I assume because of the nulls:
Error converting data type varchar to numeric. The statement has been terminated.
So I thought to remove the nulls I could just set them to zero:
ALTER TABLE table1
ALTER COLUMN data decimal(19,6) NOT NULL DEFAULT 0
but I dont seem to have the correct syntax.
Whats the best way to convert this column?
edit
People have suggested it's not the nulls that are causing me the problem, but non-numeric data. Is there an easy way to find the non-numeric data and either disregard it, or highlight it so I can correct it.
If it were just the presence of NULLs, I would just opt for doing this before the alter column:
update table1 set data = '0' where data is null
That would ensure all nulls are gone and you could successfully convert.
However, I wouldn't be too certain of your assumption. It seems to me that your new column is perfectly capable of handling NULL values since you haven't specified not null for it.
What I'd be looking for is values that aren't NULL but also aren't something you could turn in to a real numeric value, such as what you get if you do:
insert into table1 (data) values ('paxdiablo is good-looking')
though some may argue that should be treated a 0, a false-y value :-)
The presence of non-NULL, non-numeric data seems far more likely to be causing your specific issue here.
As to how to solve that, you're going to need a where clause that can recognise whether a varchar column is a valid numeric value and, if not, change it to '0' or NULL, depending on your needs.
I'm not sure if SQL Server has regex support but, if so, that'd be the first avenue I'd investigate.
Alternatively, provided you understand the limitations (a), you could use isnumeric() with something like:
update table1 set data = NULL where isnumeric(data) = 0
This will force all non-numeric values to NULL before you try to convert the column type.
And, please, for the love of whatever deities you believe in, back up your data before attempting any of these operations.
If none of those above solutions work, it may be worth adding a brand new column and populating bit by bit. In other words set it to NULL to start with, and then find a series of updates that will copy data to this new column.
Once you're happy that all data has been copied, you should then have a series of updates you can run in a single transaction if you want to do the conversion in one fell swoop. Drop the new column and then do the whole lot in a single operation:
create new column;
perform all updates to copy data;
drop old column;
rename new column to old name.
(a) From the linked page:
ISNUMERIC returns 1 for some characters that are not numbers, such as plus (+), minus (-), and valid currency symbols such as the dollar sign ($).
Possible solution:
CREATE TABLE test
(
data VARCHAR(100)
)
GO
INSERT INTO test VALUES ('19.01');
INSERT INTO test VALUES ('23.41');
ALTER TABLE test ADD data_new decimal(19,6)
GO
UPDATE test SET data_new = CAST(data AS decimal(19,6));
ALTER TABLE test DROP COLUMN data
GO
EXEC sp_RENAME 'test.data_new' , 'data', 'COLUMN'
As people have said, that error doesn't come from nulls, it comes from varchar values that can't be converted to decimal. Most typical reason for this I've found (after checking that the column doesn't contain any logically false values, like non-digit characters or double comma values) is when your varchar values use comma for decimal pointer, as opposed to period.
For instance, if you run the following:
DECLARE #T VARCHAR(256)
SET #T = '5,6'
SELECT #T, CAST(#T AS DEC(32,2))
You will get an error.
Instead:
DECLARE #T VARCHAR(256)
SET #T = '5,6'
-- Let's change the comma to a period
SELECT #T = REPLACE(#T,',','.')
SELECT #T, CAST(#T AS DEC(32,2)) -- Now it works!
Should be easy enough to look if your column has these cases, and run the appropriate update before your ALTER COLUMN, if this is the cause.
You could also just use a similar idea and make a regex search on the column for all values that don't match digit / digit+'.'+digit criteria, but i suck with regex so someone else can help with that. :)
Also, the american system uses weird separators like the number '123100.5', which would appear as '123,100.5', so in those cases you might want to just replace the commas with empty strings and try then?
I have a column which I believe has been declared wrongly. It contains data and I do not wish to lose the data.
I wish to change the definition from varchar(max) to varchar(an integer). I was under the impression I cannot just alter the column type?
Is the best method to create a temp column, "column2", transfer the data to this column, from the column with the problematic type, delete the problem column and then rename the temp column to the original problematic column?
If so, how do I copy the values from the problem column to the new column?
EDIT: For anyone with same problem, you can just use the ALTER statements.
As long as the data types are somewhat "related" - yes, you can absolutely do this.
You can change an INT to a BIGINT - the value range of the second type is larger, so you're not in danger of "losing" any data.
You can change a VARCHAR(50) to a VARCHAR(200) - again, types are compatible, size is getting bigger - no risk of truncating anything.
Basically, you just need
ALTER TABLE dbo.YourTable
ALTER COLUMN YourColumn VARCHAR(200) NULL
or whatever. As long as you don't have any string longer than those 200 characters, you'll be fine. Not sure what happens if you did have longer strings - either the conversion will fail with an error, or it will go ahead and tell you that some data might have been truncated. So I suggest you first try this on a copy of your data :-)
It gets a bit trickier if you need to change a VARCHAR to an INT or something like that - obviously, if you have column values that don't "fit" into the new type, the conversion will fail. But even using a separate "temporary" new column won't fix this - you need to deal with those "non-compatible" cases somehow (ignore them, leave NULL in there, set them to a default value - something).
Also, switching between VARCHAR and NVARCHAR can get tricky if you have e.g. non-Western European characters - you might lose certain entries upon conversion, since they can't be represented in the other format, or the "default" conversion from one type to the other doesn't work as expected.
Calculate the max data length store int that column of that table.
Select max(len(fieldname)) from tablename
Now you can decrease the size of that column up to result got in previous query.
ALTER TABLE dbo.YourTable
ALTER COLUMN YourColumn VARCHAR(200) NULL
According to the PostgreSQL docs, you can simply alter table
ALTER TABLE products ALTER COLUMN price TYPE numeric(10,2);
But here's the thing
This will succeed only if each existing entry in the column can be converted to the new type by an implicit cast. If a more complex conversion is needed, you can add a USING clause that specifies how to compute the new values from the old.
add a temp column2 with type varchar(NN), run update tbl set column2 = column, check if any error happens; if everything is fine, alter your original column, copy data back and remove column2.