I have a query that's written dynamically (OO PHP via Joomla) to insert some values into a MySQL database. The form that a user fills out has a field on it for dollar amount, and if they leave that blank I want the value going into the system to be NULL. I've written out the query to the error log as it's running; this is what the query looks like:
INSERT INTO arrc_Voucher
(VoucherNbr,securityCode,sequentialNumber, TypeFlag, CreateDT, ActivatedDT, BalanceInit, BalanceCurrent, clientName)
VALUES
('6032100199108006', '99108006','12','V','2010-10-29 12:50:01','NULL','NULL','NULL','')
When I look in the database table, though, although ActivatedDT is set correctly to NULL, BalanceInit and BalanceCurrent are both 0.00. The ActivatedDT field is a datetime, while the other two are decimal(18,2), and all three are set in the table structure as default value NULL.
If I run a query like this:
UPDATE arrc_Voucher
SET BalanceInit = null
WHERE BalanceInit like "0%"
...it does set the value to null, so why isn't the initial insert query doing so? Is it because null is in quotes? And if so, why is it setting correctly for ActivatedDT?
remove the quotes around NULL. What's actually happening is it's trying to insert the string 'NULL' as a number, and since it can't be converted to a number it uses the default value 0.
As for why ActivatedDT works, I'm guessing that's a date field. Failure to convert a string into a date would normally result in setting the value to 0 (which gets formatted as something like '1969-12-31'), but if you have NO_ZERO_DATE mode enabled, then it would be set to NULL instead.
If you'd like MySQL to throw an error in cases like this, when invalid values are passed, you can set STRICT_ALL_TABLES or STRICT_TRANS_TABLES (make sure you read the part about the difference between them) or one of the emulation modes, like TRADITIONAL.
You can try this with the command SET sql_mode='TRADITIONAL', or by adding sql-mode="TRADITIONAL" in my.cnf.
When you insert NULL into a MySQL database, you cannot insert it with quotes around it. It tries to insert the varchar 'NULL'. If your idea worked, you would never be able to insert the actual word NULL into the DB.
Remove the single quotes when you want to insert NULL.
You are not setting the fields to NULL but to strings ('NULL').
Related
I am trying to replace a NULL date with a blank. I end up with invalid date string. Ive Tried COALESCE, ISNULL, IFNULL, CASE STATEMENTS and nothing seems to work. I am querying a LINKED PROGRESS ODBC connection and using
declare #Data varchar(max)
set #Data= N'
SELECT MyCode
FROM TABLE
'
exec (#Data ) AT PROGRESS;
Ive done this many times before, I can do ISNULL, COALESCE etc just fine on all my other fields, but not the case with this Date field. Any help is greatly appreciated
I understand you are accessing a Progress database.
The Progress unknown value ? is what SQL calls NULL.
Date fields can only contain valid dates or (if the field is not mandatory) the unknown value ?. Unlike other data types, unknown values are sometimes displayed as blanks instead of ?, for example when displayed as GUI widgets.
There are no actual blank dates in Progress.
Unfortunately I don't know enough SQL to tell exactly what you are trying to achieve.
I had to make my query an open query and in the outer select do ISNULL(datefield,'')
We have a statement in some legacy SQL Server 2005 code like
insert into myTable
select distinct
wherefield1,
wherefield2,
anotherfield,
convert(numeric(10,2), varcharfield1),
convert(numeric(10,2), varcharfield2),
convert(numeric(10,2), varcharfield3),
convert(datetime, varcharfield4),
otherfields
from myStagingTable
where insertflag='true'
and wherefield1 = #wherevalue1
and wherefield2 = #wherevalue2
Earlier in the code, a variable is set to determine whether varcharfield1 or varcharfield2 is null, and the insert is programmed to execute as long as one of them is not null.
We know that if varcharfield1, varcharfield2, or varcharfield3 is a nonnumeric character string, an exception will be thrown and the insert will not occur. But I am perplexed by the behavior when one of these variables is null, as it often is. Actually, it is always the case that one of these values is null. But it seems that the insertion does take place. It looks like the legacy code relies on this to prevent only insertion of nonnumeric character data, while allowing insertion of null or empty values (in an earlier step, all empty strings in these fields of myStagingTable are replaced with null values).
This has been running on a Production SQL Server 2005 instance with all default settings for a number of years. Is this behavior we can rely on if we upgrade to a newer version of SQL Server?
Thanks,
Rebeccah
conversion of NULL to anything is still NULL. If the column allows NULL, that's what you'll get. If the column is not nullable, it will fail.
You can see this yourself without even doing an INSERT. Just run this:
SELECT CONVERT(numeric(10,2), NULL)
and note how it produces a NULL result. Then run this:
SELECT CONVERT(numeric(10,2), 'x')
and note how it throws an error message instead of returning anything.
i have created a table with 3 columns in which 2 are defined as NOT NULL in SQL, now i created a new column i wanted to insert values to only the new variable, but i'm having an error while using Insert into statement
If you are using MySQL, here is your answer:
Inserting NULL into a column that has been declared NOT NULL. For multiple-row INSERT statements or INSERT INTO ... SELECT statements, the column is set to the implicit default value for the column data type. This is 0 for numeric types, the empty string ('') for string types, and the “zero” value for date and time types. INSERT INTO ... SELECT statements are handled the same way as multiple-row inserts because the server does not examine the result set from the SELECT to see whether it returns a single row. (For a single-row INSERT, no warning occurs when NULL is inserted into a NOT NULL column. Instead, the statement fails with an error.)
The documentation is here: MySQL 5.7 Reference Manual: Insert
As said above, just use ' ' and 0 depending on the type of column you have. I believe it's the same for any other DBMS system anyone could use.
Disregarding that this normally indicates you should rethink your database design, some database engines will let you do this through temporary schema manipulation (removing and afterwards re-adding the 'not null' constraint).
Alternatively you could either define default values for the not null columns, or pass in through the insert command the appropriate "default" values, or select a database mode where either the checks are not enforced (thus allowing nulls) or default values are automatically generated (eg mysql in non strict mode which replaces the nulls with a calculated default value).
The only valid use case I can think of is replicating the situation where a database has rows with nulls in certain fields, and then the schema is changed to make those columns not null. Mysql for instance will allow you to add 'not null' constraints to columns that already have data with nulls in that column.
How do I alter a sql varchar column to a decimal column when there are nulls in the data?
I thought:
ALTER TABLE table1
ALTER COLUMN data decimal(19,6)
But I just get an error, I assume because of the nulls:
Error converting data type varchar to numeric. The statement has been terminated.
So I thought to remove the nulls I could just set them to zero:
ALTER TABLE table1
ALTER COLUMN data decimal(19,6) NOT NULL DEFAULT 0
but I dont seem to have the correct syntax.
Whats the best way to convert this column?
edit
People have suggested it's not the nulls that are causing me the problem, but non-numeric data. Is there an easy way to find the non-numeric data and either disregard it, or highlight it so I can correct it.
If it were just the presence of NULLs, I would just opt for doing this before the alter column:
update table1 set data = '0' where data is null
That would ensure all nulls are gone and you could successfully convert.
However, I wouldn't be too certain of your assumption. It seems to me that your new column is perfectly capable of handling NULL values since you haven't specified not null for it.
What I'd be looking for is values that aren't NULL but also aren't something you could turn in to a real numeric value, such as what you get if you do:
insert into table1 (data) values ('paxdiablo is good-looking')
though some may argue that should be treated a 0, a false-y value :-)
The presence of non-NULL, non-numeric data seems far more likely to be causing your specific issue here.
As to how to solve that, you're going to need a where clause that can recognise whether a varchar column is a valid numeric value and, if not, change it to '0' or NULL, depending on your needs.
I'm not sure if SQL Server has regex support but, if so, that'd be the first avenue I'd investigate.
Alternatively, provided you understand the limitations (a), you could use isnumeric() with something like:
update table1 set data = NULL where isnumeric(data) = 0
This will force all non-numeric values to NULL before you try to convert the column type.
And, please, for the love of whatever deities you believe in, back up your data before attempting any of these operations.
If none of those above solutions work, it may be worth adding a brand new column and populating bit by bit. In other words set it to NULL to start with, and then find a series of updates that will copy data to this new column.
Once you're happy that all data has been copied, you should then have a series of updates you can run in a single transaction if you want to do the conversion in one fell swoop. Drop the new column and then do the whole lot in a single operation:
create new column;
perform all updates to copy data;
drop old column;
rename new column to old name.
(a) From the linked page:
ISNUMERIC returns 1 for some characters that are not numbers, such as plus (+), minus (-), and valid currency symbols such as the dollar sign ($).
Possible solution:
CREATE TABLE test
(
data VARCHAR(100)
)
GO
INSERT INTO test VALUES ('19.01');
INSERT INTO test VALUES ('23.41');
ALTER TABLE test ADD data_new decimal(19,6)
GO
UPDATE test SET data_new = CAST(data AS decimal(19,6));
ALTER TABLE test DROP COLUMN data
GO
EXEC sp_RENAME 'test.data_new' , 'data', 'COLUMN'
As people have said, that error doesn't come from nulls, it comes from varchar values that can't be converted to decimal. Most typical reason for this I've found (after checking that the column doesn't contain any logically false values, like non-digit characters or double comma values) is when your varchar values use comma for decimal pointer, as opposed to period.
For instance, if you run the following:
DECLARE #T VARCHAR(256)
SET #T = '5,6'
SELECT #T, CAST(#T AS DEC(32,2))
You will get an error.
Instead:
DECLARE #T VARCHAR(256)
SET #T = '5,6'
-- Let's change the comma to a period
SELECT #T = REPLACE(#T,',','.')
SELECT #T, CAST(#T AS DEC(32,2)) -- Now it works!
Should be easy enough to look if your column has these cases, and run the appropriate update before your ALTER COLUMN, if this is the cause.
You could also just use a similar idea and make a regex search on the column for all values that don't match digit / digit+'.'+digit criteria, but i suck with regex so someone else can help with that. :)
Also, the american system uses weird separators like the number '123100.5', which would appear as '123,100.5', so in those cases you might want to just replace the commas with empty strings and try then?
The question
Is it possible to ask SSIS to cast a value and return NULL in case the cast is not allowed instead of throwing an error ?
My environment
I'm using Visual Studio 2005 and Sql Server 2005 on Windows Server 2003.
The general context
Just in case you're curious, here is my use case. I have to store data coming from somewhere in a generic table (key/value structure with history) witch contains some sort of value that can be strings, numbers or dates. The structure is something like this :
table Values {
Id int,
Date datetime, -- for history
Key nvarchar(50) not null,
Value nvarchar(50),
DateValue datetime,
NumberValue numeric(19,9)
}
I want to put the raw value in the Value column and try to put the same value
in the DateValue column when i'm able to cast it to Datetime
in the NumberValue column when i'm able to cast it to a number
Those two typed columns would make all sort of aggregation and manipulation much easier and faster later.
That's it, now you know why i'm asking this strange question.
============
Thanks in advance for your help.
You could also try a Derived Column component and test the value of the potential date/number field or simply cast it and redirect any errors as being the NULL values for these two fields.
(1) If you just simply cast the field every time with a statement like this in the Derived Column component: (DT_DATE)[MYPOTENTIALDATE] - you can redirect the rows that fail this cast and manipulate the data from there.
OR
(2) You can do something like this in the Derived Column component: ISNULL([MYPOTENTIALDATE]) ? '2099-01-01' : (DT_DATE)[MYPOTENTIALDATE]. I generally send through '2099-01-01' when a date is NULL rather than messing with NULL (works better with Cubes, etc).
Of course (2) won't work if the [MYPOTENTIALDATE] field comes through as other things other than a DATETIME or NULL, i.e., sometimes it is a word like "hello".
Those are the options I would explore, good luck!
In dealing with this same sort of thing I found the error handling in SSIS was not specific enough. My approach has been to actually create an errors table, and query a source table where the data is stored as varchar, and log errors to the error table with something like the below. I have one of the below statements for each column, because it was important for me to know which column failed. Then after I log all errors, I do a INSERT where I select those records in SomeInfo that do not have an errors. In your case you could do more advanced things based on the ColumnName in the errors table to insert default values.
INSERT INTO SomeInfoErrors
([SomeInfoId]
,[ColumnName]
,[Message]
,FailedValue)
SELECT
SomeInfoId,
'PeriodStartDate',
'PeriodStartDate must be in the format MM/DD/YYYY',
PeriodStartDate
FROM
SomeInfo
WHERE
ISDATE(PeriodStartDate) = 0 AND [PeriodStartDate] IS NOT NULL;
Tru using a conditional split and have the records where the data is a date go along one path and the other go along a different path where they are updated to nullbefore being inserted.