When you get this error, the first thing you ask is, which column? Unfortunately, SQL Server is no help here. So you start doing trial and error. Well, right now I have a statement like:
INSERT tbl (A, B, C, D, E, F, G)
SELECT A, B * 2, C, D, E, q.F, G
FROM tbl
,othertable q
WHERE etc etc
Note that
Some values are modified or linked in from another table, but most values are coming from the original table, so they can't really cause truncation going back to the same field (that I know of).
Eliminating fields one at a time eventually makes the error go away, if I do it cumulatively, but — and here's the kicker — it doesn't matter which fields I eliminate. It's as if SQL Server is objecting to the total length of the row, which I doubt, since there are only about 40 fields in all, and nothing large.
Anyone ever seen this before?
Thanks.
UPDATE: I have also done "horizontal" testing, by filtering out the SELECT, with much the same result. In other words, if I say
WHERE id BETWEEN 1 AND 100: Error
WHERE id BETWEEN 1 AND 50: No error
WHERE id BETWEEN 50 AND 100: No error
I tried many combinations, and it cannot be limited to a single row.
Although the table had no keys, constraints, indexes, or triggers, it did have statistics, and therein lay the problem. I killed all the table's stats using this script
http://sqlqueryarchive.blogspot.com/2007/04/drop-all-statistics-2005.html
And voila, the INSERT was back to running fine. Why are the statistics causing this error? I don't know, but that's another problem...
UPDATE: This error came back even with the stats deleted. Because I was convinced that the message itself was inaccurate (there is no evidence of truncation), I went with this solution instead:
SET ANSI_WARNINGS OFF
INSERT ...
SET ANSI_WARNINGS ON
Okay, it's more of a hack than a solution, but it allows me — and hopefully someone else — to move on to other things.
Is there a reason you can't simply cast the fields as the structural equivalent of their destination column like so:
Select Cast(A as varchar(42))
, Cast(B * 2 as Decimal(18,4))
, Cast(C As varchar(10))
...
From Table
The downside to this approach is that it will truncate the text values at their character limit. However, if you are "sure" that this shouldn't happen, then no harm will come.
In some cases you can run into a problem if you have any other column with default values which might cause the problem.
Ex. you might have added a column to trace the user who created the row, like USER_ENTERED with default value of suser_sname() but the column length is less than the current username.
There is a maximum row size limit in SQL Server 2005. See here.
Most of the time you'll run into this w/lots of nvarchar columns.
Yes, when I ran into this, I had to create another table/tables which mimic the current structure. I then did not change the code, but changed my data type sizes to all nvarchar (MAX) for each field till it stopped, then eliminated them one by one. Yes long and dragged out but I had major issues trying anything else. Once I tried a bunch of stuff that was causing too much of a headache I just decided to take the "Cave Man" Approach as we laughed about it later.
Also I have seen a similar issue with FKs, where you must ask:
What are the foriegn key constraints? Are there any?
Since there are not any , try this guy's DataMgr component:
http://www.bryantwebconsulting.com/blog/index.cfm/2005/11/21/truncated
Also check this out:
http://forums.databasejournal.com/showthread.php?t=41969
http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=138456
http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=97349
You could also dump the table you are selecting from to a temp table and find out what line gives the error if it errors out to a temp table.
It should tell you the line in the error, if you put each column on another line, it should tell you exactly where it is bombing.
If it looks like "total length" then do you have audit trigger concatenating the columns for logging?
Edit: after your update, I really would consider the fact you have a trigger causing this...
Edit 2, after seeing your statistics answer...
Because the total stats attribute length was probably greater then 900 bytes?
Not sure if this applies to statistics though and I'm not convinced.
Do you have a reference please because I'd like to know why stats would truncate when they are simply binary histograms (IIRC)
Related
Some background, I have a code column that is char(6). In this field, I have the values of 0,00,000,0000,000000,000000. It seems illogical but that's how it is. What i need to do is delete all rows that possess these code values. I know how to do it individually as such
delete from [dbo.table] where code='0'
delete from [dbo.table] where code='00'
and so on.
How does one do this one section of code instead of 6
Try this:
delete from [dbo.table] where code='0'
or code='00'
or code='000'
etc. You get the idea.
There can be more efficient ways when the set of vales gets larger, but your 5 or 6 values is still quite a ways from that.
Update:
If your list grows long, or if your table is significantly larger than can reside in cache, you will likely see a significant performance gain by storing your selection values into an indexed temporary table and joining to it.
It strongly depends on your DBMS, but I suggest to use regular expressions. For example, with MySQL you just need simple query like this:
delete from dbo.table where code regexp '(0+)'
For most of popular DBMS you can do the same, but syntax may be various
I can't test it right now, but the following should work:
DELETE FROM dbo.table WHERE CONVERT(int, code) = 0
edit- Just thought of another way, that should be safer:
DELETE FROM dbo.table WHERE LEN(code) > 0 AND LEFT(code + '0000000000', 10) = '0000000000'
I have a sample of a stored procedure like this (from my previous working experience):
Select * from table where (id=#id or id='-999')
Based on my understanding on this query, the '-999' is used to avoid exception when no value is transferred from users. So far in my research, I have not found its usage on the internet and other company implementations.
#id is transferred from user.
Any help will be appreciated in providing some links related to it.
I'd like to add my two guesses on this, although please note that to my disadvantage, I'm one of the very youngest in the field, so this is not coming from that much of history or experience.
Also, please note that for any reason anybody provides you, you might not be able to confirm it 100%. Your oven might just not have any leftover evidence in and of itself.
Now, per another question I read before, extreme integers were used in some systems to denote missing values, since text and NULL weren't options at those systems. Say I'm looking for ID#84, and I cannot find it in the table:
Not Found Is Unlikely:
Perhaps in some systems it's far more likely that a record exists with a missing/incorrect ID, than to not be existing at all? Hence, when no match is found, designers preferred all records without valid IDs to be returned?
This however has a few problems. First, depending on the design, user might not recognize the results are a set of records with missing IDs, especially if only one was returned. Second, current query poses a problem as it will always return the missing ID records in addition to the normal matches. Perhaps they relied on ORDERing to ease readability?
Exception Above SQL:
AFAIK, SQL is fine with a zero-row result, but maybe whatever thing that calls/used to call it wasn't as robust, and something goes wrong (hard exception, soft UI bug, etc.) when zero rows are returned? Perhaps then, this ID represented a dummy row (e.g. blanks and zeroes) to keep things running.
Then again, this also suffers from the same arguments above regarding "record is always outputted" and ORDER, with the added possibility that the SQL-caller might have dedicated logic to when the -999 record is the only record returned, which I doubt was the most practical approach even in whatever era this was done at.
... the more I type, the more I think this is the oven, and only the great grandmother can explain this to us.
If you want to avoid exception when no value transferred from user, in your stored procedure declare parameter as null. Like #id int = null
for instance :
CREATE PROCEDURE [dbo].[TableCheck]
#id int = null
AS
BEGIN
Select * from table where (id=#id)
END
Now you can execute it in either ways :
exec [dbo].[TableCheck] 2 or exec [dbo].[TableCheck]
Remember, it's a separate thing if you want to return whole table when your input parameter is null.
To answer your id = -999 condition, I tried it your way. It doesn't prevent any exception
I need to replace all occurrences "google.com" that are met in the SQL db table Column1 with "newurl". It can be a full cell value, a part of it (substring of varchar()), can be met even several times in a cell.
Based on SO answer search-and-replace-part-of-string-in-database
this is what I need:
UPDATE
MyTable
SET
Column1 = Replace(Column, 'google.com', 'newurl')
WHERE
xxx
However, in that answer it is mentioned that
You will want to be extremely careful when doing this! I highly recommend doing a backup first.
What are the pitfalls of doing this query? Looks like it does the same what any texteditor would do by clicking on Replace All button. I don't think it is possible in my case to check the errors even with reserve copy as I would like to know possible errors in advance.
Any reasons to be careful with this query?
Again, I expect it replaces all occurences of google.com with 'newurl' in the Column1 of MyTable table in the SQL db.
Thank you.
Just create a test table, as a replica of your original source table, complete the update on there and check results.
You would want to do this as good SQL programming practice to ensure you don't mess up columns that should not be updated.
Another thing you can do is get a count of the records before hand that fit the criteria using a SELECT statement.
Run your update statement and if it's a 1-1 match on count, you should be good to go.
The only thing i can think of that would happen negatively in this respect is that additional columns get updated. Your WHERE clause is not specific for us to see, so there's no way to validate that what you're doing will do what you expect it to.
I think the person posting the answer is just being cautious - This will modify the value in Column1 for every row in MyTable, so make sure you mean it when you execute. Another way to be cautious would be to wrap it in a transaction so you could roll it back if you don't like the results.
On a simple INSERT command, I am getting an error:
Error converting data type...
The source data has multiple sources and combined makes hundreds of thousands of rows.
Can I re-write my statement to catch the error and show the offending data?
Thanks!
EDIT:
Requests for code:
insert Table_A
([ID]
,[rowVersion]
,[PluginId]
,[rawdataId]
...
...
...
)
select [ID]
,[rowVersion]
,[PluginId]
,[rawdataId]
...
...
...
FROM TABLE_B
Here are two approaches that I've taken, when dealing with this problem. The issue is caused by an implicit conversion from a string to a date.
If you happen to know which field is being converted (which may be true in your example, but not always in mine), then just do:
select *
from table_B
where isdate(col) = 0 and col is not null
This may not be perfect for all data types, but it has worked well for me in practice.
Sometimes, when I want to find the offending row in a select statement, I would run the select, outputting the data into text rather than a grid. This is one of the options in SSMS, along the row of icons beneath the menus. It will output all the rows before the error, which sort of lets you identify the row with the error. This works best when there is an order by clause, but for debugging purpose it has worked for me.
In your case, I might create a temporary table that holds strings, and then do the analysis on this temporary table, particularly if Table_B is not really a table but a more complicated query.
The query statement insert into...select or select ... into ... from has no capability to find the offending data. Instead you can use BCP to set the max_erros and err_files to output all the offending data into an error file. Then you can simply analyze the error file to find all offending rows. [MSDN BCP]1
One solution is to do a binary search to find the problematic value(s). You can do that both by column and by row:
Try to insert only half the columns, if that works the other half of the columns.
Try to insert only half the number of rows. If that works the other half.
And repeat until you found the problem.
Basically, I am the new IT type guy, old guy left a right mess for me! We have a MS-Access DB which stores the answers to an online questionnaire, this particular DB has about 45,000 records and each questionnaire has 220 questions. The old guy, in his wisdom decided to store the answers to the questionnaire questions as text even though the answers are 0-5 integers!
Anyway, we now need to add a load of new questions to the questionnaire taking it upto 240 questions. The 255 field limit for access and the 30ish columns of biographical data also stored in this database means that i need to split the DB.
So, I have managed to get all the bioinfo quite happily into a new table with:
SELECT id,[all bio column names] INTO resultsBioData FROM results;
this didn't cause to much of a problem as i am not casting anything, but for the question data i want to convert it all to integers, at the moment I have:
SELECT id,CInt(q1) AS nq1.......CInt(q220) AS nq220 INTO resultsItemData FROM results;
This seems to work fine for about 400 records but then just stops, I thought it may be because it hit something it cant convert to a integer to start with, so i wrote a little java program that deleted any record where any of ther 220 answers wasnt 0,1,2,3,4 or 5 and it still gives up around 400 (never the same record though!)
Anyone got any ideas? I am doing this on my test system at the moment and would really like something robust before i do it to our live system!
Sorry for the long winded question, but its doing my head in!
I'm unsure whether you're talking about doing the data transformation in Access or SQL Server. Either way, since you're redesigning the schema, now is the time to consider whether you really want resultsItemData table to include 200+ fields, from nq1 through nq220 (or ultimately nq240). And any future question additions would require changing the table structure again.
The rule of thumb is "columns are expensive; rows are cheap". That applies whether the table is in Access or SQL Server.
Consider one row per id/question combination.
id q_number answer
1 nq1 3
1 nq2 1
I don't understand why your current approach crashes at 400 rows. I wouldn't even worry about that, though, until you are sure you have the optimal table design.
Edit: Since you're stuck with the approach you described, I wonder if it might work with an "append" query instead of a "make table" query. Create resultsItemData table structure and append to it with a query which transforms the qx values to numeric.
INSERT INTO resultsItemData (id, nq1, nq2, ... nq220)
SELECT id, CInt(q1), CInt(q2), ... CInt(q220) FROM results;
Try this solution:
select * into #tmp from bad_table
truncate table bad_table
alter bad_table alter column silly_column int
insert bad_table
select cast(silly_column as int), other_columns
from #tmp
drop table #tmp
Reference: Change type of a column with numbers from varchar to int
Just wrote a small java program in the end that created the new table and went through each record individually casting the fields to integers, takes about an hour and a half to do the whole thing though so i am still after a better solution when i come to do this with the live system.