I'm having a problem trying to convert a MapInfo file from MID/MIF format to TAB format.
This problem occurs from version GDAL 3.0.4 and higher. On version 2.1.2, everything works without problems.
I use the following command
ogr2ogr -f "MapInfo file" "test.tab" "test.mif"
Error following
ERROR 1: Cannot format 1234.1 as a 20.16 field
ERROR 3: Failed writing attributes for feature id 1 in test.tab
ERROR 1: Unable to write feature 1 from layer test.
ERROR 1: Terminating translation prematurely after failed
translation of layer test (use -skipfailures to skip errors)
Here example of MapInfo file MID/MIF format
test.mif
test.mid
Can anyone explain what is the reason for this error?
Im trying to use GDAL version 3.5, but still getting this error.
If I change the column type to Float than everything works fine.
But I can't just change the format of the existing file
Your value "1234.1" is to big.
From the documentation:
Decimal fields store single and double precision floating point values.
Width is the total number of characters allocated to the field, including the decimal point.
Precision controls the precision of the data and is the number of digits to the right of the decimal.
Your decimal definition "Decimal (20,16)" leaves only 3 digits for the integer part. Try a lesser value, i.e: 999.4 or change the decimal format to Decimal (20,15)
Related
We are attempting to import a SAS7BDAT file into a SQL Server database.
The only issue we're running into is that source decimal values are being read as float.
Using a command line tool named dsread, we found an option that:
Converting the IEEE floating-point numeric values in the SAS7BDAT file to their decimal representation may cause a loss of precision. To get a lossless representation of the data, use the /l flag:
sashelp> dsread /v /l prdsale
ACTUAL,PREDICT,COUNTRY,...
0x0000000000e88c40,0x0000000000908a40,CANADA...
0x0000000000388f40,0x0000000000907240,CANADA...
0x0000000000008340,0x0000000000708a40,CANADA...
0x0000000000108440,0x0000000000a88040,CANADA...
0x0000000000808440,0x0000000000308440,CANADA...
0x0000000000a08d40,0x0000000000607e40,CANADA...
...etc...
The numerics are output as eight hexadecimal bytes (16 digits) giving the internal floating-point representation, which can then be used to reconstruct the exact same value in the receiving software. Use /L to get the bytes in big-endian order
Running some tests, we can see that the decimal value -1.457263 is being represented by the value 0xcbbbea01f350f7bf when we use that /L flag. What we haven't been able to figure out is, how can we convert that hexadecimal value into a SQL Server decimal value?
We've tried many variants, including:
select CONVERT(decimal, convert(varbinary,'0xcbbbea01f350f7bf',1))
but that results in:
Msg 8115, Level 16, State 6, Line 17
Arithmetic overflow error converting varbinary to data type numeric.
If 0xcbbbea01f350f7bf = -1.457263, then it looks like that's a IEEE double-precision floating point number with a big endian byte order.
So reverse the bytes (or get it to export in little endian instead)
0xcbbbea01f350f7bf -> 0xbff750f301eabbcb
Then convert it to a float. You can use CLR or there's a TSQL function you can try here:
Unpacking a binary string with TSQL
Then convert it to a decimal.
select convert(decimal(36,17), dbo.[fnBinaryFloat2Float]( 0xbff750f301eabbcb ), 3)
Which, you can see has preserved a closer approximation to the floating point value
-1.45726299999999998
Is there a way to reconstruct the "exact same value" with no differences?
Then leave the data as float(53) which is exactly the same data type from the source, and don't convert it to decimal at all. decimal and float each store finite subsets of the rational numbers, and many numbers can be exactly represented in either system. But some float values don't have an exact match in decimal, and vice versa.
I've got a program, which compute a several variables and then these variables are writing in to the output file.
Is it possilbe, that when my program can't get a correct results for my formula, it does'nt terminate?
To clarify what I do, here is part of my code, where the variable of my interest are compute:
dx=x(1,i)-x(nk,i)
dy=y(1,i)-y(nk,i)
dz=z(1,i)-z(nk,i)
call PBC(dx,dy,dz)
r2i=dx*dx+dy*dy+dz*dz
r2=r2+r2i
r2g0=0.0d0
r2gx=0.0d0
dx=x(1,i)-x(2,i)
call PBC(dx,dy,dz)
rspani=dsqrt(dx*dx)
do ii=1,nk-1
rx=x(ii,i)
ry=y(ii,i)
rz=z(ii,i)
do jj=ii+1,nk
dx=x(jj,i)-rx
dy=y(jj,i)-ry
dz=z(jj,i)-rz
call PBC(dx,dy,dz)
r21=dx*dx+dy*dy+dz*dz
r21x=dx*dx
r2g=r2g+r21
r2gx=r2gx+r21x
r2g0=r2g0+r21
rh=rh+1.0d0/dsqrt(r21)
rh1=rh1+1.0d0
ir21=dnint(dsqrt(r21)/dr)
p(ir21)=p(ir21)+2.0D0
dxs=dsqrt(r21x)
if(dxs.gt.rspani) rspani=dxs
end do
and then in to the output I just write these variables:
write(12,870)r2i,sqrt(r2i),r2g0,r2gx/(nk*nk)
870 FORMAT(3(f15.7,3x),f15.7)
The x, y, z are actully generate via a random number generator.
The problem is that my output contains, correct values for lets say 457 lines, and then a one line is just "*********" when I use mc viewer and then the output continues with correct values, but let's say 12 steps form do cycle which compute these variables is missing.
So my questions are basic:
Is it possible, that my program can't get a correct numbers, and that's why the result is not writing in to the program?
or
Could it this been caused due to wrong output formating or something related with formating?
Thank you for any suggestion
********* is almost certainly the result of trying to write a number too large for the field specified in a format string.
For example, a field specified as f15.7 will take 1 spot for the decimal point, 1 spot for a leading sign (- will always be printed if required, + may be printed if options are set), 7 for the fractional digits, leaving 6 digits for the whole part of the number. There may therefore be cases where the program won't fit the number into the field and will print 15 *s instead.
Programs compiled with an up to date Fortran compiler will write a string such as NaN or -Inf if they encounter a floating-point number which represents one of the IEEE special values
I have created a Fortran 90 code to filter and convert the text output of another program in a csv form. The file contains a table with columns of various types (character, real, integer). There is a column that generally contains decimal values (probability values). BUΤ, in some rows, where the value should be decimal "1.000", the value is actually integer "1".
I use "F5.3" specifier to read this column and I have the same format statement for every row of the table. So, when the code finds "1", it reads ".001", because it does not find a decimal point.
What ways could I use to correctly (and generally) read integers among other decimals?
Could I specify "unformatted" input only for a number of "spaces"?
The data edit descriptor fw.d for floating point format specification is for input normally used with zero d (it cannot be ommited). Nonzero d is used in the rare case when the floating point data is stored as scaled integers, or you do some unit conversion from the integer values.
You could try using list-directed input: use a * instead of a format specifier. This would be for the entire read, not selected items. Or you could read the lines into a string test their contents to decide how to read them. If the sub-string has a decimal point: read (string(M:N), '(F5.3)') value. If it doesn't, use a different format, e.g., perhaps read as as F5.0.
P.S. "unformatted" is reading binary data without conversion ... it is a direct copy of the data from the file to the data item. "listed-directed" is the Fortran term for reading & converting data without using a format specification.
well here's someting new to me: f90 allows a mix of comma and space delimiters for a simple list directed read:
read(unit,*)v1,v2,v3,v4
with input
1.222 2 , 3.14 , 4
yields
1.222000 2.000000 3.140000 4.000000
I have db2 import statement which reads from a file and writes to a database.
Column data type for column 18 (where i am getting error) is Decimal(18,2)
The value for that column coming in the file is -502.47
However, I am getting the below error:
SQL3123W The field value in row "1" and column "18" cannot be converted to a PACKED DECIMAL value. A null was loaded.
And the value is not going into database.
What is the reason for this error ? What is the solution ?
There was an issue with the number of column. I was passing more number of columns then the program expected. So we can get above error in that case as well.
It was because of the double quotes in the loaded CSV files at that particular cell mentioned in the error.
You should try opening the file in Notepad++ or any other text editor, remove the double quotes, save and load back into the DB.
Your error should be resolved.
I'm trying to process a fixed width input file in pentaho and validate the format. The file will be a mixture of strings, numbers and dates. However when attempting to process a number field that has an incorrect character present (which i had expected would throw an error) it just reads the first part of the number and ignores the bad char.
I can recreate this issue with a very simple input file containing a single field:
I specify the expected number format, along with start position and length:
On running the transformation i would have expected the 'Q' to cause an error instead the following result is displayed, just reading the first two digits "67" and padding the rest to match the specified format:
If the input file is formatted correctly it runs perfectly well, but need it to throw an error otherwise. Any suggestions would be awesome. Thanks!
Just an FYI in case someone stumbles accross this question after hitting the same issues as myself.
I was able to construct a workaround by reading all values in the "Text File Input" step as strings, and then using a "Data Validator" step equipped with regex evaluation to ensure numbers were correctly formatted before parsing to number type with a following "Select Values" step.
Takes a bit longer to do this for every field, but was the most robust solution i could come up with.
Thanks