Small number out of DOUBLE range - sql

I created a table on a mariaDB with the following definition. Note the longitude and latitude fields.
Create Table geo_data (
geo_data_id int NOT NULL AUTO_INCREMENT,
place_id int NOT NULL,
longitude DOUBLE(18,18) SIGNED,
latitude DOUBLE(18,18) SIGNED,
Primary Key (geo_data_id),
Foreign Key (place_id) References place (place_id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
When I try to insert data into the geo_data table using
Insert into geo_data (place_id, longitude, latitude) Values (1, 1.2, 3.4);
I receive the following error message:
Error: ER_WARN_DATA_OUT_OF_RANGE: Out of range value for column 'longitude' at row 1
I guess I am missing something here, since I don't believe 1.2 could in any way be out of range of a Double(18,18). So what on earth is going on here?

Your column is defined as DOUBLE(18,18). The first number is the scale (the total number of digits in the whole number, including decimals); the second is the precision (the number of decimal positions).
Giving the same value to both the scale and precision means that your value cannot be greater than 1 (all 18 digits are decimals).
You want to decrease the precision to something smaller in order to leave room for non-decimal digits, like: DOUBLE(18, 6), which gives you 12 non-decimal positions.

Related

How to create a column with the datatype float which only stores up to 3 decimals in PostgreSQL?

I want to create a table in my PostgreSQL:
CREATE TABLE my_table(
id INT GENERATED ALWAYS AS IDENTITY NOT NULL PRIMARY KEY,
description TEXT,
score FLOAT NOT NULL
);
How do I limit the number of decimals stored in the "score" column to a maximum of 3 decimals?
You would use numeric. However, you need a precision as well, which limits the maximum value:
CREATE TABLE my_table(
id INT GENERATED ALWAYS AS IDENTITY NOT NULL PRIMARY KEY,
description TEXT,
score NUMERIC(10, 3) NOT NULL
);
This will store numbers up to 9,999,999.999.

DECIMAL values being converted to INT + 1 on its own

I have a C# app where the user fills a form to store a product in a SQL Server database.
The problem is that every time a product is stored (through the user filling a form), the price (a decimal) is automatically converted to int and has 1 added to it.
I initially thought it was an issue with the app, however, the registration process is pretty simple and I didn't find any error there, so I inserted a row directly from SQL Server and the issue presented itself, so this tells me the issue is in SQL Server, not in the app.
Executing
insert into product (code, description, unit_price, stock, category_code)
values (7, 'Window Cleaner', 20.50, 20, 3)
Results into price being 21.
This is the table definition
CODE INT NOT NULL,
DESCRIPTION VARCHAR(30) NOT NULL,
UNIT_PRICE DECIMAL NOT NULL,
STOCK INT NOT NULL,
CATEGORY_CODE INT NOT NULL
CONSTRAINT PK_PRODUCT PRIMARY KEY(CODE),
CONSTRAINT FK_CATEGORY_CODE FOREIGN KEY (CATEGORY_CODE) REFERENCES Category(CODE),
CONSTRAINT PRODUCT_POSITIVE_VALUES CHECK(UNIT_PRICE > 0 AND CODE >= 0 AND STOCK >= 0)
You have used the following to define your column:
UNIT_PRICE DECIMAL NOT NULL
This has no precision nor scale and will therefore use the default precision (18) and scale (0). The default scale of 0 is effectively an int. So when you insert/update a value the value will get rounded to an int. To solve your problem define your column with the correct precision and scale e.g.
UNIT_PRICE DECIMAL(9,2) NOT NULL
Reference

Postgres automatically rounds the number on insert

I have a table that looks like the following:
CREATE TABLE word(
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
word TEXT NOT NULL,
count INT NOT NULL,
ratio NUMERIC(10, 3) NOT NULL,
percent_of_total NUMERIC(10, 3) NOT NULL,
daily_count_id UUID REFERENCES daily_count(id)
);
I then tries to insert with this statement:
INSERT INTO word (word, count, ratio, percent_of_total, daily_count_id)
VALUES ('test', 5, 5/214, 5*100/214,
(SELECT id from daily_count WHERE day_of_count = CURRENT_DATE+1));
It works. It inserts the value when selecting it from the table then the numeric values has been rounded like the following:
67035a35-e5df-495b-95d5-cb3b4041c7b4 test 5 0.000 2.000 91858e7a-3440-4959-9074-9d197d6c97fc
The values 2.000 and 0.000 are rounded but I need them to be the precise value.
I'm using the DataGrip IDE but I do not think it has anything to do with it.
5/214 will be executed as integer division. I.e. the result will be integer.
If you want floating point division, you can simply do 5.0/214.
(Or use cast, e.g cast(5 as NUMERIC(10, 3)) / 214.)

How do I set the precision and scale in SQL (access)

I am trying to create a table in Access.
I have the following code:
CREATE TABLE Class Enrollement (
OfferNo INTEGER PRIMARY KEY,
StdNo Text(9) NULL,
EnrGrade Decimal(2) Percision(8) scale(4) NULL
);
EnrGrade needs to be a decimal, Precision of 8, Scale of 4, and 2 decimal places.
The last line of code is not correct. How would I do this?
I believe you're looking for:
Where the first value is the precision (number of decimal digits, followed by scale, or numbers after the decimal)
CREATE TABLE ClassEnrollment (
OfferNo INTEGER PRIMARY KEY,
StdNo Text(9) NULL,
EnrGrade Decimal(8, 2) NULL
);
You must enable ANSI-92 Query Mode. After that in your query you can write:
CREATE TABLE Offering (
OfferNo INTEGER PRIMARY KEY,
StdNo Text(9) NULL,
EnrGrade Decimal(8,4) NULL
);
Precision of 8, Scale of 4, and 2 decimal places
These requirements appear contradictory. A decimal column with a precision of 8 and a scale of 4 can store up to 4 decimal places.
Perhaps the intention of the spec is to display at least two decimal places? e.g.
SQL DDL:
CREATE TABLE ClassEnrollement (
OfferNo INTEGER PRIMARY KEY,
StdNo NVARCHAR(9),
EnrGrade DECIMAL(8, 4)
);
SQL DML:
SELECT OfferNo, StdNo,
FORMAT$(ClassEnrollement, '0.00##')
AS ClassEnrollement__formatted
FROM ClassEnrollement;
Or perhaps the extra numeric scale is to enable custom rounding? The DECIMAL type exhibits rounding by trucation, a feature often missed because all other numeric types exhibit bankers' rounding. A rule of thumb is to store an extra place of numeric scale so that the inherent rounding, whatever it is, does not affect the raw value being stored, enabling custom rounding to be applied later. Two extra may simply be over-engineering ;)

MySQL float values jumping around on insert?

So i have a SQL table setup as such
CREATE TABLE IF NOT EXISTS `points` (
`id` int(11) NOT NULL auto_increment,
`lat` float(10,6) NOT NULL,
`lng` float(10,6) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM;
And im inserting stuff like
INSERT INTO `points` (`lat`, `lng`) VALUES ('89.123456','-12.123456');
Gives me a row with lat and lng being 89.123459 and -12.123455
Whats up?
In computers a "float" value is stored as a number times a number squared generally, and some numbers can't be stored exactly as entered.
If you need it exact you should store it as DECIMAL(8,6)
FLOAT data types have rounding effects, because not every fractional base-10 number can be represented in the base-2 format of IEEE 754. For this reason, FLOAT and DOUBLE PRECISION should be considered inexact numeric data types.
Read: "What Every Computer Scientist Should Know About Floating-Point Arithmetic" (http://www.validlab.com/goldberg/paper.pdf)
+1 to #MindStalker's answer -- use NUMERIC or DECIMAL if you need exact numeric data types.