I have a table with a field name "BID" with its data type set as NUMERIC(20,0). Now i know that it will never be a decimal/float value but always be a int/long i.e a natural number.
Is there a way for the code generator to make the variable inside the generated model class a Long as opposed to java.math.BigDecimal which it makes currently. Can it be set in jooq.properties??
This "feature" is currently only available for the Oracle database, where NUMBER is the only numeric type available. Using Oracle with jOOQ, NUMBER(20, 0) would map to java.math.BigInteger instead of java.math.BigDecimal.
This behaviour will also be available for other RDBMS in the next release 1.6.3. See
https://sourceforge.net/apps/trac/jooq/ticket/639
Besides that, overriding of data type mappings in the code generator is currently not possible, but you could use BIGINT as suggested by gbn, if you don't really need 20 decimal digits. BIGINT can hold most 19-digit decimals...
Related
Let's I have a large data set with a "CustomerId" column. This data comes from a some external source and is used in some transformations and exposed to other systems.
CustomerId is in practice always an integer but is received as a string type so in theory it can contain non-integer characters. I don't want to explicitly cast it to integer to guard against a potential case where it's a string as that would break the flow, but I also want to cast it to integer since that would considerably speed up its usage.
So what I was considering is creating a CustomerId column which is an integer cast from the original and a CustomerIdVarchar which is the same value but with a varchar type. This CustomerIdVarchar will be null unless the original can't be cast to integer. That way for all intents and purposes the former can be used in an efficient manner, but it allows for guarding against the potential outlier and avoid breaking the system.
Is this a good idea?
You are saying that CustomerId is in practice always an integer, so if it comes as something that cannot be convert to INT, then there must be something wrong.
Why you want to risk to insert and have wrong data, that can potentially cause errors?
The better will be to explicitly cast the value to INT on INSERT and if there is error to check what's the issue and resolve it.
Can't you write a conditional statement?
You can use ISNUMERIC() function to determine whether CustomerID is valid. It would return '1' if the given value is numeric, and 0 if not.
EX:
DECLARE #ISNUMERIC Integer
SET #ISNUMERIC = (SELECT ISNUMERIC(CUSTOMERID))
Then you can use the variable to determine whether if you want to convert it to an int or not.
Is it possible to filter SQLite column values in SQL based on whether the value is numeric or textual? I have seen references to using CAST for this purpose. However, it appears to be useless as SELECT CAST('1a' AS NUMERIC) passes the check for a numeric type.
The typeof() SQL function is designated for type checking. However, its result depends on both column type definition (according to the official docs) and the format used during insertion. For example, when a number is inserted as a text literal into a NUMERIC column, it is converted into a number if possible, and typeof() will return an appropriate numeric type or text, if conversion did not occur. The TEXT column, on the other hand, stores all numeric literals as text. BLOB column stores textual and numeric literals without interpretation. Therefore, a mixed-type column should be probably declared as BLOB or NUMERIC (depending on whether textual literals needs to be converted to numbers, if possible). With this behavior in mind, typeof() is well suitable for type checking.
Thats just an idea:
SELECT [FilterColumn] FROM [Table] WHERE [FilterColumn]='0' OR (ceiling(log([FilterColumn],10)) =LENGTH([FilterColumn]) AND CAST([FilterColumn] AS INTEGER)>0)
This works for integer numbers where number of digits=log([FilterColumn],10). To distinguish a single letter from casting to 0, [FilterColumn]='0' OR [FilterColumn]>0 included.
I suppose there are more elegant solutions
I'm using vapor/postgres-kit (and by extension vapor/sql-kit) to insert some Codable conformant Swift objects into a PostgreSQL table. Some of the Swift object properties are of type Decimal, which are being used to represent monetary values accurately where Float or Double do not otherwise work reliably. The Postgres columns that store these Swift Decimal types are of the Postgres type numeric, which is a great match for accuracy/precision.
The default Encodable conformance of Decimal results in a Swift String, which then arrives in the SQL statement as a quoted value, which causes Postgres to throw an error because it's not expecting a string value for a numeric column type.
As far as I can see from looking around inside the SQLQueryEncoder class, there's nowhere to hook in to instead produce a value of something like SQLLiteral.numeric(String), which is a type that already exists in sql-kit.
My question is, can this be achieved somehow or do I need to abandon generating Inserts/Updates using Codable conformance in favour of manually forming queries in order to customise how the Decimal types are handled?
Thanks in advance for any advice!
I have a simple select statement like this:
SELECT [dok__Dokument].[dok_Id],
[dok__Dokument].[dok_WartUsNetto],
[dok__Dokument].[dok_WartUsBrutto],
[dok__Dokument].[dok_WartTwNetto],
[dok__Dokument].[dok_WartTwBrutto],
[dok__Dokument].[dok_WartNetto],
[dok__Dokument].[dok_WartVat],
[dok__Dokument].[dok_WartBrutto],
[dok__Dokument].[dok_KwWartosc]
FROM [dok__Dokument]
WHERE [dok_NrPelnyOryg] = 2753
AND [dok_PlatnikId] = 174
AND [dok_OdbiorcaId] = 174
AND [dok_PlatnikAdreshId] = 625
AND [dok_OdbiorcaAdreshId] = 624
Column dok_NrPelnyOryg is of type varchar(30), and not null.
The table contained both integer and string values in this column and this select statement was fired millions of times.
However recently this started crashing with message:
Conversion failed when converting the varchar value 'garbi czerwiec B' to data type int.
Little explanation: the table contains multiple "document" records and the mentioned column contains document original number (which comes from multiple different sources).
I know I can fix this by adding '' around the the number, but I'm rather looking for an explanation why this used to work and while not changing anything now it crashes.
It's possible that a plan change (due to changed statistics, recompile etc) led to this data being evaluated earlier (full scan for example), or that this particular data was not in the table previously (maybe before this started happening, there wasn't bad data in there). If it is supposed to be a number, then make it a numeric column. If it needs to allow strings as well, then stop treating it like a number. If you properly parameterize your statements and always pass a varchar you shouldn't need to worry about whether the value is enclosed in single quotes.
All those equality comparison operations are subject to the Data Type Precedence rules of SQL Server:
When an operator combines two
expressions of different data types,
the rules for data type precedence
specify that the data type with the
lower precedence is converted to the
data type with the higher precedence.
Since character types have lower precedence than int types, the query is basically the same as:
SELECT ...
FROM [dok__Dokument]
WHERE cast([dok_NrPelnyOryg] as int) = 2753
...
This has two effects:
it makes all indexes on columns involved in the WHERE clause useless
it can cause conversion errors.
You're not the first to have this problem, in fact several CSS cases I faced had me eventually write an article about this: On SQL Server boolean operator short-circuit.
The correct solution to your problem is that if the field value is numeric then the column type should be numeric. since you say that the data come from a 3rd party application you cannot change, the best solution is to abandon the vendor of this application and pick one that knows what is doing. Short of that, you need to search for character types on character columns:
SELECT ...
FROM [dok__Dokument]
WHERE [dok_NrPelnyOryg] = '2753'
...
In .Net managed ADO.Net parlance this means you use a SqlCommand like follows:
SqlCommand cmd = new SqlCommand (#" SELECT ...
FROM [dok__Dokument]
WHERE [dok_NrPelnyOryg] = #nrPelnyOryg
... ");
cmd.Parameters.Add("#nrPelnyOryg", SqlDbType.Varchar).Value = "2754";
...
Just make sure you don't fall into he easy trap of passing in a NVARCHAR parameter (Unicode) for comparing with a VARCHAR column, since the same data type precendence rules quoted before will coerce the comparison to occur on the NVARCHAR type, thus rendering indexes, again, useless. the easiest way to fall for this trap is to use the dredded AddWithValue and pass in a string value.
Your query stopped working because someone inserted the text string in to the field you are querying using INT. Up until that time it was possible to implicitly convert the data but now that's no longer the case.
I'd go check your data and, more importantly, the model; as Aaron said do you need to allow strings in that field? If not, change the data type to prevent this happening in the future.
A cohort of mine is building a somewhat long search query based on various input from the user. We've got NHibernate mapped up using Fluent NHibernate, and aside from some noob mistakes, all seems to be going well.
One issue we can't resolve in the case of this search is that for a particular parameter, NHibernate is creating sql that treats the input as int when we explicitly need it to be a string. We have a string property mapped to an nvarchar(255) column which mostly contains integer numbers, excluding some arbitrary inputs like "temporary" or long numbers like 4444444444444444 which is beyond the int limit.
In the course of testing, I've seen a couple things: 1) If I prepend a 0 to the incoming value, NH generates the sql param as a string, appropriately so; 2) If the value can realistically be converted to an int, the resulting sql treats it as so. In case #2, if I run the generated sql directly through sql server, I get an exception when the query comes across an non-integer value (such as the examples I listed above). For some reason, when I just let NH do it's thing, I'm getting appropriate records back, but it doesn't make sense; I would expect it to fail or at least tell me that something is wrong with some records that can't be evaluated by SqlServer.
The mapping is simple, the data store is simple; I would be ok leaving well enough alone if I at least understood why/how NHibernate is making this work when running the same state manually fails... Any thoughts?
Are you running the exact same code directly into SQL Server?
NHibernate parameterises all of its queries, and will in doing so define what value is passed through to SQL in the parameters. Which is probably what you're asking about, the reason SQL my fail, is that by default it will only know the difference if you input:
select * from table_name
where col_name = 5
in comparison with
select * from table_name
where col_name = '5'
If you do not define it as a string with the 's it will search for an int, and try to convert all the varchar's to ints, which will obviously fail in some cases with strings.