SQLite ORDER BY string containing number starting with 0 - sql

as the title states:
I have a select query, which I'm trying to "order by" a field which contains numbers, the thing is this numbers are really strings starting with 0s, so the "order by" is doing this...
...
10
11
12
01
02
03
...
Any thoughts?
EDIT: if I do this: "...ORDER BY (field+1)" I can workaround this, because somehow the string is internally being converted to integer. Is this the a way to "officially" convert it like C's atoi?

You can use CAST http://www.sqlite.org/lang_expr.html#castexpr to cast the expression to an Integer.
sqlite> CREATE TABLE T (value VARCHAR(2));
sqlite> INSERT INTO T (value) VALUES ('10');
sqlite> INSERT INTO T (value) VALUES ('11');
sqlite> INSERT INTO T (value) VALUES ('12');
sqlite> INSERT INTO T (value) VALUES ('01');
sqlite> INSERT INTO T (value) VALUES ('02');
sqlite> INSERT INTO T (value) VALUES ('03');
sqlite> SELECT * FROM T ORDER BY CAST(value AS INTEGER);
01
02
03
10
11
12
sqlite>
if I do this: "...ORDER BY (field+1)" I can workaround this, because somehow the string is internally being converted to integer. Is the a way to "officially" convert it like C's atoi?
Well thats interesting, though I dont know how many DBMS support such an operation so I don't recommend it just in case you ever need to use a different system that doesn't support it, not to mention you are adding an extra operation, which can affect performance, though you also do this ORDER BY (field + 0) Im going to investigate the performance
taken from the sqlite3 docs:
A CAST expression is used to convert the value of to a different storage class in a similar way to the conversion that takes place when a column affinity is applied to a value. Application of a CAST expression is different to application of a column affinity, as with a CAST expression the storage class conversion is forced even if it is lossy and irrreversible.
4.0 Operators
All mathematical operators (+, -, *, /, %, <<, >>, &, and |) cast both operands to the NUMERIC storage class prior to being carried out. The cast is carried through even if it is lossy and irreversible. A NULL operand on a mathematical operator yields a NULL result. An operand on a mathematical operator that does not look in any way numeric and is not NULL is converted to 0 or 0.0.
I was curios so I ran some benchmarks:
>>> setup = """
... import sqlite3
... import timeit
...
... conn = sqlite3.connect(':memory:')
... c = conn.cursor()
... c.execute('CREATE TABLE T (value int)')
... for index in range(4000000, 0, -1):
... _ = c.execute('INSERT INTO T (value) VALUES (%i)' % index)
... conn.commit()
... """
>>>
>>> cast_conv = "result = c.execute('SELECT * FROM T ORDER BY CAST(value AS INTEGER)')"
>>> cast_affinity = "result = c.execute('SELECT * FROM T ORDER BY (value + 0)')"
>>> timeit.Timer(cast_conv, setup).timeit(number = 1)
18.145697116851807
>>> timeit.Timer(cast_affinity, setup).timeit(number = 1)
18.259973049163818
>>>
As we can see its a bit slower though not by much, interesting.

You could use CAST:
ORDER BY CAST(columnname AS INTEGER)

In ListView with cursor loader!
String projection= some string column;
String selection= need to select;
String sort="CAST ("+ YOUR_COLUMN_NAME + " AS INTEGER)";
CursorLoader(getActivity(), Table.CONTENT_URI, projection, selection, selectionArgs, sort);

CONVERT CAST function using order by column value number format in SQL SERVER
SELECT * FROM Table_Name ORDER BY CAST(COLUMNNAME AS INT);

Thanks to Skinnynerd. with Kotlin, CAST worked as follows:
CAST fix the problems of prioritizing 9 over 10 OR 22 over 206.
define global variable to alter later on demand, and then plug it in the query:
var SortOrder:String?=null
to alter the order use:
For descendant:
SortOrder = "CAST(MyNumber AS INTEGER)" + " DESC"
(from highest to lowest)
For ascending:
SortOrder = "CAST(MyNumber AS INTEGER)" + " ASC"
(from lowest to highest)

Related

Cast a hexadecimal string to an array of bigint in hive

I have a column that contains a length 16 hexademical string. I would like to convert it to a bigint. Is there any way to accomplish that? The usual approach returns null since the input string could represent a number > 2^63-1.
select
cast(conv(hash_col, 16, 10) as bigint) as p0,
conv(hash_col, 16, 10) as c0
from mytable limit 10
I have also tried using unhex(..),
cast(unhex(hash_col) as bigint) as p0 from mytable limit 10
but got the following error
No matching method for class org.apache.hadoop.hive.ql.udf.UDFToLong
with (binary). Possible choices: FUNC(bigint) FUNC(boolean)
FUNC(decimal(38,18)) FUNC(double) FUNC(float) FUNC(int) FUNC(smallint) FUNC(string) FUNC(timestamp) FUNC(tinyint) FUNC(void)
If I don't do the cast(.. as bigint) part, I get some undisplayable binary value for p0. It seems unhex is not exactly the inverse of hex in hive.
Your values are out of range for BigInt
Ref : https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types
Max range for BigInt is 9,223,372,036,854,775,807
Use decimal(20,0) instead.
select cast(conv('85A58F8B014692CA',16,10) as decimal(20,0))

Treat TO_NUMBER() invalid format errors as NULL

I have a string column which usually contains integers in two formats... zero-padded, and not:
5
05
I want to sort based on these values numerically. To do that I do something like:
SELECT * FROM things ORDER BY TO_NUMBER(num, '0000');
This works fine, but sometimes there is invalid data, like abc, or !## in this column. Postgres becomes unhappy with me:
ERROR: invalid input syntax for type numeric: " "
What I'd like to do is treat invalid values/failures of TO_NUMBER() as NULL so that they are sorted accordingly. Is this possible? Or, some other alternative?
If you are using PostgreSQL, you can use this query:
SELECT * FROM things ORDER BY
TO_NUMBER((case when num ~ '^[0-9\.]+$' THEN num else '0' end),'0000');

How to check if a value is a number in SQLite

I have a column that contains numbers and other string values (like "?", "???", etc.)
Is it possible to add an "is number" condition to the where clause in SQLite? Something like:
select * from mytable where isnumber(mycolumn)
From the documentation,
The typeof(X) function returns a string that indicates the datatype of the expression X: "null", "integer", "real", "text", or "blob".
You can use where typeof(mycolumn) = "integer"
You could try something like this also:
select * from mytable where printf("%d", field1) = field1;
In case your column is text and contains numeric and string, this might be somewhat helpful in extracting integer data.
Example:
CREATE TABLE mytable (field1 text);
insert into mytable values (1);
insert into mytable values ('a');
select * from mytable where printf("%d", field1) = field1;
field1
----------
1
SELECT *
FROM mytable
WHERE columnNumeric GLOB '*[0-9]*'
select * from mytable where abs(mycolumn) <> 0.0 or mycolumn = '0'
http://sqlfiddle.com/#!5/f1081/2
Based on this answer
To test whether the column contains exclusively an integer with no other alphanumeric characters, use:
NOT myColumn GLOB '*[^0-9]*' AND myColumn LIKE '_%'
I.e., we test whether the column contains anything else than a digit and invert the result. Additionally we test whether it contains at least one character.
Note that GLOB '*[0-9]*' will find digits nested between other characters as well. The function typeof() will return 'text' for a column typed as TEXT, even if the text represents a number. As #rayzinnz mentioned, the abs() function is not reliable as well.
As SQLite and MySQL follow the same syntax and loose datatypes.
The query below is also possible
SELECT
<data>
, (
LENGTH(CAST(<data> AS UNSIGNED))
)
=
CASE WHEN CAST(<data> AS UNSIGNED) = 0
THEN CAST(<data> AS UNSIGNED)
ELSE (LENGTH(<data>)
) END AS is_int;
Note the <data> is BNF you would have the replace those values.
This answer is based on mine other answer
Running SQLite demo
For integer strings, test whether the roundtrip CAST matches the original string:
SELECT * FROM mytable WHERE cast(cast(mycolumn AS INTEGER) AS TEXT) = mycolumn
For consistently-formatted real strings (for example, currency):
SELECT * FROM mytable WHERE printf("%.2f", cast(mycolumn AS REAL)) = mycolumn
Input values:
Can't have leading zeroes
Must format negatives as -number rather than (number).
You can use the result of the function CAST( field as INTEGER) for numbers greater than zero and the simple condition like '0' per numbers equal to zero
SELECT *
FROM tableName
WHERE CAST(fieldName AS INTEGER) > 0
UNION
SELECT *
FROM tableName
WHERE fieldName like '0';
This answer is comprehensive and eliminates the shortcomings of all other answers. The only caveat is that it isn't sql standard... but neither is SQLite. If you manage to break this code please comment below, and I will patch it.
Figured this out accidentally. You can check for equality with the CAST value.
CASE {TEXT_field}
WHEN CAST({TEXT_field} AS INTEGER) THEN 'Integer' -- 'Number'
WHEN CAST({TEXT_field} AS REAL) THEN 'Real' -- 'Number'
ELSE 'Character'
END
OR
CASE
WHEN {TEXT_field} = CAST({TEXT_field} AS INTEGER) THEN 'Integer' --'Number'
WHEN {TEXT_field} = CAST({TEXT_field} AS Real) THEN 'Real' --'Number'
ELSE 'Character'
END
(It's the same thing just different syntax.)
Note the order of execution. REAL must come after INTEGER.
Perhaps their is some implicit casting of values prior to checking for equality so that the right-side is re-CAST to TEXT before comparison to left-side.
Updated for comment: #SimonWillison
I have added a check for 'Real' values
'1 frog' evaluated to 'Character' for me; which is correct
'0' evaluated to 'Integer' for me; which is correct
I am using SQLite version 3.31.1 with python sqlite3 version 2.6.0. The python element should not affect how a query executes.

I'm confused about Sqlite comparisons on a text column

I've got an Sqlite database where one of the columns is defined as "TEXT NOT NULL". Some of the values are strings and some can be cast to a DOUBLE and some can be case to INTEGER. Once I've narrowed it down to DOUBLE values, I want to do a query that gets a range of data. Suppose my column is named "Value". Can I do this?
SELECT * FROM Tbl WHERE ... AND Value >= 23 AND Value < 42
Is that going to do some kind of ASCII comparison or a numeric comparison? INTEGER or REAL? Does the BETWEEN operator work the same way?
And what happens if I do this?
SELECT MAX(Value) FROM Tbl WHERE ...
Will it do string or integer or floating-point comparisons?
It is all explained in the Datatypes In SQLite Version 3 article. For example, the answer to the first portion of questions is
An INTEGER or REAL value is less than any TEXT or BLOB value. When an INTEGER or REAL is compared to another INTEGER or REAL, a numerical comparison is performed.
This is why SELECT 9 < '1' and SELECT 9 < '11' both give 1 (true).
The expression "a BETWEEN b AND c" is treated as two separate binary comparisons "a >= b AND a <= c"
The most important point to know is that column type is merely an annotation; SQLite is dynamically typed so each value can have any type.
you cant convert text to integer or double so you wont be able to do what you want.
If the column were varchar you could have a chance by doing:
select *
from Tbl
WHERE ISNUMERIC(Value ) = 1 --condition to avoid a conversion from string to int for example
and cast(value as integer) > 1 --rest of your conditions

Preserving NULL values in a Double Variable

I'm working on a vb.net application which imports from an Excel spreadsheet.
If rdr.HasRows Then
Do While rdr.Read()
If rdr.GetValue(0).Equals(System.DBNull.Value) Then
Return Nothing
Else
Return rdr.GetValue(0)
End If
Loop
Else
I was using string value to store the double values and when preparing the database statement I'd use this code:
If (LastDayAverage = Nothing) Then
command.Parameters.AddWithValue("#WF_LAST_DAY_TAG", System.DBNull.Value)
Else
command.Parameters.AddWithValue("#WF_LAST_DAY_TAG", Convert.ToDecimal(LastDayAverage))
End If
I now have some data with quite a few decimal places and the data was put into the string variable in scientific notation, so this seems to be the wrong approach. It didn't seem right using the string variable to begin with.
If I use a double or decimal type variable, the blank excel values come across as 0.0.
How can I preserve the blank values?
Note: I have tried
Variable as Nullabe(Of Double)
But when passing the value to the SQL insert I get: "Nullable object must have a value."
Solution:
Fixed by changing the datatype of the parameter in the sub I was calling and then using Variable.HasValue to do the conditional DBNull insert.
I don't know which API you're using to do database inserts, but with many of them, including ADO.NET, the proper way to insert nulls is to use DBNull.Value. So my recommendation is that you use Nullable(Of Double) in your VB code, but then when it comes time to do the insert, you'd substitute any null values with DBNull.Value.
You need the question mark ? to let Double (or any value type) can store null (or Nothing). E.g.:
Dim num as Double? = Nothing
Note the ? mark.
To store in the db:
If num Is Nothing Then
... System.DBNull.Value ...
Else
... num ...
End If
or better:
If num.HasValue Then
... System.DBNull.Value ...
Else
... num.Value ...
End If
I am posting an article HERE while I keep looking for how to acheive a solution to your situation, but the article might have another solution which is to remove the null values, and add a default value. If I find anything else I will post it.
When you set up a database (at least
in MS SQL Server) you can flag a field
as allowing NULL values and which
default values to take. If you look
through people's DB structures, you'll
see that a lot of people allow NULL
values in their database. This is a
very bad idea. I would recommend never
allowing NULL values unless the field
can logically have a NULL value (and
even this I find this only really
happens in DATE/TIME fields).
NULL values cause several problems. For starters, NULL values
are not the same as data values. A
NULL value is basically an undefined
values. On the ColdFusion end, this is
not terrible as NULL values come
across as empty strings (for the most
part). But in SQL, NULL and empty
string are very different and act very
differently. Take the following data
table for example:
id name
---------------
1 Ben
2 Jim
3 Simon
4 <NULL>
5 <NULL>
6 Ye
7
8
9 Dave
10
This table has some empty strings (id: 7, 8, 10) and some NULL values
(id: 4, 5). To see how these behave
differently, look at the following
query where we are trying to find the
number of fields that do not have
values:
Launch code in new window » Download code as text file »
* SELECT
* (
* SELECT
* COUNT( * )
* FROM
* test t
* WHERE
* LEN( t.name ) = 0
* ) AS len_count,
* (
* SELECT
* COUNT( * )
* FROM
* test t
* WHERE
* t.name IS NULL
* ) AS null_count,
* (
* SELECT
* COUNT( * )
* FROM
* test t
* WHERE
* t.name NOT LIKE '_%'
* ) AS like_count,
* (
* SELECT
* COUNT( * )
* FROM
* test t
* WHERE
* t.name IS NULL
* OR
* t.name NOT LIKE '_%'
* ) AS combo_count
This returns the following record:
LEN Count: 3
NULL Count: 2
LIKE Count: 3
Combo Count: 5
We were looking for 5 as records 4, 5, 7, 8, and 10 do not have values
in them. However, you can see that
only one attempt returned 5. This is
because while a NULL value does NOT
have a length, it is not a data type
that makes sense with length. How can
nothing have or not have a length?
It's like asking "What does that math
equation smell like?" You can't make
comparisons like that.
So, allowing NULL values makes you work extra hard to get the kind of
data you are looking for. From a
related angle, allowing NULL values
reduces your convictions about the
data in your database. You can never
quite be sure if a value exists or
not. Does that make you feel safe and
comfortable when programming?
Furthermore, while running LEN() on a NULL value doesn't act as you
might think it to, it also does NOT
throw an error. This will make
debugging your code even harder if you
do not understand the difference
between NULL values and data values.
Bottom line: DO NOT ALLOW NULL VALUES unless absolutely necessary.
You will only be making things harder
for yourself.