Please imagine a table as such:
FirstName, LastName, Fee, Opt_Out
String String Int TinyInt(4)
And a struct as such:
type Dude struct {
Firstname string
Lastname string
Fee int
Opt_out int
}
To keep it short: I am using database/sql to scan into structs with no issue, except when it comes to that TinyInt.
err = rows.Scan(&firstname, &lastname, &fee, &opt_out)
After scanning, and before assigning values to my struct, I
fmt.PrintLn(Opt_out)
and they always return a zero value.
If I plug the query directly into the sql server, I get the correct ones and zero's I am expecting.
I have also tried changing the type of "Opt_out" in the struct to string, and attempted to cast via the query itself by doing a
IF(Opt_out=1,"yes","no")
and a similar thing happens, the query run in sql returns expected results, but golang returns empty values.
I am drawing a blank. Any ideas?
Jeez. Ok, this had nothing to do with the tinyint itself apparently.
The 'fee' column in the database had null values in it, the pre-existing query was set to replace null with empty strings.
IFNULL(fee,'')
It seems that the sql driver 'broke' very quietly on seeing those, and just gave up on processing everything afterwards.
My fix was ammending the query with IFNULL(fee,0) and everything popped back to life.
Thanks RayfenWindspear! I would not have given it the extra mile without your feedback.
Related
Have a problem using group_concat in a query made by my go application.
Any idea why a group_concat of FLOATS would look like a []uint8 on the Go side?
Cant seem to properly convert the suckers either.
It's definitely floats, I can see it in the raw query results, but when I do the same query in go and try to .Scan the result, Go complains that it's a []uint8 not a []float64 (which it actually is) Attempts to convert to floats gives me the wrong values (and way too many of them).
For example, at the database, I query and get 2 floats for the column in question, looks like this:
"5650.50, 5455.00"
On the go side however, go sees a []uint8 instead of []float64. Why does this happen? How does one workaround this to get the actual results?
My problem is that I have to use this SQL with the group_concat, due to the nature of the database I am working with, this is the best way to get the information, and more importantly the query itself works great, returns the data the function needs, but now I cant read it out because of type issues. No stranger to those, but Go isn't cooperating with me today.
I'd be more than pleased to learn WHY go is doing it this way, and delighted to learn of a way to deal with it.
Example:
SELECT ID, getDistance(33.1543,-110.4353, Loc.Lat, Loc.Lng) as distance,
GROUP_CONCAT(values) FROM stuff INNER JOIN device on device.ID = stuff.ID WHERE (someConditionsETC) GROUP BY ID ORDER BY ID
The actual result, when interfacing with the actual database (not within my application), is
"5650.00, 5850.50"
It's clearly 2 floats.
The same result produces a slice of uint8 when queried from Go and trying to .Scan the result in. If I range through and print those values, I get way more than 2, and they are uint8 (bytes) that look like this:
53,55,56,48,46,48,48
Not sure how Go expects me to handle this.
Solution.... stupid simple and not terribly obvious:
The solution:
crazyBytes := []uint8("5760.00,5750.50")
aString := string(crazyBytes)
strSlice := strings.Split(aString,",") // string representation of our array (of floats)
var floatz []float64
for _, x := range strSlice {
fmt.Printf("At last, Float: %s \r\n",x)
f,err := strconv.ParseFloat(x,64)
if err != nil { fmt.Printf("Error: %s",err) }
floatz = append(floatz, f)
fmt.Printf("as float: %s \r\n", strconv.FormatFloat(f,'f',-1,64))
}
Yea sure, it's obvious NOW.
GROUP_CONCAT returns a string. So in Go you get a byte array of characters, not a float. The result you posted 53,55,56,48,46,48,48 translates into a string "5780.00" which does look like one of your values. So you need to either fix your SQL to return floats or use strings and strconv modules in Go to parse and convert your string into floats. I think the former approach is better, but it is up to you.
I've found this strange behaviour and I searched but couldn't find anything about it.
I know that in my example I don't need to cast [affairenum] to STRING, but because of a specific syntax in Entity Framework this is how an Affairenum.Contains(), StartsWith() or EndsWith() ends up being generated.
Consider for example a table that contains an id (affaireid column) and numbers (affairenum column) with values from 1 to 5000000.
SELECT TOP (1000) [affaireid]
,[affairenum]
,STR(affairenum) AS string
FROM [dbo].[ULAffaire]
where STR(affairenum) LIKE N'%9'
Works and returns results. Same goes with N'%9%'.
SELECT TOP (1000) [affaireid]
,[affairenum]
,STR(affairenum) AS string
FROM [Ulysse].[dbo].[ULAffaire]
where STR(affairenum) LIKE N'9%'
Does not work and returns nothing. The difference here being LIKE N'9%', the equivalent of a StartsWith().
STR(affairenum) looks identical to affairenum, EndsWith() and Contains() both work normally, but this returns nothing.
I've tried with LOWER(), to no avail. Is the STR() method adding anything ? a space, some weird character ? Am I missing something silly ?
That is because str() left pads the results with spaces. The default is a length of 10 (see here).
I'm not a fan of using numbers as strings. But if you do so, explicit conversion should do what you want:
where cast(affairnum as varchar(255)) like '9%'
That is, str() is not a type conversion function. It is a string formatting function -- hence the presence of spaces where you might not expect them.
I should note that you don't even need to explicit convert the number to a string, so this works:
where affairnum like '9%'
However, I have such bad memories of hours and hours devoted to fixing problems in SQL code that used implicit conversion, so I cannot in good conscience propose implicit conversion to someone else.
Based on a very good approach for null fields proposed by the main contributor to flatbuffers:
https://github.com/google/flatbuffers/issues/333#issuecomment-155856289
The easiest way to get a null default for an integer field is to wrap
it in a struct. This will get you null if scalar isn't present. It
also doesn't take up any more space on the wire than a regular int.
struct myint { x:int; }
table mytable { scalar:myint; }enter code here
this will get you null if scalar isn't present. It also doesn't take
up any more space on the wire than a regular int.
Also based on the flatbuffers documentation:
https://google.github.io/flatbuffers/md__schemas.html
You can't change types of fields once they're used, with the exception of same-size data where a reinterpret_cast would give you a desirable result, e.g. you could change a uint to an int if no values in current data use the high bit yet.
My question is can I treat int as reinterpret_cast-able to myint?
In other words, if I start with just a simple int as a field, can I later on decide that I actually want this int to be nullable and change it to myint? I know that all values that used to be default value in the first int schema will be read as null in the myint schema and I am ok with that.
Of course the obvious follow up question is can I do the same thing for all scalar types?
Though that isn't explicitly documented, yes, int and myint are wire-format compatible (they are both stored inline). Like you say, you will lose any default value instances to become null.
I need to store an alphanumeric string in an integer column on one of my models.
I have tried:
#result.each do |i|
hex_id = []
i["id"].split(//).each{|c| hex_id.push(c.hex)}
hex_id = hex_id.join
...
Model.create(:origin_id => hex_id)
...
end
When I run this in the console using puts hex_id in place of the create line, it returns the correct values, however the above code results in the origin_id being set to "2147483647" for every instance. An example string input is "t6gnk3pp86gg4sboh5oin5vr40" so that doesn't make any sense to me.
Can anyone tell me what is going wrong here or suggest a better way to store a string like the aforementioned example as a unique integer?
Thanks.
Answering by request form OP
It seems that the hex_id.join operation does not concatenate strings in this case but instead sums or performs binary complement of the hex values. The issue could also be that hex_id is an array of hex-es rather than a string, or char array. Nevertheless, what seems to happen is reaching the maximum positive value for the integer type 2147483647. Still, I was unable to find any documented effects on array.join applied on a hex array, it appears it is not concatenation of the elements.
On the other hand, the desired result 060003008600401100500050040 is too large to be recorded as an integer either. A better approach would be to keep it as a string, or use different algorithm for producing a number form the original string. Perhaps aggregating the hex values by an arithmetic operation will do better than join ?
I'm using qt 4.8 and psql driver to connect to a postgres 9.1 database.
I'm doing a generic library to connect and insert into the database; almost all methods are ready but in the last one, I need to do a select from a table to insert values into another. When I try the select statement it behaves different. According to the code in turn but no one of the tests I made have resulted in a correct solution.
Here's my code:
struct enfriadores enf;
enf.horaE=time(&enf.horaE);
enf.horaS=time(&enf.horaS)+1900;
//base1.insertaEvento(db,enf);
QString consulta = "Select id_evento from eventos where extract(epoch from horae)=:hora_bus";
QDateTime hora_bus(QDateTime::fromTime_t(enf.horaE));
//qDebug()<< enf.horaE;
QSqlQuery query(db);
query.prepare(consulta);
query.bindValue(":hora_bus",hora_bus);
query.exec();
query.first();
while(query.next())
{
int valor = query.value(0).toInt();
qDebug() << valor << endl;
}
The base1.insertaEvento is a call from a class I did to insert data on the table where afterwards I'll need to extract the id. The
qDebug() << enf.horaE;
I put it to know if the time was in the right form before I attached it to the query, which by the way, was correct.
horaE is taken from a struct I have declaed in the previously mentioned class.
When I run the query as it is with the while(query.next()) it runs good but returns no results and if I delete the while loop but still maintain the query.next() compiler returns
QSqlQuery::value: not positioned on a valid record
0
I tried using the query.first() method and the query.setForwardOnly(true) but with same results.
Also if I try the value of hora_bus with qDebug() and replace it directly in the psql console I get a positive match so the problem is not in the way data is inserted or formatted, it's in the way the query is retrieved I believe but do not know how to resolve this
Any ideas?
Thanks
The SQL expression extract(epoch from horae) produces a number of seconds since 1/1/1970 so so that's what should be passed to the parameter :hora_bus.
The QDateTime::fromTime_t(enf.horaE) indicates that enf.horaE has this value, however instead of passing ot to the query, it's passing a QDateTime object whose text representation is probably going to be a string with year,month,etc... that can't be compared to a number of seconds.
So try this instead:
query.bindValue(":hora_bus",enf.horaE);
Also the code shouldn't ignore the boolean return values of prepare() and exec(). You don't want to try looping within results when the execution of the query has failed.
EDIT1:
indeed when passing a QDateTime set to today to a prepared query similar to yours, QSqlQuery::exec() returns false with an SQL error invalid input syntax for type double precision.
EDIT2: it appears QVariant doesn't support being initialized with a long so an explicit cast to a different supported type is necessary. I've chosen qlonglong for a safe larger value:
query.bindValue(":hora_bus",(qlonglong)enf.horaE);
Tested, it worked for me.
http://qt-project.org/doc/qt-4.8/qsqlquery.html#details
At the very end of the documentation it mentions the following:
Warning: You must load the SQL driver and open the connection before a QSqlQuery is created. Also, the connection must remain open while the query exists; otherwise, the behavior of QSqlQuery is undefined.
If the connection to the database is timing out, or one of the variables you are using goes out of scope, you might disconnect early and get undefined results.
You can also check the return values on most of your function calls to see if they were successful or not.
Hope that helps.