I am using go-sql-driver/mysql to insert a ton of movies from the OMDB data dump. I am parsing the data and inserting it with the following code:
_, err = database.Query("INSERT INTO `movies` (`name`, `year`, `release_date`, `full_plot`, `genre`, `imdb_id`) VALUES (?, ?, ?, ?, ?, ?)", movie.Name, movie.Year, movie.ReleaseDate, movie.FullPlot, movie.Genre, movie.IMDBID)
if err != nil {
return false, nil
}
return true, nil
It works, but for only 150 rows. Am I doing something wrong?
Your code seems to discard the error value returned, which you shouldn't do; handle it gracefully. See what the error is, if you're opening too many connections to the DB, you should use a database connection pool and set the (*DB) SetMaxOpenConns value.
(*DB)Query is typically used for SELECT statements that return rows, use (*DB)Exec or (*Stmt)Exec for your INSERT.
I'd advise using a db connection pool, and (*sql.DB)Prepare to prepare a statement and run your inserts (even concurrently) using the prepared statement.
See https://golang.org/pkg/database/sql/#DB.Prepare
Related
I've been trying to insert data from excel into a table I've created using sqlite3 in python. However with the current code it will only return 'none' for each value instead of the actual values from the excel spreadsheet.
row_count = sheet.max_row - 1
count1 = 2
while count1 < row_count:
cursor.execute("INSERT INTO violations (points, serial_number, violation_code, violation_description, violation_status) VALUES (?, ?, ?, ?, ?)", (sheet.cell(row=(count1), column='A').value, sheet.cell(row=(count1), column='B').value, sheet.cell(row=(count1), column='C').value, sheet.cell(row=(count1), column='D').value, sheet.cell(row=(count1), column='E').value))
count1 = count1 + 1
So far i think the issue comes from the way the values are connected into the sql code with all the question marks as place holders for the other values, but I'm not sure how else to do this successfully. All help is greatly appreciated thank you :)
I have successfully connected to my database, but when I select 2 cells from my SQL db to set the value for both of my variables it does nothing. I have no problem inserting data into the database. I have tried different approaches to this with no success. Could it be that I am trying to use a string format for doubles??
double userHeight;
double userWeight;
QSqlQuery query;
QString retreiveUserHeight =
QString("SELECT Height, Weight FROM SQL1 WHERE Username='joejoe'");
query.prepare(retreiveUserHeight);
query.bindValue(0,"Height");
query.bindValue(1, "Weight");
query.exec();
userHeight = query.value(0).toInt();
userWeight = query.value(1).toInt();
I'm pretty certain there is a small error in syntax that is causing this mishap but I have been unable to find it. Thanks for your help.
qDebug() << "calculated" << 703*(userWeight/(userHeight*userHeight))
<< userWeight << userHeight ;
Heres the debug output:
calculated nan 0 0
// Obtain username from somewhere
QString username = "joejoe";
// Check whether DB is open
if( db->isOpen( ) )
{
QSqlQuery query;
double userHeight;
double userWeight;
// Prepare select statement
query.prepare ( "SELECT Height , Weight FROM SQL1 WHERE Username = :username" );
query.bindValue ( ":username" , username );
query.exec ( );
// Check if something went wrong when executing your query
if( query.lastError( ).text( ).trimmed( ) == "" )
{
// Loop through all results and handle them accordingly
while( query.next( ) )
{
userHeight = query.value( 0 ).toDouble( );
userWeight = query.value( 1 ).toDouble( );
qDebug( ) << "calculated" << 703 * ( userWeight / ( userHeight * userHeight ) ) << userWeight << userHeight;
qDebug( ) << "---------------------------------";
}
}
else
{
// Display the error that occured
QMessageBox::critical( this , tr( "SQL Error" ) , query.lastError( ).text( ) );
}
}
I assume this is what you wanted it to look like.
I've included some error checking and corrected your query to use .bindValue( ) correctly, since it's not meant for using for return values rather than for input as seen in the WHERE.
Since I don't know anything about your sql table I've included a loop to go through all results of your query. That can obviously be changed.
Apart from that if you're using doubles you should cast the result .toDouble( ) rather than .toInt( )
There are a number of serious problems with this code. Let's start with the concept of prepared SQL queries. Wikipedia lists two reasons for using prepared statements:
The overhead of compiling and optimizing the statement is incurred
only once, although the statement is executed multiple times. [...]
Prepared statements are resilient against SQL injection, because
parameter values, which are transmitted later using a different
protocol, need not be correctly escaped.
Neither of these reasons apply in your code; you're only executing the query once, and you're not splicing any inputs into the string. In fact, the only input in your query at all is the username, which is hard-coded to "joejoe":
"SELECT Height, Weight FROM SQL1 WHERE Username='joejoe'"
Since there are no variable inputs, using a prepared query doesn't make much sense. Neither do the following lines:
query.bindValue(0,"Height");
query.bindValue(1, "Weight");
Height and Weight are outputs from this query, not inputs. See the section in the Qt docs for QSqlQuery titled "Approaches to Binding Values" for an explanation of how this is intended to work. Qt's API for binding prepared SQL queries is fairly typical among database libraries, there's nothing earth shattering here.
Then we get to this:
userHeight = query.value(0).toInt();
userWeight = query.value(1).toInt();
Both the variables you're reading into here were declared as doubles, but you're calling toInt() on the returned QVariant rather than toDouble(). I don't know what (if any!) values are in your database, but it's possible they're getting rounded down to zero during the conversion from double to int if the values are between -1.0 and 1.0.
That said, you aren't doing any error checking whatsoever. The methods prepare() and exec() return bools that indicate whether they succeeded or failed. Likewise, both toInt() and toDouble() tell you whether they've succeeded or failed if you pass in a pointer to a bool. It's worth noting that both methods also return a zero value on failure.
I just started of with databases with SQLite3 and Ruby. I have run into a problem with my ruby code here.
I want to create a code where the user adds another record to the database. Now here is my problem.
User sawa found the solution of my first problem. Thank you!
NEW PROBLEM*
puts "Enter name for the new user"
x = gets.chomp
puts "Enter the type of the user"
y = gets.chomp
$db.execute('insert into customers(id,name,type) values (11,"#{x}","#{y}")')
When I run this code and input the x any y value it will return in my database #{x} and #{y} and not the values I created.
You're actually going about this all wrong. You shouldn't be using string interpolation for SQL at all, you should be using placeholders and bound variables. The README even includes an example:
# Execute a few inserts
{
"one" => 1,
"two" => 2,
}.each do |pair|
db.execute "insert into numbers values ( ?, ? )", pair
end
and the fine manual even mentions bound variables in the second sentence:
- (Object) execute(sql, bind_vars = [], *args, &block)
Executes the given SQL statement. If additional parameters are given, they are treated as bind variables, and are bound to the placeholders in the query.
So you should be saying this:
$db.execute('insert into customers (id, name, type) values (?, ?, ?)', 11, x, y)
or this:
$db.execute('insert into customers (id, name, type) values (?, ?, ?)', [11, x, y])
As is warned, don't use single quotes. Use double quotes. Expressions in single quotes are not ignored. They are evaluated literally.
I'm switching my code to PDO for increased security. My insert works until I add a special column that create spatial data. See below for the standard insert that works, and 2nd below for what is not working.
$sql = "INSERT INTO sites_tbl (sitename, the_geom) VALUES ('$_POST[sitename]', st_geomfromtext('POINT($geomstring)',27700))";
The geomstring = a number formatted 000000 000000
Using PDO the same insert looks something like (below) this works if I just want to insert the sitename, but not when I do the_geom. The value 325123 215432 will eventually be a variable, but for now I'm testing list this.
$stmt5 = $conn ->prepare(
"INSERT INTO sites_tbl (sitename, river_id, group_id, accepted_site, the_geom, bmwp_threshold) VALUES (?, ?, ?, ?, ?, ?)");
$stmt5->bindParam(1, $sitename);
$stmt5->bindParam(2, $river_id);
$stmt5->bindParam(3, $group_id);
$stmt5->bindParam(4, $accepted_site);
$stmt5->bindParam(5, $geomstring3);
$stmt5->bindParam(6, $bmwp_threshold);
$geomstring2 = "'POINT(635230 352120)'";
$geomstring3 = st_geomfromtext($geomstring2, 27700);
you cannot
bind
an arbitrary
SQL part
using
prepared
statement
but string
or numeric
literal
only.
$geomstring4 = "'POINT(325123 215432)'";
$stmt5 = $conn ->prepare(
"INSERT INTO sites_tbl (sitename, the_geom) VALUES (?, st_geomfromtext(?,27700)))");
$stmt5->bindParam(1, $sitename);
$stmt5->bindParam(2, $geomstring4);
Working with H2 I get this error when I try to write a row with the first element being
ABC and the second being [C#26afa68a
Syntax error in SQL statement "INSERT INTO USER VALUES(ABC,[[*]C#F4D5BC9) " expected "), DEFAULT, NOT, EXISTS, SELECT, FROM"; SQL statement:INSERT INTO user VALUES(abc,[C#f4d5bc9) [42001-167]
I don't know if there is a way to get H2 to accept Special Characters, but it would be great to know how to deal with this.
Thanks!
You should use a PreparedStatement:
PreparedStatement prep = conn.prepareStatement("INSERT INTO USER VALUES(?, ?)");
prep.setString(1, "ABC");
prep.setString(2, "[C#f4d5bc9");
prep.executeUpdate();
prep.close();
Using a PreparedStatement is the preferred solution, because that way you don't have to escape the data. If ABC and / and [C#f4d5bc9 are constants, you could use:
Statement stat = conn.createStatement();
stat.executeUpdate("INSERT INTO USER VALUES('ABC', '[C#f4d5bc9');
stat.close();