I don't understand why my request returns me an empty array with the code below.
Using grails and an H2 database
Animal lion = new Animal()
lion.save()
println lion.id
println sql.rows("select * from animal")
The outputs are
1
[]
Why do I get an empty array ?
If I go and check in the memory database at
localhost/Zoo/dbconsole
I get the line as I should be having. Is there some kind of a time limit that I have to wait before doing my sql request ?
Is this in Grails? If so, try:
lion.save( flush: true )
It's probably that Hibernate hasn't flushed the changes to the database before you do your select (especially as it looks like the above code is all in the same transaction).
Related
I have piece of code written in spark that loads data from HDFS into java classes generated from avro idl. On RDD created in that way I am executing simple operation which results depends on fact whether I cache RDD before it or not
i.e if I run code below
val loadedData = loadFromHDFS[Data](path,...)
println(loadedData.map(x => x.getUserId + x.getDate).distinct().count()) // 200000
program will print 200000, on the other hand executing next code
val loadedData = loadFromHDFS[Data](path,...).cache()
println(loadedData.map(x => x.getUserId + x.getDate).distinct().count()) // 1
result in 1 printed to stdout.
When I inspect values of the fields after reading cached data it seems
I am pretty sure that root cause of described problem is issue with serialization of classes generated from avro idl, but I do not know how to resolve it. I tried to use Kryo, registering generated class (Data), registering different serializers from chill_avro for given class (SpecificRecordSerializer, SpecificRecordBinarySerializer, etc), but none of those ideas helps me.
How I can solve this problem?
Link to minimal, complete, and verifiable example.
Try the code below out -
val loadedData = loadFromHDFS[Data](path,...)
println(loadedData.map(x => x.getUserId + x.getDate).distinct().count()).cache()
I'm using mongodb to store some data. Then I have a function that gets the object with the latest timestamp and one with the oldest. I haven't experienced any issues during development or production with this method but when I try to implement a test for it the test fails approx 20% of the times. I'm using rspec to test this method and I'm not using mongoid or mongomapper. I create three objects with different timestamps but get a nil response since my dataset contains 0 objects. I have read a lot of articles about write_concern and that it might be the problem with "unsafe writes" but I have tried almost all the different combinations with these parameters (w, fsync, j, wtimeout) without any success. Does anyone have any idea how to solve this issue? Perhaps I have focused too much with the write_concern track and that the problems lies somewhere else.
This is the method that fetches the latest and oldest timestamp.
def first_and_last_timestamp(customer_id, system_id)
last = collection(customer_id).
find({sid:system_id}).
sort(["t",Mongo::DESCENDING]).
limit(1).next()
first = collection(customer_id).
find({sid:system_id}).
sort(["t",Mongo::ASCENDING]).
limit(1).next()
{ min: first["t"], max: last["t"] }
end
Im inserting data using this method where data is a json object.
def insert(customer_id, data)
collection(customer_id).insert(data)
end
I have reverted back to use the default for setting up my connection
Mongo::MongoClient.new(mongo_host, mongo_port)
I'm using the gem mongo (1.10.2). I'm not using any fancy setup for my mongo database. I've just installed mongo using brew on my mac and started it. The version of my mongo database is v2.6.1.
My Grails application is not using GORM but instead uses my own SQL and DML code to read and write the database (The database is a huge normalized legacy one and this was the only viable option).
So, I use the Groovy Sql Class to do the job. The database calls are done in Services that are called in my Controllers.
Furthermore, my datasource is declared via DBCP in Tomcat - so it is not declared in Datasource.groovy.
My problem is that I need to write some transaction code, that means to open a transaction and commit after a series of successful DML calls or rollback the whole thing back in case of an error.
I thought that it would be enough to use groovy.sql.Sql#commit() and groovy.sql.Sql#rollback() respectively.
But in these methods Javadocs, the Groovy Sql documentation clearly states
If this SQL object was created from a DataSource then this method does nothing.
So, I wonder: What is the suggested way to perform transactions in my context?
Even disabling autocommit in Datasource declaration seems to be irrelevant since those two methods "...do nothing"
The Groovy Sql class has withTransaction
http://docs.groovy-lang.org/latest/html/api/groovy/sql/Sql.html#withTransaction(groovy.lang.Closure)
public void withTransaction(Closure closure)
throws java.sql.SQLException
Performs the closure within a transaction using a cached connection. If the closure takes a single argument, it will be called with the connection, otherwise it will be called with no arguments.
Give it a try.
Thanks James. I also found the following solution, reading http://grails.org/doc/latest/guide/services.html:
I declared my service as transactional
static transactional = true
This way, if an Error occurs, the previously performed DMLs will be rolled back.
For each DML statement I throw an Error describing the message. For example:
try{
sql.executeInsert("""
insert into mytable1 (col1, col2) values (${val1}, ${val2})
""")
catch(e){
throw new Error("you cant enter empty val1 or val2")
}
try{
sql.executeInsert("""
insert into mytable2 (col1, col2) values (${val1}, ${val2})
""")
catch(e){
throw new Error("you cant enter empty val1 or val2. The previous insert is rolledback!")
}
Final gotcha! The service when called from the controller, must be in a try catch, as follows:
try{
myService.myMethod(params)
}catch(e){
//http://jts-blog.com/?p=9491
Throwable t = e instanceof UndeclaredThrowableException ? e.undeclaredThrowable : e
// use t.toString() to send info to user (use in view)
// redirect / forward / render etc
}
I have made a definition which fetches a user from the database.
def user(userId: Int) : User = database withSession {
(for{
u <- Users if u.id === userId}
yield u).first
}
Potetially the database could return an empty list if used with an non existing userId.
However I can't see when a non existing userId would be provided. For example my userId is fetched from the logged in user. And if a non existing userId is provided then I think it's ok to fail the request hard.
Any thoughts?
No it's not ok to fail the request hard :
def user(userId: Int) : Option[User] // is OK
def user(userId: Int) : Either[String,User] // is OK
def user(usedId: Int) : User // is not OK
or else you could create a type (a concept) which encapsulate an Integer which make sure it's a valid UserId (at birthing).
sealed case class UserId(u:Int) //extends AnyVal // If it's scala 2.10.0
object UserId {
def get(i:Int) : Option[UserId] = //some validation
} /// ....
def user(userId:UserId) : User //is OK // well it depends on the semantic of user destruction.
When you make a def, you must make sure there is a proper relation between the domain (this and args) of your function and the codomain (result).
Anyways, do not hesitate to type (create concepts), it will help you to reason about your code.
Why def user(userId: Int) :User is not Ok ?
Because a relation between the elements of Integer to the elements of User doesn't exist. What if UserIds are all positive integers, but you ask for user(-10) ? (it won't happen, right ?) Should this call raise an exception ? Or return null ?
If you think it should return null, then return an Option, it encapsulates the potential missing correspondance.
If you think it should raise an exception, then return :
a Validation[SomethingRepresentingAnError, User] (scalaz),
an Either[SomethingRepresentingAnError, User] (scala 2.7, 2.8, 2.9)
or a Try[User] (scala 2.10)
Having rich return types will help you to use your API correctly.
Btw Scala doesn't use checked exception, so you cannot use exception as an alternative result. Exception should be keept for truly exceptional behaviour (as Runtime Exceptions).
See also :
http://www.scala-lang.org/api/current/index.html#scala.util.control.Exception$
I think it's always good idea to return Option[] when fetching data by id. You can not be sure that user with such id exist. E. g. another request has deleted this user or somebody was trying to tamper with your input data. Database is an external system to your application and if you know how to recover from such failures then you should do it. Especially in Scala where Option is a good tool for such task.
Option is the most minimalistic way to represent the return value from some computation that may fail. Throwing exceptions or returning null are acceptable only when dealing with Java code and your hands are somehow tied by an existing API (and when you're code is being called from Java code).
The next step up from Option would be Either[FailureIndication, SuccessValue].
A further improvement is ScalaZ's Validation.
I am using hsqldb for database. i am using jdbctemplate for sqlqueries. i just want to know how i can confirm that jdbctemplate executed query successfully, as i can't see the result in database, because my database is hsqldb.
Thank in advance
JdbcTemplate.update(..) returns the number of updated rows as an integer. Check if that is greater than zero or not:
if(jdbcTemplate.update("insert into mytable..") > 0) {
// all ok
} else {
// not inserted anything
}
Instead of using HSQLDB as a pure memory DB, you can write out the contents on disc by initializing HSQLDB with the following URL:
jdbc:hsqldb:file:/opt/db/testdb
I presume you are using a "memory" URL like this (all contents as you notice are gone after the JVM shuts down):
jdbc:hsqldb:mem:mycooldb
When you shut down the database after the test, you can either view the resulting script-file using a texteditor, or start the HSQLDB-manager contained in the main HSQLDB jar.
java -jar hsqldb-version.jar