How do you invoke database functions from grails?
Such as the CONVERT(decimal, timestamp)
Without using groovy SQL like this:
def sql = new Sql(dataSource)
I would like to avoid any SQL statements and take full advantage of the grails magic.
My understanding is that grails does not have native support for database functions. You will need to write SQL. Gorm is an abstraction of Hibernate, you may want to refer to:
How can you call custom database functions with Hibernate?
In your Domain class First you must -
import groovy.sql.Sql
Then you can create Sql instance and call a function like below -
def date = "'21-Feb-19 14:10:10.123000'"
def timestamp_format = "'DD-Mon-RR HH24:MI:SS.FF'"
String query = 'select to_timestamp('+date+','+timestamp_format+') from dual'
def sql = Sql.newInstance('url', 'username', 'password', 'driverClassName')
def result= sql.rows(query);
def rows = result.collect{it.values()}
println rows
Related
I have a table of call data and I want to query all unanswered calls, which means that the call start time is equal to the call end time. I currently use the following plain SQL which works as expected:
select * from calls where calls.start = calls.end
I was wondering if there is a more "rails" way to do this using the ActiveRecord Query Interface. Ideally I'd like to set up a scope in my Call model that returns me all unanswered calls. Something like:
scope :unanswered, -> { where(start: :end) }
The above doesn't work because Rails treats :end as a string instead of the end column in the DB.
I'm using PostgreSQL as my DB engine.
The SQL query
select * from calls where calls.start = calls.end
could be done in a rails way using a scope as follows:
scope :unanswered, -> { where('start = end') }
I think you can do the following:
scope :unanswered, -> (end) { where(start: end) }
I am new to spark and spark sql and i was trying to query some data using spark SQL.
I need to fetch the month from a date which is given as a string.
I think it is not possible to query month directly from sparkqsl so i was thinking of writing a user defined function in scala.
Is it possible to write udf in sparkSQL and if possible can anybody suggest the best method of writing an udf.
You can do this, at least for filtering, if you're willing to use a language-integrated query.
For a data file dates.txt containing:
one,2014-06-01
two,2014-07-01
three,2014-08-01
four,2014-08-15
five,2014-09-15
You can pack as much Scala date magic in your UDF as you want but I'll keep it simple:
def myDateFilter(date: String) = date contains "-08-"
Set it all up as follows -- a lot of this is from the Programming guide.
val sqlContext = new org.apache.spark.sql.SQLContext(sc)
import sqlContext._
// case class for your records
case class Entry(name: String, when: String)
// read and parse the data
val entries = sc.textFile("dates.txt").map(_.split(",")).map(e => Entry(e(0),e(1)))
You can use the UDF as part of your WHERE clause:
val augustEntries = entries.where('when)(myDateFilter).select('name, 'when)
and see the results:
augustEntries.map(r => r(0)).collect().foreach(println)
Notice the version of the where method I've used, declared as follows in the doc:
def where[T1](arg1: Symbol)(udf: (T1) ⇒ Boolean): SchemaRDD
So, the UDF can only take one argument, but you can compose several .where() calls to filter on multiple columns.
Edit for Spark 1.2.0 (and really 1.1.0 too)
While it's not really documented, Spark now supports registering a UDF so it can be queried from SQL.
The above UDF could be registered using:
sqlContext.registerFunction("myDateFilter", myDateFilter)
and if the table was registered
sqlContext.registerRDDAsTable(entries, "entries")
it could be queried using
sqlContext.sql("SELECT * FROM entries WHERE myDateFilter(when)")
For more details see this example.
In Spark 2.0, you can do this:
// define the UDF
def convert2Years(date: String) = date.substring(7, 11)
// register to session
sparkSession.udf.register("convert2Years", convert2Years(_: String))
val moviesDf = getMoviesDf // create dataframe usual way
moviesDf.createOrReplaceTempView("movies") // 'movies' is used in sql below
val years = sparkSession.sql("select convert2Years(releaseDate) from movies")
In PySpark 1.5 and above, we can easily achieve this with builtin function.
Following is an example:
raw_data =
[
("2016-02-27 23:59:59", "Gold", 97450.56),
("2016-02-28 23:00:00", "Silver", 7894.23),
("2016-02-29 22:59:58", "Titanium", 234589.66)]
Time_Material_revenue_df =
sqlContext.createDataFrame(raw_data, ["Sold_time", "Material", "Revenue"])
from pyspark.sql.functions import *
Day_Material_reveneu_df = Time_Material_revenue_df.select(to_date("Sold_time").alias("Sold_day"), "Material", "Revenue")
If I were retrieving the data I wanted from a plain sql query, the following would suffice:
select * from stvterm where stvterm_code > TT_STUDENT.STU_GENERAL.F_Get_Current_term()
I have a grails domain set up correctly for this table, and I can run the following code successfully:
def a = SaturnStvterm.findAll("from SaturnStvterm as s where id > 201797") as JSON
a.render(response)
return false
In other words, I can hardcode in the results from the Oracle function and have the HQL run correctly, but it chokes any way that I can figure to try it with the function. I have read through some of the documentation on Hibernate about using procs and functions, but I'm having trouble making much sense of it. Can anyone give me a hint as to the proper way to handle this?
Also, since I think it is probably relevant, there aren't any synonyms in place that would allow the function to be called without qualifying it as schema.package.function(). I'm sure that'll make things more difficult. This is all for Grails 1.3.7, though I could use a later version if needed.
To call a function in HQL, the SQL dialect must be aware of it. You can add your function at runtime in BootStrap.groovy like this:
import org.hibernate.dialect.function.SQLFunctionTemplate
import org.hibernate.Hibernate
def dialect = applicationContext.sessionFactory.dialect
def getCurrentTerm = new SQLFunctionTemplate(Hibernate.INTEGER, "TT_STUDENT.STU_GENERAL.F_Get_Current_term()")
dialect.registerFunction('F_Get_Current_term', getCurrentTerm)
Once registered, you should be able to call the function in your queries:
def a = SaturnStvterm.findAll("from SaturnStvterm as s where id > TT_STUDENT.STU_GENERAL.F_Get_Current_term()")
I am trying to use the Arel#extract method. I have seen an example in a test case in test_extract.rb in the source but when I try to reproduce it in my app, I get undefined method.
table = Arel::Table.new :users
puts table[:created_at].extract('hour').to_sql
=> NoMethodError: undefined method `extract' for #<Arel::Attributes::Attribute:0x7..8>
I am using pg as the database.
Update:
My goal is to end up with this result in sql:
SELECT users.* FROM users WHERE EXTRACT(HOUR from users."created_at") = '1'
I would like to find all users that were created on the hour equal to one of any day. This works in sql but I am wondering how to create it in arel. Here is an example of how it's used in the arel test suite.
extract is node's method, you can cast it on any column(such as users[:id]), but not on Arel::Table instance.
So, to construct your query you need:
get Arel::Table instance users = Arel::Table.new(:users) or if you use ActiveRecord - users = User.arel_table
set SELECT statement on Arel::Table instance with project method: users = users.project(Arel.sql('*'))
set WHERE statement with where method: users.where(users[:created_at].extract(:hour).eq(1))
In one block:
query = User.arel_table.
project(Arel.sql('*')).
where(users[:created_at].extract(:hour).eq(1))
users = User.find_by_sql(query)
# => "SELECT * FROM \"users\" WHERE EXTRACT(HOUR FROM \"users\".\"created_at\") = 1"
I've had to perform an extract DOW on an instance of Arel::Nodes::NamedFunction, which does not expose the method #extract (as of Arel 6.0). I managed to achieve this by manually creating an instance of Arel::Nodes::Extract. Here is what worked for me, in case anyone have a similar issue:
Arel::Nodes::Extract.new(
Arel::Nodes::NamedFunction.new('some_function_name', [param1, param2, ...]),
:dow
)
You can use an Arel node directly with ActiveRecord's #where instead of building the full query through Arel as exemplified by Alexander Karmes's answer. So here is another way to perform the query required by the answer:
User.where(
Arel::Nodes::Extract.new(User.arel_table[:created_at], :hour).eq(1)
)
Which yields:
SELECT "users".* FROM "users" WHERE EXTRACT(HOUR FROM "users"."created_at") = 1
With the added benefit you can keep chaining other scopes defined in your User model.
I want to do a database-side string concatenation in a Rails query, and do it in database-independent way.
SQL-92 specifies double-bar (||) as the concatenation operator. Unfortunately it looks like MS SQL Server doesn't support it; it uses + instead.
I'm guessing that Rails' SQL grammar abstraction has solved the db-specific operator problem already. If it does exist, how do I use it?
I had the same problem and never came up with anything that was built into Rails. So I wrote this little method.
# Symbols should be used for field names, everything else will be quoted as a string
def db_concat(*args)
adapter = configurations[RAILS_ENV]['adapter'].to_sym
args.map!{ |arg| arg.class==Symbol ? arg.to_s : "'#{arg}'" }
case adapter
when :mysql
"CONCAT(#{args.join(',')})"
when :sqlserver
args.join('+')
else
args.join('||')
end
end
I'm thinking somebody should really write some sort of SQL helper plugin that could automatically format simple SQL expressions based using the correct functions or operators for the current adapter. Maybe I'll write one myself.
It hasn't received much usage yet but I wrote the following code which seems to solve the problem. This monkey-patches the adapters to have a method to support it:
module ActiveRecord
module ConnectionAdapters
class AbstractAdapter
# Will return the given strings as a SQL concationation. By default
# uses the SQL-92 syntax:
#
# concat('foo', 'bar') -> "foo || bar"
def concat(*args)
args * " || "
end
end
class AbstractMysqlAdapter < AbstractAdapter
# Will return the given strings as a SQL concationation.
# Uses MySQL format:
#
# concat('foo', 'bar') -> "CONCAT(foo, bar)"
def concat(*args)
"CONCAT(#{args * ', '})"
end
end
class SQLServerAdapter < AbstractAdapter
# Will return the given strings as a SQL concationation.
# Uses MS-SQL format:
#
# concat('foo', 'bar') -> foo + bar
def concat(*args)
args * ' + '
end
end
end
end
With this you should be able to do the following in your code:
class User < ActiveRecord::Base
def self.find_by_name(name)
where("#{connection.concat('first_name', 'last_name')} = ?", name)
end
end
This outputs the following SQL query on a SQL-92 database (Oracle, SQLite, PostgreSQL):
SELECT * FROM users WHERE first_name || last_name = ?
For MySQL it outputs:
SELECT * FROM users WHERE CONCAT(first_name, last_name) = ?
For SQL Server it outputs
SELECT * FROM users WHERE first_name + last_name = ?
Obviously you could extend this concept to other database adapters.
If you want something Rails neutral, you're going to need to return the values you want concatenated and do that once the data has been delivered to rails (or do it in rails before you give it to the database).
It looks like Mysql uses CONCAT(), Postgres ||, Oracle CONCAT() or ||, T-SQL +.
Any rails abstraction of the same would have to take place at a point where you could just be doing concatenation using regular Ruby, or I've completely misunderstood the question.