I am using tire gem to query elasticsearch in my database. Now, I have a model ContentElastic which connects to index posts and mapping post. I am using elasticsearch as a single database and not as a search engine over another database. So, the model ContentElastic goes like this:
class ContentElastic
include Tire::Model::Persistence
include Tire::Model::Search
include Tire::Model::Callbacks
index_name 'posts'
document_type 'post'
# fields
property :id
property :type, :type => "string"
property :username
property :name
property :datasource
property :content
property :language_id, :type => "long"
property :geotag
property :geo_enabled, :type => "boolean"
property :language_confidence, :type => "long"
property :tweet_id
...
end
Now, I have written a query like this, to get 10,000 records from the elasticsearch database:
ContentElastic.search do
fields wanted_fields if wanted_fields.present?
query do
filtered do
query { string search_query } if search_query.present?
#### Filters
range_filters.to_a.each do |rf|
filter :range, rf if rf.present?
end
not_filters.to_a.each do |nf|
filter :not, nf if nf.present?
end
or_filters.to_a.each do |of|
filter :or, of if of.present?
end
and_filters.to_a.each do |af|
filter :and, af if af.present?
end
filter :script, {:script => script} if script.present?
terms_filters.to_a.each do |tf|
filter :terms, tf if tf.present?
end
end
end
size 10000
end
The query is working fine, and we are getting the results. The problem, is that, for 10,000 records, the conversion to ruby objects takes time. Is there a way to get the elasticsearch query directly from the tire declaration and call it separately via RestClient.
Is there a way to get the elasticsearch query without executing the query in Tire. I tried to_curl method in Tire immediately after the tire search block, but it seems the query is already executed and it says Undefined method 'to_curl' for <#Tire::Collection>.
I was able to find a workaround for this. Instead of calling ContentElastic.search directly, I was able to call Tire.search like this:
tire_block = Proc.new do
fields wanted_fields if wanted_fields.present?
query do
filtered do
query { string search_query } if search_query.present?
#### Filters
range_filters.to_a.each do |rf|
filter :range, rf if rf.present?
end
not_filters.to_a.each do |nf|
filter :not, nf if nf.present?
end
or_filters.to_a.each do |of|
filter :or, of if of.present?
end
and_filters.to_a.each do |af|
filter :and, af if af.present?
end
filter :script, {:script => script} if script.present?
terms_filters.to_a.each do |tf|
filter :terms, tf if tf.present?
end
end
end
size 10000
end
t = Tire.search(ContentElastic.index.name, :type => ContentElastic.document_type, :size => limit, :from => from, &tire_block)
This tire call was not executing the query, but was giving the tire search object. Then, I was able to execute the elastic query corresponding using RestClient like this:
post_url = t.url + (t.params.empty? ? '?' : t.params.to_s)
response = RestClient.post(post_url, t.to_json.gsub("'",'\u0027'))
The response will be the json string from elasticsearch and I was able to use this json_string by parsing using JSON.parse and taking the appropriate tags.
Related
This must be an easy one, but I'm stuck...
So I'm using Rails#3 with Mongoid and want to dynamically build query that would depend upon passed parameters and then execute find().
Something like
def select_posts
query = :all # pseudo-code here
if (params.has_key?(:author))
query += where(:author => params[:author]) # this is pseudo-code again
end
if (params.has_key?(:post_date))
query += where(:create_date => params[:post_date]) # stay with me
end
#post_bodies = []
Post.find(query).each do |post| # last one
#post_bodies << post.body
end
respond_to do |format|
format.html
format.json { render :json => #post_bodies }
end
end
You have a few different options to go with here - depending on how complex your actual application is going to get. Using your example directly - you could end up with something like:
query = Post.all
query = query.where(:author => params[:author]) if params.has_key?(:author)
query = query.where(:create_date => params[:post_date]) if params.has_key?(:post_date)
#post_bodies = query.map{|post| post.body}
Which works because queries (Criteria) in Mongoid are chainable.
Alternatively, if you're going to have lots more fields that you wish to leverage, you could do the following:
query = Post.all
fields = {:author => :author, :post_date => :create_date}
fields.each do |params_field, model_field|
query = query.where(model_field => params[params_field]) if params.has_key?(params_field)
end
#post_bodies = query.map{|post| post.body}
And finally, you can take it one level further and properly nest your form parameters, and name the parameters so that they match with your model, so that your params object looks something like this:
params[:post] = {:author => "John Smith", :create_date => "1/1/1970", :another_field => "Lorem ipsum"}
Then you could just do:
#post_bodies = Post.where(params[:post]).map{|post| post.body}
Of course, with that final example, you'd want to sanitize the input fields - to prevent malicious users from tampering with the behaviour.
I have a simple form, where I set up a query that I want to browse, for example panasonic viera.
This is on how I search the term in database:
Product.where("name ilike ?", "%#{params[:q]}%").order('price')
The query looks like %panasonic viera%, but I would need to search the query this way: %panasonic%viera% - I need to find all products, where is in the title the word panasonic or viera... but how to make this query?
One solution would be to break up your query into individual terms and build a set of database queries connected by OR.
terms = params[:q].split
query = terms.map { |term| "name like '%#{term}%'" }.join(" OR ")
Product.where(query).order('price')
If you're using PostgreSQL, you can use pg_search gem. It's support full text search, with option any_word:
Setting this attribute to true will perform a search which will return all models containing any word in the search terms.
Example from pg_search:
class Number < ActiveRecord::Base
include PgSearch
pg_search_scope :search_any_word,
:against => :text,
:using => {
:tsearch => {:any_word => true}
}
pg_search_scope :search_all_words,
:against => :text
end
one = Number.create! :text => 'one'
two = Number.create! :text => 'two'
three = Number.create! :text => 'three'
Number.search_any_word('one two three') # => [one, two, three]
Number.search_all_words('one two three') # => []
How about via ARel
def self.search(query)
words = query.split(/\s+/)
table = self.arel_table
predicates = []
words.each do |word|
predicates << table[:name].matches("%#{word}%")
end
if predicates.size > 1
first = predicates.shift
conditions = Arel::Nodes::Grouping.new(predicates.inject(first) {|memo, expr| Arel::Nodes::Or.new(memo, expr)})
else
conditions = predicates.first
end
where(conditions).to_a
end
This isn't working?
WHERE name LIKE "panasonic" OR name LIKE "viera"
In order to avoid having to construct complicated dynamic SQL queries, I'd like to be able to just pass in nil values in my conditions, and have those ignored. Is that supported by ActiveRecord?
Here is an example.
event = Event.find(:all, :conditions => {
:title => params[:title],
:start_time => params[:start_time],
:end_time => params[:end_time]
}).first
In that particular case, if params[:start_time] is set to nil, ActiveRecord will search for those Events that have their start_time set to null. Instead, I'd like it to just ignore start_time. How do I do that?
You don't have to "create complicated dynamic SQL queries" to do what you need. Simply construct your conditions hash separately, and either exclude the null values at the time of creation or after you've created the hash.
conditions = {}
conditions[:title] = params[:title] unless params[:title].blank?
conditions[:start_time] = params[:start_time] unless params[:start_time].blank?
conditions[:end_time] = params[:end_time] unless params[:end_time].blank?
or
conditions = {:title => params[:title], :start_time => params[:start_time], :end_time => params[:end_time]}
conditions.delete_if {|k,v| v.blank? }
or
conditions = params.reject {|k,v| !([:title, :start_time, :end_time]).include?(k) }
but that last form will only work if the keys are actually symbols. In Rails the params hash is a HashWithIndifferentAccess which allows you to access the text keys as symbols. Of course you could just use the text values in your array of keys to include if necessary.
and then query with your pre-built conditions hash:
event = Event.find(:all, :conditions => conditions).first
I'm making a rails application so that users can search a database of midi records and find midi files that correspond to the attributes that I've given them.
For example, a user might enter data into an html form for a midi file with name = "blah" composer= "buh" and difficulty = "insane".
This is all fine and well, except that I would like when the user enters no data for a field, that field is ignored when doing the select statement on the database.
Right now this is what my select statement looks like:
#midis=Midi.where(:name => params[:midi][:name],
:style => params[:midi][:style],
:numparts => params[:midi][:numparts],
:composer=> params[:midi][:composer],
:difficulty => params[:midi[:difficulty])
This works as expected, but if for example he/she leaves :composer blank, the composer field should not considered at all. This is probably a simple syntax thing but i wasn't able to find any pages on it.
Thanks very much!
Not sure if Arel supports that directly, but you could always do something like:
conditions = {
:name => params[:midi][:name],
:style => params[:midi][:style],
:numparts => params[:midi][:numparts],
:composer=> params[:midi][:composer],
:difficulty => params[:midi[:difficulty]
}
#midis=Midi.where(conditions.select{|k,v| v.present?})
Try this:
# Select the key/value pairs which are actually set and then convert the array back to Hash
c = Hash[{
:name => params[:midi][:name],
:style => params[:midi][:style],
:numparts => params[:midi][:numparts],
:composer => params[:midi][:composer],
:difficulty => params[:midi][:difficulty]
}.select{|k, v| v.present?}]
Midi.where(c)
I'm calling a pretty simple function, and can't seem to figure out whats going on. (I'm using rails 3.0.3 and the master branch of 'will_paginate' gem). I have the following code:
results = Article.search(params) # returns an array of articles
#search_results = results.paginate :page => params[:page], :per_page=>8, :order => order_clause
No matter what I make the order_clause (for example 'article_title desc' and 'article_title asc'), the results are always the same in the same order. So when I check using something like #search_results[0], the element is always the same. In my view, they are obviously always the same as well. Am I totally missing something?
I'm sure its something silly, but I've been banging my head against the wall all night. Any help would be much appreciated!
Edited to Add: The search clause does the following:
def self.search(params)
full_text_search(params[:query].to_s).
category_search(params[:article_category].blank? ? '' : params[:article_category][:name]).
payout_search(params[:payout_direction], params[:payout_value]).
length_search(params[:length_direction], params[:length_value]).
pending.
distinct.
all
end
where each of these guys is a searchlogic based function like this:
#scopes
scope :text_search, lambda {|query|
{
:joins => "INNER JOIN users ON users.id IN (articles.writer_id, articles.buyer_id)",
:conditions => ["(articles.article_title LIKE :query) OR
(articles.description LIKE :query) OR
(users.first_name LIKE :query) OR
(users.last_name LIKE :query)", { :query => "%#{query}%" }]
}
}
scope :distinct, :select => "distinct articles.*"
#methods
def self.payout_search(dir, val)
return no_op if val.blank?
send("payment_amount_#{dir.gsub(/\s+/,'').underscore}", val)
end
def self.length_search(dir, val)
return no_op if val.blank?
send("min_words_#{dir.gsub(/\s+/,'').underscore}", val)
end
Thanks.
If you look at the example from the will_paginate github page you can spot one important difference between their use of the :order clause and yours:
#posts = Post.paginate :page => params[:page], :order => 'created_at DESC'
This calls paginate on the Post object (with no objects being selected yet - no SQL has been executed before paginate comes along). This is different in your example: as you state in the first line of code "returns an array of articles". The simplest I can come up with showing the problem is
results = Model.limit(5).all
#results = results.paginate :order => :doesnt_matter_anymore
won't sort, but this will:
results = Model.limit(5)
#results = results.paginate :order => :matters
It should suffice to take the all out of the search method. It makes ActiveRecord actually perform the SQL query when calling this method. Will_paginate will do that for you when you call paginate (if you let it...). Check out the section on Lazy Loading in this post about Active Record Query Interface 3.0