thinking sphinx search has_many not working no error - ruby-on-rails-3

model
class Person < ActiveRecord::Base
attr_accessible :alignment, :description, :first_name, :last_name
has_many :roles #table roles with active as one of the field with value equal t or f (boolean)
end
class Role < ActiveRecord::Base
attr_accessible :active, :organization_id, :person_id, :position_id
belongs_to :person
belongs_to :organization
belongs_to :position
end
person_index.rb
ThinkingSphinx::Index.define :person, :with => :active_record do
#Fields
indexes last_name, :sortable => true
indexes first_name, :sortable => true
indexes alignment
#indexes role(:active), :as => :active
indexes role.active, :as => :active
#Attributes
has created_at, updated_at
has professions(:id), :as => :profession_ids
has positions(:id), :as => :position_id
has organizations(:id), :as => :organization_id
end
people_controller
filters = {}
filters[:profession_ids] = params[:profession_ids] if params[:profession_ids].present?
filters[:organization_id] = params[:organization_id] if params[:organization_id].present?
filters[:position_id] = params[:position_id] if params[:position_id].present?
filters[:active_ids] = role if params[:active].present? #added
#people = Person.search " #{params[:lastname]} #{params[:firstname]} #{params[:alignmemt]}",
:with => filters,
:star => true,
:condition => {:alignment => params[:alignment], :active => params[:active]},
:order => 'last_name ASC, first_name ASC',
:page => params[:page],
:per_page => 20
When i search active and/or alignment it is not filtering result and doesn't give me error. these are both string field, alignment is in the people table and active is in another table (roles)
Why? What am i missing?
update
Tried the recommended solution for active on this question and same result...
person_index.rb
ThinkingSphinx::Index.define :person, :with => :active_record do
#Fields
indexes last_name, :sortable => true
indexes first_name, :sortable => true
indexes alignment
#Attributes
has created_at, updated_at
has professions(:id), :as => :profession_ids
has positions(:id), :as => :position_id
has organizations(:id), :as => :organization_id
has roles.active, :as => :active_ids
end
people_controller
def index
#role = Role.find_by_active(params[:active]) #ADDED
filters = {}
filters[:profession_ids] = params[:profession_ids] if params[:profession_ids].present?
filters[:organization_id] = params[:organization_id] if params[:organization_id].present?
filters[:position_id] = params[:position_id] if params[:position_id].present?
#people = Person.search " #{params[:lastname]} #{params[:firstname]} #{params[:alignmemt]}",
:with => filters,
:star => true,
:condition => {:alignment => params[:alignment], :active_ids => #role}, #CHANGED
:order => 'last_name ASC, first_name ASC',
:page => params[:page],
:per_page => 20
but still have same result... WHY?
controller updated after Pat answer
def index
if params[:active].present?
role = Array.new
rolepid = Array.new
role = Role.find_all_by_active(params[:active])
role.each do |num|
puts num.person_id
rolepid << num.person_id #get all the person id whith the params[:active]
end
end
filters = {}
filters[:profession_ids] = params[:profession_ids] if params[:profession_ids].present?
filters[:organization_id] = params[:organization_id] if params[:organization_id].present?
filters[:position_id] = params[:position_id] if params[:position_id].present?
filters[:active_ids] = rolepid if params[:active].present?
#people = Person.search " #{params[:lastname]} #{params[:firstname]} #{params[:alignent]}",
#:classes => [Person, Role],
:with => filters,
:star => true,
:condition => {:alignment => params[:alignment]},
:order => 'last_name ASC, first_name ASC',
:page => params[:page],
:per_page => 20
But now it is looking for active in people table when it should look in roles table. So i added #:classes => [Person, Role], but no luck....
Role Load (0.7ms) SELECT "roles".* FROM "roles" WHERE "roles"."active" = 'f'
Sphinx Query (0.7ms) SELECT * FROM `person_core` WHERE `active_ids` IN (304, 34, 306, 308, 334, 295, 344, 348, 352, 354, 365, 367, 308, 429, 468, 9, 544, 590, 609, 110, 1643, 1652, 1653, 1655, 1669, 628, 1687, 1691, 1709) AND `sphinx_deleted` = 0 ORDER BY `last_name` ASC, first_name ASC LIMIT 0, 20
Sphinx Found 0 results
So i change in the controller
filters[:active_ids] = rolepid if params[:active].present?
to
filters[:id] = rolepid if params[:active].present?
Since rolepid is an array of integer with the person ids.
But Sphinx is just looking for 4 ids that are not in rolepid... I am confused :|
Parameters: {"utf8"=>"✓", "firstname"=>"", "lastname"=>"", "alignment"=>"", "organization_id"=>"", "position_id"=>"", "active"=>"f", "commit"=>"Search"}
Role Load (0.8ms) SELECT "roles".* FROM "roles" WHERE "roles"."active" = 'f'
Sphinx Query (0.6ms) SELECT * FROM `person_core` WHERE `id` IN (304, 34, 306, 308, 334, 295, 344, 348, 352, 354, 365, 367, 308, 429, 468, 9, 544, 590, 609, 110, 1643, 1652, 1653, 1655, 1669, 628, 1687, 1691, 1709) AND `sphinx_deleted` = 0 ORDER BY `last_name` ASC, first_name ASC LIMIT 0, 20
Sphinx Found 4 results
Person Load (0.4ms) SELECT "people".* FROM "people" WHERE "people"."id" IN (84, 1, 61, 50)
Why is it not returning the 29 records from rolepid array ?
filtering for alignment IS working. thanks for catching the misspelled word.

If you're using active_ids as an attribute (which, if it's integers, is certainly appropriate), then it should be a filter in the :with option, not in the :conditions option.
I'm not sure if this is related, but it's worth noting you've misspelled alignment in the query string (you've got alignmemt instead).

Related

How to find the data corresponding to the last date for every category in rails

I am currently trying to list different assets on a rails application. I want to access the name, and last values of each asset, all of that ordered by marketcap.
here is my database structure
`create_table "asset_values", force: :cascade do |t|
t.integer "date"
t.integer "asset_id"
t.float "price"
t.float "marketcap"
t.float "volume"
t.datetime "created_at", precision: 6, null: false
t.datetime "updated_at", precision: 6, null: false
end
create_table "assets", force: :cascade do |t|
t.string "name"
t.string "symbol"
t.datetime "created_at", precision: 6, null: false
t.datetime "updated_at", precision: 6, null: false
end`
date is in unix time
Here is my model structure corresponding to asset
has_many :asset_value
validates :name, :symbol, uniqueness: true
validates :name, :symbol, presence: true
and here is my model structure corresponding to asset_values
belongs_to :asset, foreign_key: "asset_id"
EXAMPLE:
Assets:
id: 1, name: Bitcoin, symbol: BTC
id: 2, name: Ethereum, symbol: ETH
Asset_values:
id:1, price: 12000, marketcap: 211 000 000 000, volume 100, asset_id: 1, date: 1 598 300 000
id:2, price: 11000, marketcap: 210 000 000 000, volume 110, asset_id: 1, date: 1 598 200 000
id:3, price: 400, marketcap: 42 000 000 000, volume 20, asset_id: 2, date: 1 598 300 000
id:4, price: 420, marketcap: 43 000 000 000, volume 20, asset_id: 2, date: 1 598 200 000
The_table_i_want :
id: 1, name: Bitcoin, symbol: BTC, price: 12000, marketcap: 211 000 000 000, volume 100, date: 1 598 300 000
id: 2, name: Ethereum, symbol: ETH, price: 400, marketcap: 42 000 000 000, volume 20, date: 1 598 300 000
I tried different calls in order to access the data but none of them is working...
here is what i think is best
sql = ("
SELECT a.id, a.name, a.symbol, v.price, v.marketcap, v.volume, v.date
FROM assets AS a
INNER JOIN asset_values AS v ON a.id = v.asset_id
INNER JOIN (
SELECT MAX(date)
FROM asset_values
GROUP BY asset_id
) AS p
ON a.id = p.asset_id
ORDER BY v.marketcap DESC
")
#data = ActiveRecord::Base.connection.execute(sql)
I am using postgresql and rails 6
Let me know if these adjustments to your query produce the results you would expect
SELECT a.id
,a.name
,a.symbol
,v.price
,v.marketcap
,v.volume
,v.date
FROM assets AS a
INNER JOIN asset_values AS v
ON a.id = v.asset_id
INNER JOIN (
SELECT asset_id
,MAX(date) as max_date
FROM asset_values
GROUP BY asset_id
) AS p
ON v.asset_id = p.asset_id
AND v.date = p.max_date
ORDER BY v.marketcap DESC
If you are using postgresql 9.3+ you can use lateral join
SELECT a.id
,a.name
,a.symbol
,v.price
,v.marketcap
,v.volume
,v.date
FROM assets AS a
INNER JOIN LATERAL
( SELECT asset_id
,price
,marketcap
,volume
,date
FROM asset_values
WHERE asset_id=a.id
ORDER BY created_at ASC LIMIT 1 ) v
ON v.asset_id=a.id
ORDER BY v.marketcap DESC

"TypeError: no implicit conversion of nil into String" when eager loading results

I'm using ruby '2.3.0' and 'rails', '3.2.22.2'.
I need a little help & explanations about a query I've made. Here's my models:
class AssessmentRaw < ActiveRecord::Base
belongs_to :session
has_many :schedulers, :class_name => 'MailingScheduler', :as => :owner, :dependent => :destroy
end
class MailingScheduler < ActiveRecord::Base
belongs_to :owner, :polymorphic => true
end
class Session < ActiveRecord::Base
has_many :assessment_raws, :dependent => :destroy
end
I want to retrieve all the assessment_raws, and eager load the associated sessions and mailing_schedulers.
1. eager load only sessions
ars = AssessmentRaw.includes(:session).where("sessions.start_at >= ?", 1.year.ago).limit(10)
ars.map { |ar| ar.session.id }
=> [2877, 2878, 2879, 2880, 2881, 2882, 2883, 2884, 2902, 2903]
`ars.map { |ar| ar.schedulers.try(:size) }`
MailingScheduler Load (0.6ms) SELECT "mailing_schedulers".* FROM "mailing_schedulers" WHERE "mailing_schedulers"."owner_id" = 622 AND "mailing_schedulers"."owner_type" = 'AssessmentRaw'
MailingScheduler Load (0.6ms) SELECT "mailing_schedulers".* FROM "mailing_schedulers" WHERE "mailing_schedulers"."owner_id" = 725 AND "mailing_schedulers"."owner_type" = 'AssessmentRaw'
MailingScheduler Load (0.3ms) SELECT "mailing_schedulers".* FROM "mailing_schedulers" WHERE "mailing_schedulers"."owner_id" = 771 AND "mailing_schedulers"."owner_type" = 'AssessmentRaw'
MailingScheduler Load (0.3ms) SELECT "mailing_schedulers".* FROM "mailing_schedulers" WHERE "mailing_schedulers"."owner_id" = 782 AND "mailing_schedulers"."owner_type" = 'AssessmentRaw'
MailingScheduler Load (0.3ms) SELECT "mailing_schedulers".* FROM "mailing_schedulers" WHERE "mailing_schedulers"."owner_id" = 881 AND "mailing_schedulers"."owner_type" = 'AssessmentRaw'
MailingScheduler Load (0.2ms) SELECT "mailing_schedulers".* FROM "mailing_schedulers" WHERE "mailing_schedulers"."owner_id" = 996 AND "mailing_schedulers"."owner_type" = 'AssessmentRaw'
MailingScheduler Load (0.3ms) SELECT "mailing_schedulers".* FROM "mailing_schedulers" WHERE "mailing_schedulers"."owner_id" = 1087 AND "mailing_schedulers"."owner_type" = 'AssessmentRaw'
MailingScheduler Load (0.3ms) SELECT "mailing_schedulers".* FROM "mailing_schedulers" WHERE "mailing_schedulers"."owner_id" = 1155 AND "mailing_schedulers"."owner_type" = 'AssessmentRaw'
MailingScheduler Load (0.2ms) SELECT "mailing_schedulers".* FROM "mailing_schedulers" WHERE "mailing_schedulers"."owner_id" = 653 AND "mailing_schedulers"."owner_type" = 'AssessmentRaw'
MailingScheduler Load (0.2ms) SELECT "mailing_schedulers".* FROM "mailing_schedulers" WHERE "mailing_schedulers"."owner_id" = 940 AND "mailing_schedulers"."owner_type" = 'AssessmentRaw'
=> [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
Of course to get the count of the mailing_schedulers, rails must query (N+1 problem)
2. eager load sessions and mailing_schedulers
ars = AssessmentRaw.includes(:schedulers,:session).where("sessions.start_at >= ?", 1.year.ago).limit(10)
TypeError: no implicit conversion of nil into String
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/arel-3.0.3/lib/arel.rb:40:in `initialize'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/arel-3.0.3/lib/arel.rb:40:in `new'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/arel-3.0.3/lib/arel.rb:40:in `sql'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/activerecord-3.2.22.2/lib/active_record/associations/join_helper.rb:47:in `block in sanitize'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/activerecord-3.2.22.2/lib/active_record/associations/join_helper.rb:45:in `map'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/activerecord-3.2.22.2/lib/active_record/associations/join_helper.rb:45:in `sanitize'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/activerecord-3.2.22.2/lib/active_record/associations/join_dependency/join_association.rb:104:in `block in join_to'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/activerecord-3.2.22.2/lib/active_record/associations/join_dependency/join_association.rb:74:in `each'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/activerecord-3.2.22.2/lib/active_record/associations/join_dependency/join_association.rb:74:in `each_with_index'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/activerecord-3.2.22.2/lib/active_record/associations/join_dependency/join_association.rb:74:in `join_to'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/activerecord-3.2.22.2/lib/active_record/relation/query_methods.rb:370:in `block in build_joins'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/activerecord-3.2.22.2/lib/active_record/relation/query_methods.rb:369:in `each'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/activerecord-3.2.22.2/lib/active_record/relation/query_methods.rb:369:in `build_joins'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/activerecord-3.2.22.2/lib/active_record/relation/query_methods.rb:266:in `build_arel'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/activerecord-3.2.22.2/lib/active_record/relation/query_methods.rb:260:in `arel'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/activerecord-3.2.22.2/lib/active_record/relation/finder_methods.rb:259:in `construct_limited_ids_condition'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/activerecord-3.2.22.2/lib/active_record/relation/finder_methods.rb:243:in `apply_join_dependency'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/activerecord-3.2.22.2/lib/active_record/relation/finder_methods.rb:232:in `construct_relation_for_association_find'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/activerecord-3.2.22.2/lib/active_record/relation/finder_methods.rb:211:in `find_with_associations'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/activerecord-3.2.22.2/lib/active_record/relation.rb:171:in `exec_queries'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/activerecord-3.2.22.2/lib/active_record/relation.rb:160:in `block in to_a'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/activerecord-3.2.22.2/lib/active_record/explain.rb:41:in `logging_query_plan'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/activerecord-3.2.22.2/lib/active_record/relation.rb:159:in `to_a'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/activerecord-3.2.22.2/lib/active_record/relation.rb:498:in `inspect'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/railties-3.2.22.2/lib/rails/commands/console.rb:47:in `start'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/railties-3.2.22.2/lib/rails/commands/console.rb:8:in `start'
from /Users/oim/.rbenv/versions/2.3.0/gemsets/project-gems/gems/railties-3.2.22.2/lib/rails/commands.rb:41:in `<top (required)>'
from script/rails:6:in `require'
from script/rails:6:in `<main>'irb(main):064:0>
ouch. I think I need a LEFT OUTER JOIN here because all assessment_raws doesn't have mailing_schedulers, right?
Any help appreciated.
This issue occurs due to a change made in Ruby 2.3 where Hash now responds to to_proc which confuses the interpolate method. See this bug report for more details: https://github.com/rails/rails/issues/25010. Also note there is a workaround mentioned which resolved the issue for me, but unless you're sure you don't use the new to_proc it could potentially be dangerous.
What worked for me: I added the suggested monkey patch (as shown below) to the top of my config/application.rb
class Hash
undef_method :to_proc if self.method_defined?(:to_proc)
end

Inheritance of rows in SQL

I have a table of rows, here is a sample (as hashes):
{:id => 8, :n => 774, :inherits_from => [], :properties => ["a", "b", "c"]}
{:id => 9, :n => 915, :inherits_from => [8], :properties => ["d"]}
{:id => 10, :n => 754, :inherits_from => [1, 2], :properties => []}
{:id => 11, :n => 774, :inherits_from => [10], :properties => ["a", "b"]}
The idea is that properties can be inherited from other rows:
table[:id => 9] #=> {:id => 9, :n => 915, :inherits_from => [8], :properties => ["a", "b", "c", "d"]}
Rows can have one or several inheritances, or can have none. There can be many levels of inheritance, i.e., 5 inherits from 2 and 3, 7 inherits from 5 (which means that 7 inherits from 5, but also from 2 and 3, because 5 inherits from them) etc.
I know I could do it with brute force (i.e., for every queried row check if it inherits from somewhere). Also, I could use Class.new to generate a class for each row and set inheritances (I use ruby), but there will be thousands of rows.
Any advise on how could I do it in the most efficient way?

Calculate Percent of total from scoped records and display virtual attributes (Rails / Active Record)

Create a scope, maybe something like this..
scope :mv, select('*,quantity*market_price as market_value, quantity*market_price/sum(quantity*market_price) as percent')
that creates two virtual attributes, market_value and percent. The problem I am having is creating the percent with the sum() included. If I add sum(), the scope returns one record.
I need to calculate the percent of market_value relative to the total market value of the scoped record set.
example:
1, market_value: 100, percent: .10
2, market_value: 100, percent: .10
3, market_value: 100, percent: .10
4, market_value: 100, percent: .10
5, market_value: 100, percent: .10
6, market_value: 500, percent: .50
Total is 1000
however, if I scope it to where market_value < 6, I should see
1, market_value: 100, percent: .20
2, market_value: 100, percent: .20
3, market_value: 100, percent: .20
4, market_value: 100, percent: .20
5, market_value: 100, percent: .20
Total 500
How can I accomplish this?
I created a self.pct method but problem with the self.pct method is that it needs to be run after all the scopes. if rescoped, the solution is wrong
So far,
class Position < ActiveRecord::Base
attr_accessible :account_id, :account_type, :market_price, :quantity, :report_date, :symbol
scope :long_only, where(:account_type => 'L')
scope :short_only, where(:account_type=>"S")
scope :equity_only, :conditions => ["symbol <> 'USD'"]
scope :mv, select('*,quantity*market_price as market_value, quantity*market_price/sum(quantity*market_price) as percent')
scope :mv1, lambda{|total| select(total) }
#the problem with the self.pct method is that it needs to be run after all the scopes. if rescoped, the solution is wrong
def self.pct
string="*,(quantity*market_price) as market_value, (market_price*quantity/#{sum_market_value}) as percent"
mv1(string)
end
def market_value
self.market_price*self.quantity
end
def self.sum_market_value
sum('quantity*market_price')
end
end
I don't know if there's a way to do this in a single query, but we can get it in two queries:
require 'active_record'
ActiveRecord::Base.establish_connection adapter: 'sqlite3', database: ':memory:'
ActiveRecord::Schema.define do
self.verbose = false
create_table :positions do |t|
t.integer :quantity
t.integer :market_price
end
end
class Position < ActiveRecord::Base
def self.with_market_value
select "*,
quantity*market_price as market_value,
quantity*market_price/#{total.to_f} as percent"
end
def self.total
select('sum(quantity*market_price) as sum_of_market_values').first.sum_of_market_values
end
end
Position.create! quantity: 25, market_price: 4
Position.create! quantity: 25, market_price: 4
Position.create! quantity: 25, market_price: 4
Position.create! quantity: 25, market_price: 4
Position.create! quantity: 25, market_price: 4
Position.create! quantity: 25, market_price: 20
Position.with_market_value.map { |p| [p.market_value, p.percent] }
# => [[100, 0.1], [100, 0.1], [100, 0.1], [100, 0.1], [100, 0.1], [500, 0.5]]
Position.where('market_price < 10').with_market_value.map { |p| [p.market_value, p.percent] }
# => [[100, 0.2], [100, 0.2], [100, 0.2], [100, 0.2], [100, 0.2]]
# ** NOTE THAT IT EXECUTES EAGERLY **
Position.with_market_value.where('market_price < 10').map { |p| [p.market_value, p.percent] }
# => [[100, 0.1], [100, 0.1], [100, 0.1], [100, 0.1], [100, 0.1]]

Group By Year, Month & then Count in ActiveRecord 3

I'm trying to pull a count of all users created in a year, and month but the following doesn't seem to work as expected.
User.group("YEAR(created_AT), MONTH(created_at)").
count("DISTINCT(id), YEAR(created_at), MONTH(created_at)")
i'm looking for something like
{2011 => {1 => 222, 2 => 333, 4 => 444, 5 => 667 ... }}
but i'm getting
{1 => 222, 2 => 333, 4 => 444, 5 => 667 ... }
Am i missing something, or can ActiveRecord not give me this result in one query?
The count method doesn't work like you think it does. You end up doing this:
select count(distinct(id), year(created_at), month(created_at))
from users
group by year(created_at), month(created_at)
That SELECT clause is pretty dodgy but MySQL will, in its usual sloppy manner, let it through. I think you want this query:
select count(distinct(id)), year(created_at), month(created_at)
from users
group by year(created_at), month(created_at)
I'd probably go straight to select_all like this:
a = User.connection.select_all(%q{
select count(distinct(id)) as c, year(created_at) as y, month(created_at) as m
from users
group by y, m
})
Or you could do it like this:
a = User.connection.select_all(
User.select('count(distinct(id)) as c, year(created_at) as y, month(created_at) as m').
group('y, m')
)
Those will give you an array, a, of Hashes with c, y, and m keys like this:
a = [
{ 'c' => '23', 'y' => '2010', 'm' => '11' },
{ 'c' => '1', 'y' => '2011', 'm' => '1' },
{ 'c' => '5', 'y' => '2011', 'm' => '3' },
{ 'c' => '2', 'y' => '2011', 'm' => '4' },
{ 'c' => '11', 'y' => '2011', 'm' => '8' }
]
Then a bit of data wrangling is all you need to finish the job:
h = a.group_by { |x| x['y'] }.each_with_object({}) do |(y,v), h|
h[y.to_i] = Hash[v.map { |e| [e['m'].to_i, e['c'].to_i] }]
end
# {2010 => {11 => 23}, 2011 => {1 => 1, 3 => 5, 4 => 2, 8 => 11}}