im trying to find records that are NOT created in a certain date range using a thinking sphinx query in my rails 4 project with postgresql as database
THIS WORKS
Housing.search "some text",
star: true,
ranker: :bm25,
with: {created_at (3.weeks.ago..Time.zone.now)}
BUT THIS DOESN´T
Housing.search "some text",
star: true,
ranker: :bm25,
without: {created_at:(3.weeks.ago..Time.zone.now)}
I´m thinking its a bug in the query since it finds all record within the given date range, but fails to find the opposite?
This is my gemfile
gem 'mysql2', '0.3.12b4'
gem 'thinking-sphinx', github: 'pat/thinking-sphinx'
gem 'flying-sphinx', github: 'flying-sphinx/flying-sphinx'
gem 'delayed_job_active_record', '>= 4.0.0.beta2'
gem 'delayed_job_web'
gem 'ts-delayed-delta', github: 'pat/ts-delayed-delta'
This is my housing_indices
ThinkingSphinx::Index.define :housing, :with => :active_record, :delta => ThinkingSphinx::Deltas::DelayedDelta do
has created_at
< + OTHER FIELDS THAT ARE INDEXED >
set_property :enable_star => true
set_property :min_infix_len => 1
set_property :html_strip => true
set_property :charset_type => "utf-8"
end
this is the query from the log when using without: {created_at:(3.weeks.ago..Time.zone.now)}
Sphinx Query (1.0ms) SELECT * FROM `housing_core`, `housing_delta` WHERE MATCH('#location_name *Krimml*') AND max_capacity BETWEEN 1 AND 1000 AND number_of_bathrooms BETWEEN 1 AND 3 AND number_of_bed_places BETWEEN 2 AND 10 AND sphinx_deleted = 0 AND created_at < 1376554070 OR created_at > 1378368470 AND activated_at <> 0 LIMIT 0, 20 OPTION ranker=bm25
And this Is the error I am getting
sphinxql: syntax error, unexpected OR, expecting $end near 'OR created_at > 1378368470 AND activated_at <> 0 LIMIT 0, 20 OPTION ranker=bm25; SHOW META'
this is the query from the log when using with: {created_at:(3.weeks.ago..Time.zone.now)}
Sphinx Query (3.9ms) SELECT * FROM `housing_core`, `housing_delta` WHERE MATCH('#location_name *Krimml*') AND created_at BETWEEN 1376555152 AND 1378369552 AND max_capacity BETWEEN 1 AND 1000 AND number_of_bathrooms BETWEEN 1 AND 3 AND number_of_bed_places BETWEEN 2 AND 10 AND sphinx_deleted = 0 AND activated_at <> 0 LIMIT 0, 20
so i think what I´m looking for is this query to be executed
AND created_at NOT BETWEEN 1376555152 AND 1378369552
I think that the without method doesn't have ranges, i imagine is because you want to index portions of data and not the all data without some portion, so you can use the inverse expression.
A without like this
without: {created_at (3.weeks.ago..Time.zone.now)}
will be something like this(with a safe year for your data)
with: {created_at (Time.now.change(:year => 2012)..3.weeks.ago)}
Of course your case is a good case for this because you wouldn't have created things in the "future" if the data is consistence.
Related
I want to perform a query on first 10 records.
So, from Rails console I type:
Log.all.limit(10).where({"username"=>"peeyush"}).explain
This gives:
Log Load (0.8ms) SELECT "logs".* FROM "logs" WHERE "logs"."username" = 'peeyush' LIMIT 10
Clearly, LIMIT 10 happens later.
I try running:
Log.all.first(10).where({"username"=>"peeyush"}).explain
But this gives an error:
NoMethodError: undefined method `where' for #<Array:0x0000000539acd8>
How should the query be performed?
If I understand you correctly, you want to retrieve the 1st 10 rows and then filter those 10 records by username?
Filter in ruby
all.first(10).find_all {|i| i.username == "peeyush" }
Filter on the database
all.where(:id => all.first(10).map {|x| x.id}, :username => "peeyush")
I have an Item model with a name and user_id. I would like to search all Items and group by User so I can display each user with their items:
User 1:
Item A
Item B
User 2
Item C
User 3
Item D
Item E
Item D
...
In the console, I try this: (From the documentation)
Item.search({group_by: :user_id, limit: 50}).all
And I get this:
Sphinx Query (0.4ms)
Sphinx Caught Sphinx exception: can't dup Symbol (0 tries left)
TypeError: can't dup Symbol
from /Users/pinouchon/.rvm/gems/ruby-1.9.3-p392#gemset/gems/riddle-1.5.6/lib/riddle/client/message.rb:18:in `dup'
Same error with this:
Item.search({group_by: :user_id, order_group_by: '#count desc'}).each_with_group
Search with no group by returns results without any problem.
What's wrong ?
The quick answer: try sending through the attribute name as a string, not a symbol.
The longer answer: that query isn't going to give you the results you want – it'll return one item per user. You'd be better served sorting by user_id instead:
items = Item.search(
:order => 'user_id ASC, #weight DESC',
:sort_mode => :extended,
:limit => 50
)
From there, you could then get the layer of users grouping each items using Ruby/Rails:
items.group_by(&:user)
I have a Ruby on Rails 2.3.x application that I'm trying to migrate from my own VPS to Heroku, including porting from SQLite (development) and MySQL (production) to Postgres.
This is a typical Rails call I'm using:
spots = Spot.paginate(:all, :include => [:thing, :user, :store, {:thing => :tags}, {:thing => :brand}], :group => :thing_id, :order => order, :conditions => conditions, :page => page, :per_page => per_page)
Question 1: I get a lot of errors like PG::Error: ERROR: column "spots.id" must appear in the GROUP BY clause or be used in an aggregate function. SQLite/MySQL was evidently more forgiving here. Of course I can easily fix these by adding the specified fields to my :group parameter, but I feel I'm messing up my code. Is there a better way?
Question 2: If I throw in all the GROUP BY columns that Postgres is missing I end up with the following statement (only :group has changed):
spots = Spot.paginate(:all, :include => [:thing, :user, :store, {:thing => :tags}, {:thing => :brand}], :group => 'thing_id,things.id,users.id,spots.id', :order => order, :conditions => conditions, :page => page, :per_page => per_page)
This in turn produces the following SQL code:
SELECT * FROM (SELECT DISTINCT ON ("spots".id) "spots".id, spots.created_at AS alias_0 FROM "spots"
LEFT OUTER JOIN "things" ON "things".id = "spots".thing_id
WHERE (spots.recommended_to_user_id = 1 OR spots.user_id IN (1) OR things.is_featured = 't')
GROUP BY thing_id,things.id,users.id,spots.id) AS id_list
ORDER BY id_list.alias_0 DESC LIMIT 16 OFFSET 0;
...which produces the error PG::Error: ERROR: missing FROM-clause entry for table "users". How can I solve this?
Question 1:
...Is there a better way?
Yes. Since PostgreSQL 9.1 the primary key of a table logically covers all columns of a table in the GROUP BY clause. I quote the release notes for version 9.1:
Allow non-GROUP BY columns in the query target list when the primary
key is specified in the GROUP BY clause (Peter Eisentraut)
Question 2:
The following statement ... produces the error
PG::Error: ERROR: missing FROM-clause entry for table "users"
How can I solve this?
First (as always!), I formatted your query to make it easier to understand. The culprit has bold emphasis:
SELECT *
FROM (
SELECT DISTINCT ON (spots.id)
spots.id, spots.created_at AS alias_0
FROM spots
LEFT JOIN things ON things.id = spots.thing_id
WHERE (spots.recommended_to_user_id = 1 OR
spots.user_id IN (1) OR
things.is_featured = 't')
GROUP BY thing_id, things.id, users.id, spots.id
) id_list
ORDER BY id_list.alias_0 DESC
LIMIT 16
OFFSET 0;
It's all obvious now, right?
Well, not all of it. There is a lot more. DISTINCT ON and GROUP BY in the same query for one, which has its uses, but not here. Radically simplify to:
SELECT s.id, s.created_at AS alias_0
FROM spots s
WHERE s.recommended_to_user_id = 1 OR
s.user_id = 1 OR
EXISTS (
SELECT 1 FROM things t
WHERE t.id = s.thing_id
AND t.is_featured = 't')
ORDER BY s.created_at DESC
LIMIT 16;
The EXISTS semi-join avoids the later need to GROUP BY a priori. This should be much faster (besides being correct) - if my assumptions about the missing table definitions hold.
Going the "pure SQL" route opened up a can of worms for me, so I tried keeping the will_paginate gem and tweak the Spot.paginate parameters instead. The :joins parameter turned out to be very helpful.
This is currently working for me:
spots = Spot.paginate(:all, :include => [:thing, {:thing => :tags}, {:thing => :brand}], :joins => [:user, :store, :thing], :group => 'thing_id,things.id,users.id,spots.id', :order => order, :conditions => conditions, :page => page, :per_page => per_page)
My starting point is basically Ryan Bates Railscast.
I have User model that I need to do some queries on. The model has a couple hourly rate attributes as follows:
#User migration
...
t.decimal :hour_price_high
t.decimal :hour_price_low
...
I have the query working in the User.where(Array) format where Array is formatted
["users.hour_price_high <= ? OR users.hour_price_low >= ? OR users.hour_price_low <= ? AND users.hour_price_high >= ?", hour_price_high, hour_price_low, hour_price_low, hour_price_high]
#This is simply a search of two ranges. Search for $40/h - $60/h.
#It will return an User who charge any overlapping price range. Such as one who charges $45/h - $65/h, etc.
I simply wish to convert this into Ruby syntax in the where statement.
My problem is how to represent the OR.
#This isn't the full query shown above..
User.where(:high_price_high => hour_price_low..hour_price_high, :hour_price_low => hour_price_low..hour_price_high)
Produces this SQL:
=> "SELECT `users`.* FROM `users` WHERE (`users`.`hour_price_high` BETWEEN 45 AND 60) AND (`users`.`hour_price_low` BETWEEN 45 AND 60)"
Which, of course, is wrong because of the AND. I need it to be:
=> "SELECT `users`.* FROM `users` WHERE (`users`.`hour_price_high` BETWEEN 45 AND 60) OR (`users`.`hour_price_low` BETWEEN 45 AND 60)"
How can I make this be an OR statement without busting out the old truth table from my sophomore E.E. classes?
Thanks!
When you chain several where methods or have a few arguments in one where method, then you always get AND between them, so when you want to have OR you need to use this syntax:
User.where("users.hour_price_high <= ? OR users.hour_price_low >= ? OR users.hour_price_low <= ? AND users.hour_price_high >= ?", hour_price_high, hour_price_low, hour_price_low, hour_price_high)
Note: please watch http://railscasts.com/episodes/202-active-record-queries-in-rails-3 in order to get more information about active record queries.
I have an app where users can vote for entries. They are limited to a total number of votes per 24 hours, based on a configuration stored in my Setting model. Here's the code I'm using in my Vote model to check and see if they've hit their limit.
def not_voted_too_much?
#votes_per_period = find_settings.votes_per_period #how many votes are allowed per period
#votes = Vote.find_all_by_user_id(user_id, :order => 'id DESC')
#index = #votes_per_period - 1
if #votes.nil?
true
else
if #votes.size < #votes_per_period
true
else
if #votes[#index].created_at + find_settings.voting_period_in_hours.hours > Time.now.utc
false
else
true
end
end
end
end
When that returns, true -- they're allowed to vote. If false -- they can't. Right now, it relies on the records being retrieved in a certain order and that the one it selects is the oldest. This seems to work, but feels fragile to me.
I'd like to use :order => 'created_at DESC', but when I apply a limit to the query (I'd need to only get as many records as votes are allowed for that period), it seems to always pull the oldest records instead of the latest records and I'm not sure how to go about changing the query to pull the latest votes and not the oldest.
Any thoughts on the best way to go about this?
Can't you just count the user's votes which are newer than 24 hours old and check it against your limits? Am I missing something?
def not_voted_too_much?
votes_count = votes.where("created_at >= ?", 24.hours.ago).count
votes_count < find_settings.votes_per_period
end
(this is assuming you've got the votes association setup correctly in the user model)