Im current working on a small project and I want to seed my database faster. I migrated a new column called "grand_total_points" to my table of users. So originally I was using this code.
user = User.all
user.each do |x|
x.grand_total_points = x.total_points
x.save!
end
This takes me ages, because I have to update a million record.
Total_points have already been defined in my user model where it calculates all the users points that have been submitted. Forgive me for my explanation. Is there a way to use update_all method but with each included in it?
Yep, possible:
User.update_all('grand_total_points = total_points')
It will generate the following SQL query:
UPDATE "users" SET "grand_total_points" = 'total_points'
If total_points is not a column but an instance method, move the logic into update_all query.
User.update_all("grand_total_points = #{total_points calculation translated into SQL terms}")
I found something that could work. So basically i combine a ruby code with an execute SQL statement, and I put it in a migration file. Here's how the code works. I hope this helps. Make sure you follow the query according to your data.
class ChangeStuff < ActiveRecord::Migration
def change
points = Point.select('user_id, SUM(value) AS value').group(:user_id)
points.each do |point|
execute "UPDATE users SET grand_total_points = #{point.value} WHERE users.id = #{point.user_id}"
end
end
end
You should run bundle exec rake db:migrate after that. The normal way takes me 2-3hours. This only took me 2minutes.
Related
I have a query that goes something like this (in song.rb):
def self.new_songs
Song.where(id: Song.grouped_order_published).select_important_stuff
end
Later on in my app, it is then passed the limit and offset, lets say in the controller:
#songs = Song.new_songs.limit(10).offset(10)
The way my app is structured, I'd like to keep this method of setting things, but unfortunately it is really slow as it is limiting the outer query rather than the subquery.
Is there a way I can expose the subquery such that it receives the limit and offset rather than the outer query?
Edit: I should add I am using postgres 9.2.
Edit 2: The reason why I want to do it in this fashion is I am doing pagination and I need to get the "count" of the total number of rows. So I do something like this:
#songs = Song.new_songs
...
#pages = #songs.count / 10
...
render #songs.limit(params[:page]).offset(0)
If I were to change it somehow, I'd have to redo this entirely (which is in a ton of places). By not limiting it until it's actually called, I can do the count in between and then get just the page at the end. I guess I'm looking more for advice on how this can be done with the inner query, without becoming horribly slow as the database grows.
I could not try the solution and I am not a ruby expert either, but as far as I understand the problem you would need an object that passes all method-calls but limit and offset onto the full query and store the limited sub_query in the meantime.
It could probably look like this:
class LimitedSubquery < Object
# sub_query has to be stored so we can limit/offset it
def initialize(sub_query)
#sub_query = sub_query
end
# Make sure everybody knows we can be used like a query
def self.respond_to?(symbol, include_private=false)
super || full_query.respond_to?(symbol, include_private)
end
# Missing methods are probably meant to be called on the whole query
def self.method_missing(method_sym, *arguments, &block)
if full_query.respond_to?(method_sym)
full_query.send(method_sym, *arguments, &block)
else
super
end
end
# Generate the query for execution
def self.full_query
Song.where(id: #sub_query).select_important_stuff
end
# Apply limit to sub_query
def self.limit(*number)
LimitedSubquery.new(#sub_query.limit(*number))
end
# Apply offset to sub_query
def self.offset(*number)
LimitedSubquery.new(#sub_query.offset(*number))
end
end
And than call it like
def new_songs
LimitedSubquery.new(Song.grouped_order_published)
end
Please edit me if I got something wrong!
Regards
TC
You should consider using the will_paginate gem. This keeps you away form the hazzle to calculate all this by hand ;-)
I'm having a strange problem in Rails with a Postgres query.
The query looks something like this:
WeeklyPlanner.find_or_create_by_user_id(current_user.id).recipes.find(:all,
:conditions => ["
weekly_planner_events.time_start =
date_part('epoch', to_timestamp(?)::timestamptz at time zone 'CDT')",
Time.local(Time.now.year, Time.now.month, Time.now.day).to_i
])
This generates (as I can view in the console), the following SQL statement:
SELECT "recipes".* FROM "recipes"
INNER JOIN "weekly_planner_events" ON "recipes"."id" = "weekly_planner_events"."recipe_id"
WHERE "weekly_planner_events"."weekly_planner_id" = 2
AND (weekly_planner_events.time_start = date_part('epoch', to_timestamp(1347426000)::timestamptz at time zone 'CDT'))
My problem is that the generated SQL statement works well on psql or pgAdmin, but on rails it returns an empty array. That is, if I copy and paste it as is on a postgres console, it works perfectly fine, but when I run it on the Rails console, it returns nothing, and I have no idea why its happening.
I've tried the following:
Parametrizing 'epoch' and 'CDT'/timezone (in order to remove 's
Switching to a where statement, with the same condition
Passing the variables with #{}s
Doing the search without the date_part('epoch', [float]) function works fine in Rails, but its obviously not the result I need.
I'm finding this issue quite confusing, if there is any other data you need please let me know and I will edit the post.
Thank you.
Maybe when you are using the find_or_create method it is using the create action, so you create a new WeeklyPlanner for a user, and this brand new WeeklyPlanner doesn't have recipes attached to it because it has just been created.
When you go to psql, you probably use a existent WeeklyPlanner.
But this is just a guess.
I am trying to use the Arel#extract method. I have seen an example in a test case in test_extract.rb in the source but when I try to reproduce it in my app, I get undefined method.
table = Arel::Table.new :users
puts table[:created_at].extract('hour').to_sql
=> NoMethodError: undefined method `extract' for #<Arel::Attributes::Attribute:0x7..8>
I am using pg as the database.
Update:
My goal is to end up with this result in sql:
SELECT users.* FROM users WHERE EXTRACT(HOUR from users."created_at") = '1'
I would like to find all users that were created on the hour equal to one of any day. This works in sql but I am wondering how to create it in arel. Here is an example of how it's used in the arel test suite.
extract is node's method, you can cast it on any column(such as users[:id]), but not on Arel::Table instance.
So, to construct your query you need:
get Arel::Table instance users = Arel::Table.new(:users) or if you use ActiveRecord - users = User.arel_table
set SELECT statement on Arel::Table instance with project method: users = users.project(Arel.sql('*'))
set WHERE statement with where method: users.where(users[:created_at].extract(:hour).eq(1))
In one block:
query = User.arel_table.
project(Arel.sql('*')).
where(users[:created_at].extract(:hour).eq(1))
users = User.find_by_sql(query)
# => "SELECT * FROM \"users\" WHERE EXTRACT(HOUR FROM \"users\".\"created_at\") = 1"
I've had to perform an extract DOW on an instance of Arel::Nodes::NamedFunction, which does not expose the method #extract (as of Arel 6.0). I managed to achieve this by manually creating an instance of Arel::Nodes::Extract. Here is what worked for me, in case anyone have a similar issue:
Arel::Nodes::Extract.new(
Arel::Nodes::NamedFunction.new('some_function_name', [param1, param2, ...]),
:dow
)
You can use an Arel node directly with ActiveRecord's #where instead of building the full query through Arel as exemplified by Alexander Karmes's answer. So here is another way to perform the query required by the answer:
User.where(
Arel::Nodes::Extract.new(User.arel_table[:created_at], :hour).eq(1)
)
Which yields:
SELECT "users".* FROM "users" WHERE EXTRACT(HOUR FROM "users"."created_at") = 1
With the added benefit you can keep chaining other scopes defined in your User model.
I have the following defined on my up method on my migration to set initial data:
def up
Color.create!({:id=>1,:name=>"",:color=>"FF6633"})
Color.create!({:id=>2,:name=>"",:color=>"93B233"})
Color.create!({:id=>3,:name=>"",:color=>"4D90D9"})
Color.create!({:id=>4,:name=>"",:color=>"C43092"})
end
Is there any truncate directive I can put on the down method like:
def down
Color.truncate
end
Or since I'm setting the IDs on the create should I use only the destroy_all method of the model Color ?
You can simple use this in your up method, this will solve your both truncate and id resetting problem also.
def up
ActiveRecord::Base.connection.execute("TRUNCATE table_name")
Color.create!({:id=>1,:name=>"",:color=>"FF6633"})
Color.create!({:id=>2,:name=>"",:color=>"93B233"})
Color.create!({:id=>3,:name=>"",:color=>"4D90D9"})
Color.create!({:id=>4,:name=>"",:color=>"C43092"})
end
Cheers!
Firstly, you don't have to pass :id into create! because ActiveRecord will automatically handle that, thus :id likely to get ignored (standard case assumed).
Secondly, it is not a good practice to use ActiveRecord query builder in migration because should the model Color name be changed, you are to have a broken migration. I highly recommend you to use pure SQL and execute that query with execute().
Thirdly, for the #down method, you shouldn't truncate the table. You should destroy those 4 colors that you created in #up.
Here's how I would write it:
def down
colors = ["FF6633", "93B233", "4D90D9", "C43092"]
Color.where(:color => colors).destroy_all
end
I'm using RoR and I want to do concurrency safe update queries. For example when I have
var user = User.find(user_id)
user.visits += 1
I get the following SQL code:
SELECT * FROM Users WHERE ID=1 -- find user's visits (1)
UPDATE Users SET Visits=2 WHERE ID=1 -- 1+1=2
But if there are several queries taking place at the same time, there will be locking problems.
According to RoR API I can use :lock => true attribute, but that's not what I want.
I found an awesome function update_counters:
User.update_counters(my_user.id, :visits => 1)
This gives the following SQL code
UPDATE Users SET Visits=Visits+1 WHERE ID=#something
Works great!
Now my question is how can I override the += function to do the update_counters thing instead?
Because
user.visits += 1 # better
User.update_counters(my_user.id, :visits => 1) # than this
UPDATE
I just created the following function
class ActiveRecord::Base
def inc(column, value)
User.update_counters(self.id, column => value)
end
end
Are there any other better ideas?
Don't know about better ideas, but I would define that method to the User class, not the ActiveRecord. And maybe increment_counter (that uses update_counters) would make it a bit more readable?
def inc(column, value)
self.increment_counter(column, value)
end
Haven't tested that and not saying this is definitely better idea, but that's probably how I'd do it.
Update:
And AFAIK you can't override the "+=", because "a += b" just a shortcut for "a = a + b" and you probably don't want to override "=" to use the update_counters :)