In a Rails app I need a source of unique, sequential (no gaps) integers to use as serial numbers. It must be persistent and allow concurrent access.
Database auto-increment isn't adequate because most don't guarentee the "no gaps" property.
In straight SQL I would just create a one-line table and say (in PostgreSQL) something like:
update sequence set value = value + 1 returning value
This is apparently standard practice in the SQL world. References exist.
In ActiveRecord I easily created a model the model and found .increment! and .increment_counter in the documentation. But I can't figure out how to atomically retrieve the incremented value. Locks and transactions don't seem to help.
Since update ... returning acts like a select for output purposes, it turns out you can use find_by_sql to both update and get the updated value in one operation.
class SequenceNumber < ActiveRecord::Base
attr_accessible :tag, :value
validates :tag, :uniqueness => true
def self.get_next(tag)
find_by_sql("update sequence_numbers
set value = value + 1
where tag = '#{tag}'
returning value").first.value
end
end
The remaining problem is that this is totally non-portable because returning is a pgsql extension. Maybe the ActiveRecord developers will notice this.
If you want to use redis (and maybe you already are because of Resque or Sidekiq), you can do an INCR on a key, this is atomic and returns the new value.
Related
I have two models Issue and Label. They have a many to many relationship.
I have a method that returns the ten labels that point to the most issues.
class Label < ApplicationRecord
has_many :tags
has_many :issues, through: :tags
def self.top
Label.joins(:issues)
.group(:name)
.order('count_id desc')
.count(:id)
.take(10)
end
end
It does exactly what I expect it to but I want to know if it's possible to compose the query without the SQL string.
order('count_id DESC') is confusing me. Where does count_id come from? There isn’t a column named count_id.
Label.joins(:issues).group(:name).column_names
#=> ["id", "name", "created_at", "updated_at"]
I’ve found some SQL examples here. I think it’s basically the same as ORDER BY COUNT(Id):
SELECT COUNT(Id), Country
FROM Customer
GROUP BY Country
ORDER BY COUNT(Id) DESC
Is it possible to perform the same query without passing in the SQL string? Can it be done with the ActiveRecord querying interface alone?
If you look at your query log, you'll see something like:
select count(labels.id) as count_id ...
The combination of your group call (with any argument) and the count(:id) call gets ActiveRecord to add the count_id column alias to the query. I don't think this is documented or specified anywhere (at least that I can find) but you can see it happen if you're brave enough to walk through the Active Record source.
In general, if you add a GROUP BY and then count(:x), Active Record will add a count_x alias. There's no column for this so you can't say order(:count_id), order(count_id: :desc), or any of the other common non-String alternatives. AFAIK, you have to use a string but you can wrap it in an Arel.sql to prevent future deprecation issues:
Label.joins(:issues)
.group(:name)
.order(Arel.sql('count_id desc'))
.count(:id)
.take(10)
There's no guarantee about this so if you use it, you should include something in your test suite to catch any problems if the behavior changes in the future.
New to rails/ruby (using rails 3 and ruby 1.9.2), and am trying to get rid of some unnecessary queries being executed.
When I'm running an each do:
apples.to_a.each do |apple|
new_apple = apple.clone
new_apple.save!
end
and I check the sql LOG, I see three select statements followed by one insert statement. The select statements seem completely unnecessary. For example, they're something like:
SELECT Fruit.* from Fruits where Fruit.ID = 5 LIMIT 1;
SELECT Color.* from Colors where Color.ID = 6 LIMIT 1;
SELECT TreeType.* from TreeTypes where TreeType.ID = 7 LIMIT 1;
INSERT into Apples (Fruit_id, color_id, treetype_id) values (6, 7, 8) RETURNING "id";
Seemingly, this wouldnt' take much time, but when I've got 70k inserts to run, I'm betting those three selects for each insert will take up a decent amount of time.
So I'm wondering the following:
Is this typical of ActiveRecord/Rails .save! method, or did the previous developer add some sort of custom code?
Would those three select statements, being executed for each item, cause a noticeable amount of extra time?
If it is built into rails/active record, would it be easily bypassed, if that would make it run more efficiently?
You must be validating your associations on save for such a thing to occur, something like this:
class Apple < ActiveRecord::Base
validates :fruit,
:presence => true
end
In order to validate that the relationship, the record must be loaded, and this needs to happen for each validation individually, for each record in turn. That's the standard behavior of save!
You could save without validations if you feel like living dangerously:
apples.to_a.each do |apple|
new_apple = apple.clone
new_apple.save(:validate => false)
end
The better approach is to manipulate the records directly in SQL by doing a mass insert if your RDBMS supports it. For instance, MySQL will let you insert thousands of rows with one INSERT call. You can usually do this by making use of the Apple.connection access layer which allows you to make arbitrary SQL calls with things like execute
I'm guessing that there is a before_save EDIT: (or a validation as suggested above) method being called that is looking up the color and type of the fruit and storing that with the rest of the attributes when the fruit is saved - in which case these lookups are necessary ...
Normally I wouldn't expect activerecord to do unnecessary lookups - though that does not mean it is always efficient ...
Is there a way to get the sql for JUST a scope? So I want to do something like:
class Presentation < ActiveRecord::Base
has_many :calls
has_many :recordings, :through => :calls
scope :with_recordings, joins(:calls).joins(:recordings)
end
And then be able to get the sql for that scope.
Presentations.with_recordings.sql returns the entire sql statement, including the SELECT statement. All I want is the sql added by the scope. Figure there ought to be a way to do this.
I agree with ctcherry about this not being very useful, but having said that, I needed to do this for a project I was working on. We needed to duplicate the sql in the scopes to allow us to reuse the sql across different types of searches. Rather that have to maintain the same sql in two different places, I choose to extract the sql from the scope.
The code below is what I came up with. It's ugly, but works under Rails 3.0
def extract_named_scope_clause(scope, args)
# where_clauses will return an array of clauses for an entire relationship.
# As this is only run a single scope, we only ever care about the first.....
clause, *bind_vars = self.send(scope, args).where_clauses.first
# prefix 'and ' to the string, add some spaces and append any bind variables
if clause
[" and #{clause} ", bind_vars]
else
nil
end
end
This wouldn't really make sense, as there is no standard way to represent SQL "fragments".
The different kinds of SQL "fragments" that can be added and manipulated by a scope don't really have a clean way to be represented by themselves without being part of a complete SQL statement. A fragment could be "JOIN users ON users.id = orders.user_id" or it could be "WHERE active = 1". How would you return these without them being part of a complete SQL statement? This is most likely why there is no mechanism to retrieve them other than the one you have already discovered that just returns the complete SQL statement.
I have found the "pure SQL" answers to this question. Is there a way, in Rails, to reset the id field for a specific table?
Why do I want to do this? Because I have tables with constantly moving data - rarely more than 100 rows, but always different. It is up to 25k now, and there's just no point in that. I intend on using a scheduler internal to the Rails app (rufus-scheduler) to run the id field reset monthly or so.
You never mentioned what DBMS you're using. If this is postgreSQL, the ActiveRecord postgres adapter has a reset_pk_sequences! method that you could use:
ActiveRecord::Base.connection.reset_pk_sequence!('table_name')
I came out with a solution based on hgimenez's answer and this other one.
Since I usually work with either Sqlite or PostgreSQL, I've only developed for those; but extending it to, say MySQL, shouldn't be too troublesome.
Put this inside lib/ and require it on an initializer:
# lib/active_record/add_reset_pk_sequence_to_base.rb
module ActiveRecord
class Base
def self.reset_pk_sequence
case ActiveRecord::Base.connection.adapter_name
when 'SQLite'
new_max = maximum(primary_key) || 0
update_seq_sql = "update sqlite_sequence set seq = #{new_max} where name = '#{table_name}';"
ActiveRecord::Base.connection.execute(update_seq_sql)
when 'PostgreSQL'
ActiveRecord::Base.connection.reset_pk_sequence!(table_name)
else
raise "Task not implemented for this DB adapter"
end
end
end
end
Usage:
Client.count # 10
Client.destroy_all
Client.reset_pk_sequence
Client.create(:name => 'Peter') # this client will have id=1
EDIT: Since the most usual case in which you will want to do this is after clearing a database table, I recommend giving a look to database_cleaner. It handles the ID resetting automatically. You can tell it to delete just selected tables like this:
DatabaseCleaner.clean_with(:truncation, :only => %w[clients employees])
I assume you don't care about the data:
def self.truncate!
connection.execute("truncate table #{quoted_table_name}")
end
Or if you do, but not too much (there is a slice of time where the data only exists in memory):
def self.truncate_preserving_data!
data = all.map(&:clone).each{|r| raise "Record would not be able to be saved" unless r.valid? }
connection.execute("truncate table #{quoted_table_name}")
data.each(&:save)
end
This will give new records, with the same attributes, but id's starting at 1.
Anything belongs_toing this table could get screwy.
Based on #hgmnz 's answer, I made this method that will set the sequence to any value you like... (Only tested with the Postgres adapter.)
# change the database sequence to force the next record to have a given id
def set_next_id table_name, next_id
connection = ActiveRecord::Base.connection
def connection.set_next_id table, next_id
pk, sequence = pk_and_sequence_for(table)
quoted_sequence = quote_table_name(sequence)
select_value <<-end_sql, 'SCHEMA'
SELECT setval('#{quoted_sequence}', #{next_id}, false)
end_sql
end
connection.set_next_id(table_name, next_id)
end
One problem is that these kinds of fields are implemented differently for different databases- sequences, auto-increments, etc.
You can always drop and re-add the table.
No there is no such thing in Rails. If you need a nice ids to show the users then store them in a separate table and reuse them.
You could only do this in rails if the _ids are being set by rails. As long as the _ids are being set by your database, you won't be able to control them without using SQL.
Side note: I guess using rails to regularly call a SQL procedure that resets or drops and recreates a sequence wouldn't be a purely SQL solution, but I don't think that is what you're asking...
EDIT:
Disclaimer: I don't know much about rails.
From the SQL perspective, if you have a table with columns id first_name last_name and you usually insert into table (first_name, last_name) values ('bob', 'smith') you can just change your queries to insert into table (id, first_name, last_name) values ([variable set by rails], 'bob', 'smith') This way, the _id is set by a variable, instead of being automatically set by SQL. At that point, rails has entire control over what the _ids are (although if it is a PK you need to make sure you don't use the same value while it's still in there).
If you are going to leave the assignment up to the database, you have to have rails run (on whatever time schedule) something like:
DROP SEQUENCE MY_SEQ;
CREATE SEQUENCE MY_SEQ START WITH 1 INCREMENT BY 1 MINVALUE 1;
to whatever sequence controls the ids for your table. This will get rid of the current sequence, and create a new one. This is the simplest way I know of you 'reset' a sequence.
Rails way for e.g. MySQL, but with lost all data in table users:
ActiveRecord::Base.connection.execute('TRUNCATE TABLE users;')
Maybe helps someone ;)
There are CounterCache methods:
https://www.rubydoc.info/docs/rails/4.1.7/ActiveRecord/CounterCache/ClassMethods
I used Article.reset_counters Article.all.length - 1 and it seemed to work.
I am looking at using Ruby on Rails for a storefront that has to make use of existing data, but I can create my own database schema if I need to. A lot of the fields in the existing data are char(1) emulating a boolean field (i.e. Y/N) for, I believe, cross-platform portability. Since the data is prone to change, I don't want to have to change the existing structure and convert these fields to bit/boolean fields.
If I do use Rails I really would like to make use of Ruby's beautiful boolean syntax and say something like <%= image_tag 'recycled.jpg' if product.recycled? %>, but will Rails recognize char(1) as a boolean, or do I have to define those methods myself in the model like:
class Product < ActiveRecord::Base
# ... other stuff here
def recycled?
self.recycled == 'Y'
end
end
I'm thinking I will have to redefine them myself, which is no big deal, I just want to make sure since using char(1) as yes/no values isn't something I've used in the past.
As far as I know, what you describe is not possible with ActiveRecord out-of-the-box.
However, if you have a lot of columns like this you could look at doing a little bit of meta-programming to provide a declarative way to add the relevant accessor logic. Something like :-
class Product < ActiveRecord::Base
yes_no_accessor :recycled
end
Another possibility is to monkey-patch ActiveRecord. I think the relevant method is ActiveRecord::ConnectionAdapters::Column.value_to_boolean(value). You could try overriding the ActiveRecord::ConnectionAdapters::Column::TRUE_VALUES constant to include 'Y'. I haven't actually tried this!
I'd probably attack it at the model level - when you load a row into a model instance, compute a boolean attribute based on the char. Add a getter for the virtual attribute that returns this value, and a setter that updates both the boolean and the underlying char.
Can't you just wrap it in your database engine with a view or stored procedure to produce a consistent interface to your application?