I am using Ruby on Rails and I have to create an importer from one database to another. There are over 100 tables in each database and I don't want to create a model for each table. Is there any possibility to create queries, specifying a table name? I don't want a model bind to a table.
For example: FirstDatabase.select('*').from('some_table').where(...)
Without set_table_name ... just dynamic models
Also I need to do inserts in different tables
I would not use models for that task at all. Instead, use the #select_all, #exec_insert, #exec_update, and #exec_delete methods of the base connection as appropriate.
ActiveRecord::Base.connection.select_all("select * from #{table_name}")
Returns an array of hashes for rows.
ActiveRecord::Base.connection.exec_insert("insert into #{table} (foo, bar) values(#{foo}, #{bar})
Inserts a row. The values will need to be properly escaped strings, dates, etc. for whatever database you are using.
ActiveRecord::Base.connection.quote("fo'o")
=> "'fo''o'" # When server is PostgreSQL
Returns a quoted string representation suitable for use in a SQL statement.
now=Time.now
=> Fri Aug 23 02:24:40 -0700 2013
ActiveRecord::Base.connection.quote(now)
=> "'2013-08-23 09:24:40.365843'"
Returns a quoted date/time representation in UTC timezone.
To deal with the multiple databases, you can do something like set up a single model for each, and then get connections from those models instead of from ActiveRecord::Base.
class TableInDbA << ActiveRecord::Base
establish_connection "database_a_#{Rails.env}"
end
class TableInDbB << ActiveRecord::Base
establish_connection "database_b_#{Rails.env}"
end
TableInDbA.connection.select_all("...
Related
We have landed ourself with a column 'data' on User table that has a huge json dump in it.
Now every time we load a bunch of users we get all this data into the memory resulting into out of memory errors.
We want to be able to write some computed columns that we can use in our select statements.
eg:
Instead of doing this
user.data['profile_data']['data']['image']
We would like to add a column :image and then write a query like:
Here :name and :email are actual columns on the table and :image is a computed column:
Users.where(SOME_CONDITION).select(:name,:email,:image)
The main use case is the index page where we display all users which basically loads data column for all users
This will avoid loading the huge data column in memory and help us load the fields we want from the data column
Whats the best way to do this in Rails4?
Updates:
We use postgres on Heroku.
I would move the data column to another table, but if that's not an option try the lazy_columns gem.
class User < ActiveRecord::Base
lazy_load :data
end
Now the data column will be excluded during the initial load, but if you try to access the .data it will be retrieved from the database.
Add the column :image using the migration. Then add the below code :
class User < ActiveRecord::Base
before_save :extract_image
private
def extract_image
self.image = self.data['profile_data']['data']['image']
self.save
end
end
before_save: Is called before Base.save (regardless of whether it’s a create or update save).
Postgresql support two json data type, as described in the documentation
There are two JSON data types: json and jsonb. They accept almost identical sets of values as input. The major practical difference is one of efficiency. The json data type stores an exact copy of the input text, which processing functions must reparse on each execution; while jsonb data is stored in a decomposed binary format that makes it slightly slower to input due to added conversion overhead, but significantly faster to process, since no reparsing is needed. jsonb also supports indexing, which can be a significant advantage.
So to solve your problem, you need to change the type of the data column to jsonb through a migration:
# This should use the up and down methods, because change_column
# is not reversible
class ChangeUsersDataColumnTypeToJsonb < ActiveRecord::Migration
def up
change_column :users, :data, :jsonb
end
def down
change_column :users, :data, :text # or whatever datatype it was
end
end
than to query the image field with you use the functions that postgres provides to query the json data type :
Users.where(SOME_CONDITION).select(:name,:email,"data::json->'image' as image")
than you access the image attribute like any other attribute.
You have also to define the :data attribute as a lazy loading column, like , so that column do not get loaded when the user object is instantiated.
class User < ActiveRecord::Base
lazy_load :data
end
I have a typical single table inheritance case, something like:
class Instrument < ActiveRecord::Base
def self.search(params)
query = Instrument
# build up the query scope
query
end
end
class Guitar < Instrument
end
class Trumpet < Instrument
end
I need to get a single paginated list of instruments of all kinds.
#instruments = Instrument.search(params)
.page(params[:page])
.per(params[:size])
My problem is that each model in the search results is an Instrument, whereas I need them to be Guitars and Trumpets, etc or else subclass methods like #blow and #strum are missing.
Because of pagination, I can't really do multiple queries.
Surely this is a common use case that ActiveRecord supports, and I'm just missing something obvious, yes?
Have you have added a type column to the instruments table, migrated the database and restarted the server?
Any Instrument object that is loaded from the database, assuming it has a type set, should automatically be loaded as the subclass.
Are you positive the values in the type column are being stored correctly?
I believe the class name in the type column is case-sensitive.
data = Program.joins(:program_schedules, :channel).
where(
"program_schedules.start" => options[:time_range],
"programs.ptype" => "movie",
"channels.country" => options[:country]).
where("programs.official_rating >= ?", options[:min_rating]).
group("programs.id").
order("programs.popularity DESC")
This query retrieve only the "programs" table (I think because the "group by" clause).
How I could retrieve all data from all tables (programs, programs_schedules, channel) ?
class Program < ActiveRecord::Base
belongs_to :channel
has_many :program_schedules
Ruby on Rails 3.2
Postgresql 9.2
Are you looking for the eager loading of associations provided by the includes method ?
Eager loading is a way to find objects of a certain class and a number of named associations. This is one of the easiest ways of to prevent the dreaded 1+N problem in which fetching 100 posts that each need to display their author triggers 101 database queries. Through the use of eager loading, the 101 queries can be reduced to 2.
Update:
You can get the SQL as a string by appending .to_sql to your query. .explain can also help.
If you want to get the data as a hash and no more as model instances, you can serialize the result of the query and include the associations using .serializable_hash :
data.serializable_hash(include: [:program_schedules, :channel])
You can define which fields to include using :except or :only options in the same way as described here for .as_json (which acts similarly) :
http://api.rubyonrails.org/classes/ActiveModel/Serializers/JSON.html#method-i-as_json
In my Rails3 app, I am using ActiveRecord and Postgresql.
Say I have a model called Parts. The model has a small list of standard attributes such as price, quantity, etc.
However, Customer A might want to add LotNumber and CustomerB might want to add OriginalLocation.
How would I allow them to do that?
I thought about creating a PartsDetail model that allowed them to have a type.
class PartsDetail < ActiveRecord::Base
attr_accessible :type, :value, :part_id
belongs_to :parts
end
So that "type" could be "LotNumber", etc.
But I'm not quite sure how that would work in my associations and querying.
Any ideas?
Thanks.
Since you're using PostgreSQL, you could use hstore to store arbitrary hashes in database columns:
This module implements the hstore data type for storing sets of key/value pairs within a single PostgreSQL value. This can be useful in various scenarios, such as rows with many attributes that are rarely examined, or semi-structured data. Keys and values are simply text strings.
There's even a gem for adding hstore support to ActiveRecord:
https://github.com/softa/activerecord-postgres-hstore
Then you could create an hstore column called, say, client_specific and look inside it with things like:
M.where("client_specific -> 'likes' = 'pancakes'")
M.where("client_specific #> 'likes=>pancakes'")
to see which clients have noted that they like pancakes.
You might want to store a list of customer-specific fields somewhere with the customer record to make the UI side of things easier to deal with but that's pretty simple to do.
Here's a gist that someone wrote that allows you to use hstore attributes in your models more seamlessly: https://gist.github.com/2834785
To use add this in an initializer ( or create a new initializer called active_record_extensions.rb )
require "hstore_accessor"
Then in your model you can have:
Class User < ActiveRecord::Base
hstore_accessor :properties, :first_name, :last_name
end
That allows you to do:
u = User.new
u.first_name = 'frank'
You can still do add attributes to the hstore column and bypass the hstore_attributes though:
u.properties['middle_name'] = 'danger'
Is there a way to get the sql for JUST a scope? So I want to do something like:
class Presentation < ActiveRecord::Base
has_many :calls
has_many :recordings, :through => :calls
scope :with_recordings, joins(:calls).joins(:recordings)
end
And then be able to get the sql for that scope.
Presentations.with_recordings.sql returns the entire sql statement, including the SELECT statement. All I want is the sql added by the scope. Figure there ought to be a way to do this.
I agree with ctcherry about this not being very useful, but having said that, I needed to do this for a project I was working on. We needed to duplicate the sql in the scopes to allow us to reuse the sql across different types of searches. Rather that have to maintain the same sql in two different places, I choose to extract the sql from the scope.
The code below is what I came up with. It's ugly, but works under Rails 3.0
def extract_named_scope_clause(scope, args)
# where_clauses will return an array of clauses for an entire relationship.
# As this is only run a single scope, we only ever care about the first.....
clause, *bind_vars = self.send(scope, args).where_clauses.first
# prefix 'and ' to the string, add some spaces and append any bind variables
if clause
[" and #{clause} ", bind_vars]
else
nil
end
end
This wouldn't really make sense, as there is no standard way to represent SQL "fragments".
The different kinds of SQL "fragments" that can be added and manipulated by a scope don't really have a clean way to be represented by themselves without being part of a complete SQL statement. A fragment could be "JOIN users ON users.id = orders.user_id" or it could be "WHERE active = 1". How would you return these without them being part of a complete SQL statement? This is most likely why there is no mechanism to retrieve them other than the one you have already discovered that just returns the complete SQL statement.