Inserting Nested Object Efficiently - sql

Question

If I have the following schema:
class Sinbad < ActiveRecord::Base
has_many :tinbads
accepts_nested_attributes_for: :tinbads
end
class Tinbad < ActiveRecord::Base
belongs_to :sinbad
has_many :pinbads
accepts_nested_attributes_for: :pinbads
end
class Pinbad < ActiveRecord::Base
belongs_to :tinbad
end
and it's not uncommon for a Tinbad to have a few hundred Pinbads, is there a common way to create a nested Sinbad without invoking hundreds of queries?
I've come to the sad understanding that Active Record doesn't support batch inserts but is there a way around this that doesn't involve handwritten SQL? I've looked at https://github.com/zdennis/activerecord-import, but it doesn't support nested objects. Currently, the SinbadController#create action averages >400 insert transactions and it's the most common action used.
Here's an example of what I want not to happen:
https://gist.github.com/adamkuipers/12578343d31a651bee4a
Instead of inserting into the photos table N times, I want to insert only once.

Answer 1

I'm having the exact same problem. I'm parsing big spreadsheets and the schema used to store the data is nested, so I'm inserting only a single "Sinbad" but thousands of "Pinbad" can get inserted at once...
What I came up to speed up the insert is to bulk insert the bottom leaves of the schema (visualize the schema as a tree), as this must be the model with the highest amount of instances to create - in your case, the Pinbad instances. We cannot bulk insert middle leaves as bulk insert doesn't allow to fetch the ids of the multiplie models inserted (see discussion here concerning postgresql for instance). So it's not ideal, but that's the only way I found to make the inserts more efficient (without changing the schema itself).
You'll have to remove accepts_nested_attributes_for as you need to save the objects yourself, and it's convenient to use activerecord-import for the bulk insert:
class Sinbad
#
# Let's imagine you still receive the params as if you were using accepts_nested_attributes,
# Meaning :pinbads_attributes will be nested under :tinbads_attributes
# that will be nested under :sinbad
#
def self.efficient_create params
# I think AR doesn't like when attributes doesn't exist,
# so we should keep the tinbads attributes somewhere else
tinbads_attributes = params[:tinbads_attributes]
params.delete :tinbads_attributes
sinbad = self.create! params
# Array that will contain the attributes of the pinbads to bulk insert
pinbads_to_save = []
# ActiveRecords-Import needs to know which cols of Pinbad you insert
pinbads_cols = [:tinbad_id, :name, :other]
# We need to manually save the tinbads one by one,
# but that's what happen when using accepts_nested_attributes_for
tinbads_attributes.each do |attrs|
pinbads_attribute = attrs[:pinbads_attributes]
attrs.delete :pinbads_attibutes
tinbad = sinbad.tinbads.create! attrs
pinbads_attributes.each do |p_attrs|
# Take care to put the attributes
# in the same order than the pinbad_cols array
pinbads_to_save << [tinbad.id, p_attrs[:name], p_attrs[:other]]
end
end
# Now we can bulk insert the pinbads, using activerecord-import
Pinbad.import_without_validations_or_callbacks pinbad_cols, pinbads_to_save
end
end
That's what I've done in my situation and as the last level in the schema hierarchy has the most instances to create, the overall insert time was greatly reduced. In your case, you would replace the ~400 inserts of Pinbad with 1 bulk insert.
Hope that helps, and I'm open to any suggestion or alternative solution!