Serialized neural network size in Torch after splitting layers - serialization

Question

After training a neural network, let's say a Multi-Layer Perceptron, at prediction time I want to split the first layer from all the others.
To do so, the only way I found in order to have files of the correct size, is the following:
I loop through all the layers and I add them to one of two containers (first layer or all the others) that I'll save separetelly using torch.save function. The funny bit is that I need to retrieve the parameters of each layer before adding them to any of the two containers, otherwise when saved, both files (first layer and all the other layers) have the same file size.
A code snippet will be more helpfull than all my previous explanation:
local function split_model(network)
-- for some reason all the models when saved have the same size
-- if not splitted calling 'getParameters()' first.
first_layer = nn.Sequential()
all_the_rest = nn.Sequential()
for i = 1, network:size() do
local l = network:get(i)
local l_params, _ = l:getParameters()
if i == 1 then
first_layer:add(l)
else
all_the_rest:add(l)
end
end
return first_layer, all_the_rest
end
local first_layer, all_the_rest = split_model(network)
torch.save("checkpoints/mlp.t7", .network)
torch.save("checkpoints/first_layer.t7", first_layer)
torch.save("checkpoints/all_the_rest.t7", all_the_rest)

Answer 1

Same question posted in Google groups, this is the answer by Alban Desmaison:
Hi,
The reason for this behavior is due to the way getParameters work
https://github.com/torch/nn/blob/master/doc/module.md#flatparameters-flatgradparameters-getparameters
To be able to return a flat tensor containing all the parameters, it
actually creates a single Storage containing all the weights and then
each module's weight is part of this storage. When you save the
weights of any element of the network, it will have to save the weight
tensor and to do so, save the underlying storage. Hence, if you called
getParameters on the complete network, if you save any module, you
will save all the networks weights. Here, when you call
getParameters on the single module, it actually re-create this
single storage but for this single module, thus, when you save it, it
contains only the weights that you want. BUT note that the flattened
parameters returned by the getParameters that you did on the
complete network are not valid anymore!!!
You have two solutions here:
- If you don't want to use the params coming from the getParameters on the complete network, you can just call
getParameters on each subset of your network before saving them.
This will break change the underlying storage to only contain this
subset of the network and you will save only what you need (shared
storages are stored only once).
- If you want to be able to keep using the params from the original getParameters, you can do the same as above but use cloned version
of them to do the getParameters and saving.
And because a code snippet is always better:
require 'nn'
local subset1 = nn.Linear(2,2)
local subset2 = nn.Linear(2,2)
local network = nn.Sequential():add(subset1):add(subset2)
print("Before getParameters:", subset1.weight:storage():size()) -- 4 elements
network_params,_ = network:getParameters()
print("After getParameters:", subset1.weight:storage():size()) -- 12 elements
subset1.weight:random() -- Change weights to see if linking is still working
print("network_params is valid?", network_params[1] == subset1.weight[1][1]) -- true
-- Keeping network_params valid
local clone_subset1 = subset1:clone()
print("Cloned subset1 before getParameters:", clone_subset1.weight:storage():size()) -- 12 elements
clone_subset1:getParameters()
print("Cloned subset1 after getParameters:", clone_subset1.weight:storage():size()) -- 6 elements (4 weights + 2 bias)
subset1.weight:random() -- Change weights to see if linking is still working
print("network_params is valid?", network_params[1] == subset1.weight[1][1]) -- true
-- Not keeping network_params valid (should be faster)
local clone_subset1 = subset1:clone()
print("subset1 before getParameters:", subset1.weight:storage():size()) -- 12 elements
subset1:getParameters()
print("subset1 after getParameters:", subset1.weight:storage():size()) -- 6 elements (4 weights + 2 bias)
subset1.weight:random() -- Change weights to see if linking is still working
print("network_params is valid?", network_params[1] == subset1.weight[1][1]) -- false