Reinitialize learning rate when training from checkpoint in Tensorflow - tensorflow

Question

What is the correct way to load a Tensorflow model from a checkpoint but reset the learning_rate to start from the start instead of continuing from the checkpoint?
I am using the following code for initializing the optimizer.
learning_rate = tf.train.polynomial_decay(start_learning_rate, self.global_step,
decay_steps, end_learning_rate,
power=power,name="new_one2")
opt = tf.train.AdamOptimizer(learning_rate)
I tried to restore var_list from the checkpoint and return only those variables whose name does no have Adam so that the AdamOptimizer is reinitialized from the start. But this doesn't help. Here's the function I use.
def optimistic_restore(self, save_file, ignore_vars=None, verbose=False, ignore_incompatible_shapes=False):
"""This function tries to restore all variables in the save file.
This function ignores variables that do not exist or have incompatible shape.
Raises TypeError if the there is a type mismatch for compatible shapes.
session: tf.Session
The tf session
save_file: str
Path to the checkpoint without the .index, .meta or .data extensions.
ignore_vars: list, tuple or set of str
These variables will be ignored.
verbose: bool
If True prints which variables will be restored
ignore_incompatible_shapes: bool
If True ignores variables with incompatible shapes.
If False raises a runtime error f shapes are incompatible.
"""
def vprint(*args, **kwargs):
if verbose: print(*args, flush=True, **kwargs)
# def dbg(*args, **kwargs): print(*args, flush=True, **kwargs)
def dbg(*args, **kwargs): pass
if ignore_vars is None:
ignore_vars = []
reader = tf.train.NewCheckpointReader(save_file)
var_to_shape_map = reader.get_variable_to_shape_map()
var_list = []
for key in sorted(var_to_shape_map):
if not 'Adam' in key:
var_list.append(key)
return var_list
This will give me all the variables except the Adam ones. And than I pass the resulting list of variables to the saver, as mentioned below:
varsss = self.optimistic_restore(tf.train.latest_checkpoint(my_ckpt_dir))
b = tf.global_variables()
tf.train.Saver(var_list=b[:len(varsss)])
But this still doesn't help - my learning rate doesn't go back to start.

Answer 1

If someone else is also going through the problem.
The solution was rather simple. I just added the global_step to my polynomial_decay learning rate as a placeholder. i.e
custom_lr = tf.placeholder(tf.int32)
learning_rate = tf.train.polynomial_decay(start_learning_rate, custom_lr,
decay_steps, end_learning_rate,
power=power,name="new_one2")
And simply pass the learning_rate in the feed dict.
for step in range(0, 1000):
sess.run(feed_dict={ custom_lr: step }