Memory leak in XmlSerializer as used in VB.net - vb.net

Question

I seem to have a memory leak when using xmlSerializer in VB.net 2010.
The code below is the guts of a function which is called every 20 seconds or so forever.
Uncommenting the line 'serializer.Serialize(writer, response)' sees the memory usage climb.
Using writer As New StreamWriter(fileName)
Dim request = invoker.CreateRequest()
Dim response = invoker.Invoke(authorization, request)
If Not response Is Nothing Then 'maybe token timeout
Dim serializer = New XmlSerializer(GetType(TResponse)) ', New XmlRootAttribute("TResponse"))
Try
' serializer.Serialize(writer, response)
Catch e As InvalidOperationException
Dim msg As String = "ERROR: Failed to serialize the response:" & vbCrLf & e.Message
AddToRtfEventLog(category.ToUpper & " DATA " & msg, Drawing.Color.Red)
End Try
serializer = Nothing
AddToStatus(Convert.ToString("Response saved as ") & fileName & vbCrLf, Drawing.Color.LightGray, True)
End If
Return response
End Using
From internet reading, I gather this is a bug in the VB xmlSerializer and a possible solution is to use it in a caching manner. Unfortunately I have no idea how to do that so I wonder if someone can help me out here.
Firstly, am I doing it right in the first place (it works fine, just the memory issue) and
Secondly, if I am, can you show me how to fix the leak please?
Thanks
Malcom
Part 2-
With help from Jehof, I've now got the following cache class in place:
Public NotInheritable Class XmlSerializerCache
Private Sub New()
End Sub
Private Shared ReadOnly cache As New Dictionary(Of String, XmlSerializer)()
Public Shared Function Create(type As Type, root As XmlRootAttribute) As XmlSerializer
Dim key = [String].Format(CultureInfo.InvariantCulture, "{0}:{1}", type, root.ElementName)
If Not cache.ContainsKey(key) Then
cache.Add(key, New XmlSerializer(type, root))
End If
Return cache(key)
End Function
End Class
and have replaced the calls with :
Dim xmlRootAttribute = New XmlRootAttribute("TResponse")
Dim serializer = XmlSerializerCache.Create(GetType(TResponse), XmlRootAttribute)
Try
serializer.Serialize(writer, response)
Thats has improved the leak but is the correct way to do this? To me it seems that I am recreating the cache with each call?