Silverlight 4 WCF The server did not provide a meaningful reply - wcf

Question

I'm getting the notorious, "The server did not provide a meaningful reply; this might be caused by a contract mismatch, a premature session shutdown or an internal server error" in my project. It's a WCF PollingDuplex Service, consumed by a Silverlight 4 project.
I'm requesting a document with the service, so I can display it in a viewer in my SL application.
Here is the Server Web Config XML:
<system.serviceModel>
<extensions>
<bindingExtensions>
<add name="pollingDuplex" type="System.ServiceModel.Configuration.PollingDuplexHttpBindingCollectionElement,System.ServiceModel.PollingDuplex, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
</bindingExtensions>
</extensions>
<serviceHostingEnvironment aspNetCompatibilityEnabled="true"
multipleSiteBindingsEnabled="true" />
<behaviors>
<serviceBehaviors>
<behavior name="PortalOnlineBehavior">
<serviceMetadata httpGetEnabled="true" />
<serviceDebug includeExceptionDetailInFaults="true" />
<dataContractSerializer maxItemsInObjectGraph="2147483647"/>
</behavior>
</serviceBehaviors>
</behaviors>
<bindings>
<pollingDuplex>
<binding name="SLPollingDuplex" duplexMode="MultipleMessagesPerPoll" />
</pollingDuplex>
</bindings>
<services>
<service name="Online.Web.PortalOnline" behaviorConfiguration="PortalOnlineBehavior">
<endpoint address="" binding="pollingDuplex" bindingConfiguration="SLPollingDuplex"
contract="Notification.IPortalOnline" />
<endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange" />
<host>
<baseAddresses>
<add baseAddress="http://portalonline.com/PortalOnline/IPortalOnline" />
</baseAddresses>
</host>
</service>
</services>
</system.serviceModel>
Here is the Object I'm trying to return to SL via this WCF service
Imports System.ServiceModel
Imports System.Runtime.Serialization
Imports DocCave.Common
Imports System.Xml
Imports System.IO
<DataContract(NAMESPACE:=DataStore.NAMESPACE)>
Public Class PortalDocument
<DataMember()>
Public Property DataSource As Byte()
<DataMember()>
Public Property FileName As String
<DataMember()>
Public Property FileType As String
End Class
Here's the WCF Method that is being called:
Public Function GetDocument(sessionUserMeta As Common.UserMetaData, docId As System.Guid) As Notification.PortalDocument Implements Notification.IPortalOnline.GetDocument
Dim doc As Documents.Document = Documents.Document.GetDocument(docId, sessionUserMeta)
Dim portalDoc As New PortalDocument
portalDoc.DataSource = doc.DataSource
portalDoc.FileName = doc.QueryPackage.DocumentName
portalDoc.FileType = doc.QueryPackage.Type
Return portalDoc
End Function
Further Details:
This works perfectly for one or two document request, and the gives me the above mentioned error. For instance, I can load a default document when the SL application is loaded using this method with this service, and it populates perfectly. I can then go to a tree view I have, and select a document, and it works perfect for the first document... but after that, error. Also, I've noticed sometimes it will only work once, if I select certain pdfs that are a bit larger (250kb or so..) ... oh, and I forgot... here's the code in my SL application that is connecting to the WCF service. I'm using the "GetBaseWebAddress()" because I'm using dynamic sub domains, so part of the address can be different each time...
Private Sub LoadClient()
Dim bind As New PollingDuplexHttpBinding(PollingDuplexMode.MultipleMessagesPerPoll)
Dim endpoint As New EndpointAddress(GetBaseWebAddress() & "PortalOnline/PortalOnline.svc")
Me.client = New PortalOnline.PortalOnlineClient(bind, endpoint)
AddHandlers()
End Sub
I've struggled with this for a while, so any help would be greatly appreciated...