UDA generating error, insufficient buffer size - sql

Question

I have a UDA in SQL 2005 that keeps generating the below error. I'm guessing this is most likely due to the limitations of the max byte size of 8000....Is there any work around I can use to get around this? Any suggestions for avoiding this limitation in 2005? I know 2008 supposedly took these limitations off but I cannot upgrade for the time being.
A .NET Framework error occurred during execution of user-defined routine or aggregate "CommaListConcatenate":
System.Data.SqlTypes.SqlTypeException: The buffer is insufficient. Read or write operation failed.
System.Data.SqlTypes.SqlTypeException:
at System.Data.SqlTypes.SqlBytes.Write(Int64 offset, Byte[] buffer, Int32 offsetInBuffer, Int32 count)
at System.Data.SqlTypes.StreamOnSqlBytes.Write(Byte[] buffer, Int32 offset, Int32 count)
at System.IO.BinaryWriter.Write(String value)
at TASQLCLR.CommaListConcatenate.Write(BinaryWriter w)

Answer 1

For SQL 2005 you can solve the 8000 byte limit by turning one parameter into multiple parameters by using a delimiter. I didn't need to go into the details myself, but you can find the answer here: http://www.mssqltips.com/tip.asp?tip=2043
For SQL 2008 you need to pass MaxByteSize as -1. If you try and pass a number greater than 8000, SQL will not let you create the aggregate, complaining that there is an 8000 byte limit. If you pass in -1, it seems to get around this issue and lets you create the aggregate (which I have also tested with > 8000 bytes).
The error is:
The size (100000) for
"Class.Concatenate" is not in the
valid range. Size must be -1 or a
number between 1 and 8000.
Here is the working class definition for VB.NET to support > 8000 bytes in SQL 2008.
<Serializable(), SqlUserDefinedAggregate(Format.UserDefined,
IsInvariantToNulls:=True,
IsInvariantToDuplicates:=False,
IsInvariantToOrder:=False, MaxByteSize:=-1)>
<System.Runtime.InteropServices.ComVisible(False)> _
Public Class Concatenate Implements IBinarySerialize
End Class

Answer 2

The code below shows how to calculate the media of a set of decimal numbers within an SQLAggregate. It resolves the issue of size parameter limitation implementing a data dictionary. The idea is taken from Expert SQL Express 2005.
using System;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Linq.Expressions;
using SafeDictionary;
[Microsoft.SqlServer.Server.SqlUserDefinedAggregate(
Format.UserDefined, MaxByteSize=16)]
public struct CMedian2 : IBinarySerialize
{
readonly static SafeDictionary<Guid , List<String>> theLists = new SafeDictionary<Guid , List<String>>();
private List<String> theStrings;
//Make sure to use SqlChars if you use
//VS deployment!
public SqlString Terminate()
{
List<Decimal> ld = new List<Decimal>();
foreach(String s in theStrings){
ld.Add(Convert.ToDecimal(s));
}
Decimal median;
Decimal tmp;
int halfIndex;
int numberCount;
ld.Sort();
Decimal[] allvalues = ld.ToArray();
numberCount = allvalues.Count();
if ((numberCount % 2) == 0)
{
halfIndex = (numberCount) / 2;
tmp = Decimal.Add(allvalues[halfIndex-1], allvalues[halfIndex]);
median = Decimal.Divide(tmp,2);
}
else
{
halfIndex = (numberCount + 1) / 2;
median = allvalues[halfIndex - 1];
tmp = 1;
}
return new SqlString(Convert.ToString(median));
}
public void Init()
{
theStrings = new List<String>();
}
public void Accumulate(SqlString Value)
{
if (!(Value.IsNull))
theStrings.Add(Value.Value);
}
public void Merge(CMedian2 Group)
{
foreach (String theString in Group.theStrings)
this.theStrings.Add(theString);
}
public void Write(System.IO.BinaryWriter w)
{
Guid g = Guid.NewGuid();
try
{
//Add the local collection to the static dictionary
theLists.Add(g, this.theStrings);
//Persist the GUID
w.Write(g.ToByteArray());
}
catch
{
//Try to clean up in case of exception
if (theLists.ContainsKey(g))
theLists.Remove(g);
}
}
public void Read(System.IO.BinaryReader r)
{
//Get the GUID from the stream
Guid g = new Guid(r.ReadBytes(16));
try
{
//Grab the collection of strings
this.theStrings = theLists[g];
}
finally
{
//Clean up
theLists.Remove(g);
}
}
}
you also need to implement the dictionary as Expert 2005 does:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
namespace SafeDictionary
{
public class SafeDictionary<K, V>
{
private readonly Dictionary<K, V> dict = new Dictionary<K,V>();
private readonly ReaderWriterLock theLock = new ReaderWriterLock();
public void Add(K key, V value)
{
theLock.AcquireWriterLock(2000);
try
{
dict.Add(key, value);
}
finally
{
theLock.ReleaseLock();
}
}
public V this[K key]
{
get
{
theLock.AcquireReaderLock(2000);
try
{
return (this.dict[key]);
}
finally
{
theLock.ReleaseLock();
}
}
set
{
theLock.AcquireWriterLock(2000);
try
{
dict[key] = value;
}
finally
{
theLock.ReleaseLock();
}
}
}
public bool Remove(K key)
{
theLock.AcquireWriterLock(2000);
try
{
return (dict.Remove(key));
}
finally
{
theLock.ReleaseLock();
}
}
public bool ContainsKey(K key)
{
theLock.AcquireReaderLock(2000);
try
{
return (dict.ContainsKey(key));
}
finally
{
theLock.ReleaseLock();
}
}
}
}
The dictionary must be deployed in a separate assembly with unsafe code grants. The idea is to avoid to serializate all the numbers by keeping in memory data structure dictionary. I recommend the Expert SQL 2005 chapter:
CHAPTER 6 ■ SQLCLR: ARCHITECTURE AND
DESIGN CONSIDERATIONS.
By the way, this solution didn't work for me. Too many conversions from Decimal to String and viceversa make it slow when working with millions of rows, anyway, I enjoyed to try it. For other usages it is a good pattern.