I use the RedisConnection Set method to set the byte array but how do i get the data? The get returns a wrapped byte array?
Links:
http://code.google.com/p/booksleeve/
http://code.google.com/p/protobuf-net/
This works but it doesn't feel right:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using BookSleeve;
using ProtoBuf;
using System.IO;
namespace RedisTest001
{
[ProtoContract, Serializable]
public class Price
{
private string _ticker;
private double _value;
public Price()
{
}
public Price(string ticker, double value)
{
_ticker = ticker;
_value = value;
}
[ProtoMember(1)]
public string Ticker
{
get { return _ticker; }
set { _ticker = value; }
}
[ProtoMember(2)]
public double Value
{
get { return _value; }
set { _value = value; }
}
}
class Program
{
static void Main(string[] args)
{
using (var conn = new RedisConnection("localhost"))
{
Price p = new Price("IBM", 101.55);
byte[] raw;
using (MemoryStream ms = new MemoryStream())
{
Serializer.Serialize<Price>(ms,p);
raw = ms.ToArray();
}
conn.Open();
conn.Set(1, p.Ticker, raw);
var tb = conn.Get(1,"IBM");
var str = conn.Wait(tb);
Price p2 = Serializer.Deserialize<Price>(new MemoryStream(str));
}
}
}
}
More info:
public static class pex
{
public static byte[] ToBArray<T>(this T o)
{
using (MemoryStream ms = new MemoryStream())
{
Serializer.Serialize<T>(ms, o);
return ms.ToArray();
}
}
}
class Program
{
static void Main(string[] args)
{
Random RandomClass = new Random();
using (var conn = new RedisConnection("localhost"))
{
conn.Open();
for (int i = 0; i < 500000; i++)
{
Price p = new Price("IBM", RandomClass.Next(0, 1000));
conn.AddToSet(2, "PRICE.IBM", p.ToBArray());
}
That is entirely correct. "Get" (BookSleeve) returns a deferred byte[]. You have correctly used Wait to get the actual byte[], then used a MemoryStream over this byte[] to call Deserialize via protobuf-net.
All good.
If you make it clear any steps that you find ugly, I might be able to be more specific, but:
BookSleeve is entirely async via Task, hence the need for either Wait or ContinueWith to access the byte[]
protobuf-net is entirely Stream-based, hence the need for MemoryStream to sit on top of a byte[]
Of course, if you add a generic utility method (maybe an extension method) you only need to write it once.
And with the addition if a wrapper class (for some tracking/sliding-expiry) and a L1 cache (Redis as L2), this is pretty much exacty how we use it at stackoverflow.
One note: the connection is thread safe and intended to be massively shared; don't do a connection per operation.
Related
I'm using Azure Redis Cache for development and wanted to verify the way I'm handling the exceptions. According to the best practices, it's possible to face RedisConnectionExceptions and to resolve this, we have to dispose the old ConnectionMultiplexer and create a new one. If abortConnect is set to false, then the multiplexer will silently retry connecting without throwing the error. So if the exception is thrown, it will only be after some attempts to reconect and still failing. Is my understanding of this correct?
This is my connection string -
cachename.redis.cache.windows.net:6380,password=Password,ssl=True,abortConnect=False
I believe the connection exception will only occus when you try to call GetConnection() on the multiplexer. Find my Code below -
static Lazy<ConnectionMultiplexer> multiplexer = CreateMultiplexer();
public static ConnectionMultiplexer GetConnection() => multiplexer.Value;
private static Lazy<ConnectionMultiplexer> CreateMultiplexer()
{
return new Lazy<ConnectionMultiplexer>(() => ConnectionMultiplexer.Connect(connectionString));
}
private static void CloseMultiplexer(Lazy<ConnectionMultiplexer> oldMultiplexer)
{
if (oldMultiplexer != null)
{
oldMultiplexer.Value.Close();
}
}
public static void Reconnect()
{
var oldMultiplexer = multiplexer;
CloseMultiplexer(multiplexer);
multiplexer = CreateMultiplexer();
}
And I'm Consuming this below in another class -
public class RedisCacheManager
{
private static IDatabase _cache;
private TimeSpan expiry = new TimeSpan(hours: 6, minutes: 0, seconds: 0);
public RedisCacheManager()
{
try
{
_cache = RedisCacheHelper.GetConnection().GetDatabase();
}
catch(RedisConnectionException)
{
RedisCacheHelper.Reconnect();
new RedisCacheManager();
}
}
public async Task<RedisValue[]> GetFromCacheAsync(List<string> keys)
{
var cacheValues = await _cache.StringGetAsync(keys.Select(k => (RedisKey)k).ToArray());
return cacheValues;
}
public async Task SaveInCacheAsync<TValue>(Dictionary<string, TValue> kvps)
{
var tasks = new List<Task>();
foreach(var kvp in kvps)
{
tasks.Add(_cache.StringSetAsync(kvp.Key, JsonConvert.SerializeObject(kvp), expiry));
}
await Task.WhenAll(tasks);
}
}
I'm not sure id calling the constructor in the catch block is a good practice. And are there any other exceptions that I would need to handle while calling StringGetAsync and StringSetAsync?
The CacheManager can look like this:
using Newtonsoft.Json;
using StackExchange.Redis;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
public sealed class RedisCacheManager : IDisposable
{
private readonly TimeSpan _expiry;
private readonly Lazy<ConnectionMultiplexer> _lazyConnection;
private ConnectionMultiplexer Connection { get => _lazyConnection.Value; }
public RedisCacheManager(string connectionString, TimeSpan expiry)
{
_expiry = expiry;
_lazyConnection = new Lazy<ConnectionMultiplexer>(() => ConnectionMultiplexer.Connect(connectionString));
}
public async Task<RedisValue[]> GetFromCacheAsync(IEnumerable<string> keys)
{
var cacheValues = await Connection.GetDatabase()
.StringGetAsync(keys.Select(key => (RedisKey)key).ToArray()).ConfigureAwait(false);
return cacheValues;
}
public async Task SaveInCacheAsync<TValue>(Dictionary<string, TValue> kvps)
{
var tasks = kvps
.Select(kvp => Connection.GetDatabase().StringSetAsync(kvp.Key, JsonConvert.SerializeObject(kvp), _expiry))
.ToArray();
await Task.WhenAll(tasks).ConfigureAwait(false);
}
public void Dispose()
{
if (_lazyConnection.IsValueCreated)
{
_lazyConnection.Value.Dispose();
}
}
}
Using:
public readonly static RedisCacheManager RedisCacheManager = new RedisCacheManager("connection string", TimeSpan.FromHours(6));
Remarks:
it is intended that abortConnect=false (which means that the call succeeds even if a connection to the Azure Cache for Redis is not established) and from constructor shouldn't be thrown any Redis-exceptions
The object returned from GetDatabase is a cheap pass-thru object, and does not need to be stored.
GetFromCacheAsync / SaveInCacheAsync-methods can throw an exception to outside and it is OK. You can apply Retry-policy to resolve transient faults.
If you have any IoC-container then it should create RedisCacheManager with a single instance scope (for example, Autofac registration)
I have a complex object which is ISerializable and i want to serialize it into an XML document (node that i rather to not change the source code and add XML serialization attribute stuff).
ISerializable works fine with BinaryFormatter, but there is no standard way to serialize it into XML or Json.
The Json.NET library does support for serializing a ISerializable object into json, but there is a very small problem with that implementation, and that is the serializable constructor of class should be public in order to Json.net detect it (see this issue) and this does make Json.net unusable for my case.
Is there any other way to serialize/deserialize ISerializable object to/from xml, Json or any other plane text formats?
Json.NET does in fact support nonpublic streaming serialization constructors for ISerializable types. For confirmation see the source code for DefaultContractResolver.CreateISerializableContract().
Your actual problem is that the ISerializable type in question is also a collection, and it appears Json.NET uses an array contract in preference to a JsonISerializableContract for such types, as shown in DefaultContractResolver.CreateContract():
if (typeof(IEnumerable).IsAssignableFrom(t))
{
return CreateArrayContract(objectType);
}
if (CanConvertToString(t))
{
return CreateStringContract(objectType);
}
#if !(DOTNET || PORTABLE40 || PORTABLE)
if (!IgnoreSerializableInterface && typeof(ISerializable).IsAssignableFrom(t))
{
return CreateISerializableContract(objectType);
}
#endif
To work around this problem, you can create your own custom contract resolver that reverses this logic:
public class ISerializableCollectionContractResolver : DefaultContractResolver
{
protected override JsonContract CreateContract(Type objectType)
{
var contract = base.CreateContract(objectType);
var underlyingType = Nullable.GetUnderlyingType(objectType) ?? objectType;
if (!IgnoreSerializableInterface
&& typeof(ISerializable).IsAssignableFrom(underlyingType)
&& contract is JsonArrayContract
&& !underlyingType.GetCustomAttributes<JsonContainerAttribute>().Any())
{
contract = CreateISerializableContract(objectType);
}
return contract;
}
}
Your custom collections should now be serialized through their ISerializable interface.
You may want to cache the contract resolver for best performance.
DataContractSerializer and DataContractJsonSerializer both support ISerializable. See Types Supported by the Data Contract Serializer.
For instance, consider the following class:
[Serializable]
public class SerializableClass : ISerializable
{
readonly int valueField;
public SerializableClass(int valueField)
{
this.valueField = valueField;
}
public int Value { get { return valueField; } }
#region ISerializable Members
protected SerializableClass(SerializationInfo info, StreamingContext context)
{
this.valueField = info.GetInt32("valueField");
}
void ISerializable.GetObjectData(SerializationInfo info, StreamingContext context)
{
info.AddValue("valueField", valueField);
}
#endregion
}
And following helper methods:
public static partial class DataContractSerializerHelper
{
public static string SerializeXml<T>(T obj, DataContractSerializer serializer = null, XmlWriterSettings settings = null)
{
serializer = serializer ?? new DataContractSerializer(obj.GetType());
using (var textWriter = new StringWriter())
{
settings = settings ?? new XmlWriterSettings { Indent = true, IndentChars = " " };
using (var xmlWriter = XmlWriter.Create(textWriter, settings))
{
serializer.WriteObject(xmlWriter, obj);
}
return textWriter.ToString();
}
}
public static T DeserializeXml<T>(string xml, DataContractSerializer serializer = null)
{
using (var textReader = new StringReader(xml ?? ""))
using (var xmlReader = XmlReader.Create(textReader))
{
return (T)(serializer ?? new DataContractSerializer(typeof(T))).ReadObject(xmlReader);
}
}
}
public static partial class DataContractJsonSerializerHelper
{
private static MemoryStream GenerateStreamFromString(string value)
{
return new MemoryStream(Encoding.Unicode.GetBytes(value ?? ""));
}
public static string SerializeJson<T>(T obj, DataContractJsonSerializer serializer = null)
{
serializer = serializer ?? new DataContractJsonSerializer(obj.GetType());
using (var memory = new MemoryStream())
{
serializer.WriteObject(memory, obj);
memory.Seek(0, SeekOrigin.Begin);
using (var reader = new StreamReader(memory))
{
return reader.ReadToEnd();
}
}
}
public static T DeserializeJson<T>(string json, DataContractJsonSerializer serializer = null)
{
serializer = serializer ?? new DataContractJsonSerializer(typeof(T));
using (var stream = GenerateStreamFromString(json))
{
var obj = serializer.ReadObject(stream);
return (T)obj;
}
}
}
Then
var test = new SerializableClass(42);
var xml = DataContractSerializerHelper.SerializeXml(test);
Debug.WriteLine(xml);
Produces
<SerializableClass xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns:x="http://www.w3.org/2001/XMLSchema" xmlns="http://schemas.datacontract.org/2004/07/Question38188639">
<valueField i:type="x:int" xmlns="">42</valueField>
</SerializableClass>
And
var json = DataContractJsonSerializerHelper.SerializeJson(test);
Debug.WriteLine(json);
Produces
{"valueField":42}
Would like to make this solution as a generic solution wherein instead of Person object, GetJSON Method should accept the generic type which is has the DataContract attribute. Can anybody explain how to go about doing it.
Here is the base code
namespace TestDataContractJsonSerializer
{
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.IO;
using System.Runtime.Serialization.Json;
public class Program
{
public static void Main(string[] args)
{
List<Person> persons = new List<Person> { new Person("Person1"), new Person("Person2") };
var strJSON = from p in persons select GetJSON(p);
Console.WriteLine("In JSON Format: ");
foreach (string str in strJSON)
{
Console.WriteLine(" {0}", str);
}
Console.ReadKey();
}
private static string GetJSON(Person p)
{
if (p != null)
{
MemoryStream stream = new MemoryStream();
DataContractJsonSerializer dataContractJsonSerializer = new DataContractJsonSerializer(typeof(Person));
dataContractJsonSerializer.WriteObject(stream, p);
stream.Position = 0;
StreamReader sr = new StreamReader(stream);
return sr.ReadToEnd();
}
return string.Empty;
}
}
[DataContract]
public class Person
{
public Person(string name)
{
this.Name = name;
}
[DataMember]
public string Name { get; private set; }
}
}
It would look something like this in the very basic case. You'd probably need to add some special casing and/or error handling, for instance if the type T is not serializable.
private static string GetJSON<T>(T objToSerialize)
{
if (objToSerialize != null)
{
MemoryStream stream = new MemoryStream();
DataContractJsonSerializer dataContractJsonSerializer = new DataContractJsonSerializer(objToSerialize.GetType());
dataContractJsonSerializer.WriteObject(stream, objToSerialize);
stream.Position = 0;
StreamReader sr = new StreamReader(stream);
return sr.ReadToEnd();
}
return string.Empty;
}
I am new to servicestack and am really enjoying it, however I can not for the life of me figure of why this is occuring.
I have mapped it as ROUTES.Add<images>("/Images"); in the APPHOST.cs
I use Rest Console(chrome plugin) to test and POST the following JSON:
(I know it is not and actual base64 encoding).
Any thoughts would be appreciated.
// Testing JSON
{"base64Encoding":"asdasdasdasd"}
//
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using ServiceStack.ServiceInterface;
using RewoServiceLayer.RequestDTOs;
using System.Data.Entity;
using ServiceStack.ServiceInterface.ServiceModel;
using ServiceStack.Configuration;
using ServiceStack.Common;
using ServiceStack.ServiceHost;
using ServiceStack.ServiceInterface.Auth;
using ServiceStack.WebHost.Endpoints;
using System.IO;
using System.Web.Hosting;
namespace Blah.Services
{
public class ImageResponse
{
public images image { get; set; }
public ResponseStatus ResponseStatus { get; set; } //Where Exceptions get auto-serialized
}
public class ImageRequest
{
public String base64Encoding;
}
public class ImageService : Service
{
//Give me something that looks like this:
//"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAABG4...YII=";
public ImageResponse Post(ImageRequest request)
{
String base64EncodingImg = request.base64Encoding;
using (var db = new BlahDB())
{
//save to db
images imgToSave = new images();
//get base64
string base64 = base64EncodingImg.Substring(base64EncodingImg.IndexOf(',') + 1);
base64 = base64.Trim('\0');
byte[] imgBinData = Convert.FromBase64String(base64);
//get the
imgToSave.image_type = base64EncodingImg.Substring(0,base64EncodingImg.IndexOf(','));
db.images.Add(imgToSave);
db.SaveChanges();
//then save string contents to disk using ID
imgToSave.image_disk_loc = getPathFromImage(imgToSave);
writeByteArrToDisk(imgBinData, imgToSave.image_disk_loc);
db.SaveChanges();
ImageResponse imgResponse = new ImageResponse();
imgResponse.image = imgToSave;
return imgResponse;
}
}
private Boolean writeByteArrToDisk(byte[] toWrite, String path)
{
try
{
File.WriteAllBytes(path, toWrite);
return true;
}
catch (Exception e)
{
return false;
}
}
private String getAbsolutePathToImagesFolder()
{
return HostingEnvironment.MapPath(#"~/App_Data/UploadedImages");
}
private String getPathFromImage(images imgModel)
{
if (imgModel.image_disk_loc.IsNullOrEmpty())
{
return getAbsolutePathToImagesFolder() + imgModel.image_id;
}
else
{
return imgModel.image_disk_loc;
}
}
}
}
I am getting the following error and it does not hit the Post Method when I debug:
{
"responseStatus": {
"errorCode": "NullReferenceException",
"message": "Object reference not set to an instance of an object.",
"stackTrace": " at ServiceStack.WebHost.Endpoints.Utils.FilterAttributeCache.GetRequestFilterAttributes(Type requestDtoType)\r\n at ServiceStack.WebHost.Endpoints.EndpointHost.ApplyRequestFilters(IHttpRequest httpReq, IHttpResponse httpRes, Object requestDto)\r\n at ServiceStack.WebHost.Endpoints.RestHandler.ProcessRequest(IHttpRequest httpReq, IHttpResponse httpRes, String operationName)"
}
}
I can't see your images class but I think you need to change
ROUTES.Add<images>("/Images");
To
ROUTES.Add<ImageRequest>("/Images")
since the Post on your ImageService is looking for a request of type ImageRequest.
I am trying to create a WCF DataService using in-memory object graph. This means that the backend is not an Entity Framework store, but a bunch of objects that reside in memory.
I am trying to create a service operation called GetUsersByName that has a single parameter for name and returns the matching users as an IQueryable collection.
I followed the documentation and added the access rules for this operation
config.SetServiceOperationAccessRule("GetUsersByName", ServiceOperationRights.All);
But when the SetServiceOperationAccessRule method is called I receive an exception on the client:
System.AggregateException was unhandled.
Here is the full code for my console application
using System;
using System.ServiceModel;
using System.ServiceModel.Web;
using System.ServiceModel.Description;
using System.Data.Services;
using System.Data.Services.Common;
using System.Collections.Generic;
using System.Runtime.Serialization;
using System.Collections.ObjectModel;
using System.Linq;
using System.Web;
using System.Net.Http;
using System.Net;
using System.IO;
namespace WCF_OData
{
class Program
{
static void Main(string[] args)
{
string serviceAddress = "http://localhost:8080";
Uri[] uriArray = { new Uri(serviceAddress) };
Type serviceType = typeof(UserDataService);
using (var host = new DataServiceHost(serviceType, uriArray)) {
host.Open();
var client = new HttpClient() { BaseAddress = new Uri(serviceAddress) };
Console.WriteLine("Client received: {0}", client.GetStringAsync("Users?$format=json").Result);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://localhost:8080");
request.Method = "GET";
request.Accept = #"application/json";
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
Console.WriteLine(response.StatusCode);
Console.WriteLine(response.ContentType);
Console.WriteLine((new StreamReader(response.GetResponseStream())).ReadToEnd());
}
Console.WriteLine("Press any key to stop service");
Console.ReadKey();
}
}
}
[EnableJsonSupport]
[ServiceBehavior(IncludeExceptionDetailInFaults = true)]
public class UserDataService : DataService<UserService> {
public static void InitializeService(DataServiceConfiguration config)
{
config.SetEntitySetAccessRule("Users", EntitySetRights.All);
config.SetServiceOperationAccessRule("GetUsersByName", ServiceOperationRights.All);
config.UseVerboseErrors = true;
config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;
}
}
public class UserService
{
private List<User> _List = new List<User>();
public UserService()
{
_List.Add(new User() { ID = 1, UserName = "John Doe" });
_List.Add(new User() { ID = 2, UserName = "Jane Doe" });
}
public IQueryable<User> Users
{
get
{
HttpContext x = HttpContext.Current;
return _List.AsQueryable<User>();
}
}
[OperationContract]
[WebGet(UriTemplate="GetUsersByName")]
public IQueryable<User> GetUsersByName(string name)
{
return new List<User>().AsQueryable();
}
}
[DataServiceKey("ID")]
public class User
{
public int ID { get; set; }
public string UserName { get; set; }
}
}
It looks like there are a few things going on here, so this may take a couple of iterations to work through. The first problem that should be fixed is the service operation. Service operations need to be declared on the class that inherits from DataService: "Service operations are methods added to the data service class that derives from DataService". Here's a sample:
using System.Data.Entity;
using System.Data.Services;
using System.Data.Services.Common;
using System.Linq;
using System.ServiceModel;
using System.ServiceModel.Web;
namespace Scratch.Web
{
[ServiceBehavior(IncludeExceptionDetailInFaults = true)]
public class ScratchService : DataService<ScratchEntityFrameworkContext>
{
static ScratchService()
{
Database.SetInitializer(new ScratchEntityFrameworkContextInitializer());
}
public static void InitializeService(DataServiceConfiguration config)
{
config.SetEntitySetAccessRule("*", EntitySetRights.All);
config.SetServiceOperationAccessRule("*", ServiceOperationRights.AllRead);
config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V3;
config.UseVerboseErrors = true;
}
[WebGet]
public IQueryable<Product> FuzzySearch(string idStartsWith)
{
var context = new ScratchEntityFrameworkContext();
return context.Products.ToList().Where(p => p.ID.ToString().StartsWith(idStartsWith)).AsQueryable();
}
}
}
You should then be able to call your service operation from a browser, with a URL format similar to the following: http://localhost:59803/ScratchService.svc/FuzzySearch()?idStartsWith='1'
Can we start by trying to get this functional in a browser and then see whether the AggregateException still happens?