I am trying to write a List of a specific type using Kryo serializer but I am getting errors when I try to read/write it. I am using source provided by apache spark for KryoRegistrator
List I am trying to write is of type List<A> which could be an ArrayList or any other type of list
Code
Class FakeRegsitrator implements KryoRegistrator{
#Override
public void registerClasses(Kryo kryo) {
CollectionSerializer listSerializer = new CollectionSerializer();
listSerializer.setElementClass(A.class, kryo.getSerializer(A.class));
listSerializer.setElementsCanBeNull(false);
kryo.register(A.class, new Serializer<A>(true, true) {
public void write(Kryo kryo, Output output, A a) {
output.writeLong(a.getFieldA)
output.WriteString(a.getFieldB)
}
public A read(Kryo kryo, Input input, Class type) {
return new A(input.readLong(), input.readString())
}
}
}
What am I missing here?
I was able to get it working by registering Arraylist.class
code :
kryo.register(ArrayList.class);
in read method use
kryo.readObject(input, ArrayList.class);
in write method use
kryo.writeObject(output, entry.getArchivePortions());
Related
The main WebServerExtension example from the JUnit5 manual is incomplete and it doesn't fully show how to properly store the configuration (e.g. enableSecurity, server url).
https://github.com/junit-team/junit5/blob/master/documentation/src/main/java/example/registration/WebServerExtension.java
The example ignores or hard codes the values. The manual (section 5.11. Keeping State in Extensions) implies that the "Store" should be used but the ExtensionContext is not yet available yet when the object is constructed -- its not clear how to handle migrating this data to the Store as the ExtensionContext is not yet available in the constructor.
Also its not clear to me that using the Store API for the WebServerExtension programmatic example is even desirable and perhaps it could work just using the internal state (e.g. this.serverUrl, this.enableSecurity, etc.).
Maybe the Store is more applicable to Extensions which don't use this "programmatic" style where multiple instances of the custom extension may exist (appropriately)? In other words its not clear to me from the guide if this a supported paradigm or not?
Other JUnit 5 extension examples online (e.g. org.junit.jupiter.engine.extension.TempDirectory) show how to leverage annotations to handle passing configuration info to the Store but it would be nice if there were a complete programmatic builder type example like WebServerExtension too.
Examples like TempDirectory clearly have access to the ExtensionContext from the beforeXXX() methods whereas the WebServerExtension example does not.
Using the following approach below seems to work fine but I wanted confirmation that this is a supported paradigm (i.e. using fields instead of Stores when using this programmatic approach).
public class WebServerExtension implements BeforeAllCallback {
private final boolean securityEnabled;
private final String serverUrl;
public WebServerExtension(Builder builder) {
this.securityEnabled = builder.enableSecurity;
this.serverUrl = build.serverUrl;
}
#Override
public void beforeAll(ExtensionContext context) {
// is it ok to use this.securityEnabled, this.serverUrl instead of Store API???
}
public String getServerUrl() {
return this.serverUrl;
}
public boolean isSecurityEnabled() {
return this.securityEnabled;
}
public static Builder builder() {
return new Builder();
}
public static class Builder {
private boolean enableSecurity;
private String serverUrl;
public Builder enableSecurity(boolean b) {
this.enableSecurity = b;
return this;
}
public Builder serverUrl(String url) {
this.serverUrl = url;
return this;
}
public WebServerExtension build() {
return new WebServerExtension(this);
}
}
}
Thanks!
I am attempting to create an instance of FileHelperEngine<> using a generic type. For example this works for List
public static IList CreateList(Type type)
{
var genericListType = typeof(List<>).MakeGenericType(type);
return (IList)Activator.CreateInstance(genericListType);
}
Is it possible to do something similar for FileHelperEngine<>?
I tried
public static FileHelperEngine CreateFileHelperEngine(Type type)
{
var genericFileHelperEngineType = typeof (FileHelperEngine<>).MakeGenericType(type);
return (FileHelperEngine)Activator.CreateInstance(genericFileHelperEngineType);
}
And get this error
Unable to cast object of type 'FileHelpers.FileHelperEngine`1[NachaParser.Model.EntryDetail.PpdEntryModel]' to type 'FileHelpers.FileHelperEngine'.
This would not work because you are attempting to go from the generic engine to the standard engine. The generic engine does not inherit from the standard engine so you can't directly cast.
The following code should work for you:
public static FileHelperEngine<T> CreateFileHelperEngine<T>() where T : class
{
var genericFileHelperEngineType = typeof(FileHelperEngine<>).MakeGenericType(typeof(T));
return (FileHelperEngine<T>)Activator.CreateInstance(genericFileHelperEngineType);
}
The problem is that you need to have the type of T not as a Type variable but as as a passed generic argument. Unfortunately, the generic engine is based of EngineBase<T> with IFileHelperEngine<T> as an interface it implements so you can never get a FileHelperEngine<T> to a FileHelperEngine.
Your only other option is to use:
public static FileHelperEngine CreateFileHelperEngine(Type type)
{
if (!type.IsClass)
throw new InvalidCastException("Cannot use '" + type.FullName + "' as it is not a class");
return new FileHelperEngine(type);
}
I have a repository offering a GetAll method which again calls the Query extension method on the ISession instance of NHibernate.
public ICollection<Product> GetAll()
{
return _session.Query<Product>().ToList();
}
My unit test looks like this:
[Test]
public void GetAllReturnsCollectionFromSession()
{
IQueryable<Product> productList = new ProductListBuilder().Build().AsQueryable();
_fixture.Session.Query<Product>().Returns(productList);
var sut = _fixture.CreateSut();
var result = sut.GetAll();
Assert.AreSame(productList, result);
_fixture.Session.Received().Query<Product>();
}
In the _fixture.Session.Query().Returns(productList) statement, NSubstitute throws the following exception:
NSubstitute.Exceptions.CouldNotSetReturnDueToTypeMismatchException : Can not return value of type IQueryable`1Proxy for ISession.GetSessionImplementation (expected type ISessionImplementor).
Make sure you called Returns() after calling your substitute (for example: mySub.SomeMethod().Returns(value)),
and that you are not configuring other substitutes within Returns() (for example, avoid this: mySub.SomeMethod().Returns(ConfigOtherSub())).
If you substituted for a class rather than an interface, check that the call to your substitute was on a virtual/abstract member.
Return values cannot be configured for non-virtual/non-abstract members.
Correct use:
mySub.SomeMethod().Returns(returnValue);
Potentially problematic use:
mySub.SomeMethod().Returns(ConfigOtherSub());
Instead try:
var returnValue = ConfigOtherSub();
mySub.SomeMethod().Returns(returnValue);
at NSubstitute.Core.ConfigureCall.CheckResultIsCompatibleWithCall(IReturn valueToReturn, ICallSpecification spec)
at NSubstitute.Core.ConfigureCall.SetResultForLastCall(IReturn valueToReturn, MatchArgs matchArgs)
at NSubstitute.Core.CallRouter.LastCallShouldReturn(IReturn returnValue, MatchArgs matchArgs)
at NSubstitute.Core.SubstitutionContext.LastCallShouldReturn(IReturn value, MatchArgs matchArgs)
at NSubstitute.SubstituteExtensions.Returns[T](MatchArgs matchArgs, T returnThis, T[] returnThese)
at NSubstitute.SubstituteExtensions.ReturnsForAnyArgs[T](T value, T returnThis, T[] returnThese)
at Statoil.Wellcom.DataLayer.Implementation.Oracle.UnitTests.Repositories.DwapplicationRepositoryTests.GetAllReturnsCollectionFromSession() in C:\git\WELLCOM\source\Statoil.Wellcom.DataLayer.Implementation.Oracle.UnitTests\Repositories\DwapplicationRepositoryTests.cs:line 123
It looks like NSubstitute is unable to set the return value due to Query being an extension method. How would I go about mocking the extension method call on the ISession?
The easiest solution is to wrap your ISession in another interface/concrete class so you can stub that out:
public interface ISessionWrapper
{
IQueryable<T> Query<T>();
}
public class SessionWrapper : ISessionWrapper
{
private readonly ISession _session;
public SessionWrapper(ISession session)
{
_session = session;
}
public IQueryable<T> Query<T>()
{
return _session.Query<T>();
}
}
There is no way to mock extension method with NSubstitute, however if you know what extension method is using inside, than you can mock that. Your test will use extension method on mocked object and eventually it will use mocked method. Difficult part is to know what is going on inside.
It worked for me in projects, where I knew all the source code and I could check what's inside.
I am looking convert a List of custom objects to a Map of custom objects. I have a mapping defined with a custom method, but I keep getting a "cannot be cast to ma.glasnost.orika.MapEntry" exception. What is the proper way to go about converting a List to a Map in Orika?
mapperFactory.classMap(new TypeBuilder<List<com.printable.pti.NameValuePairType>>(){}.build(), new TypeBuilder<Map<String, com.kinetic.entity.TemplateField>>() {}.build())
.customize(new CustomMapper<List<com.printable.pti.NameValuePairType>,Map<String, com.kinetic.entity.TemplateField>>() {
#Override
public void mapAtoB(List<com.printable.pti.NameValuePairType> nameValuePairTypes,
Map<String, com.kinetic.entity.TemplateField> stringTemplateFieldMap, MappingContext context) {
Map<String, com.kinetic.entity.TemplateField> toObject = new HashMap<String, com.kinetic.entity.TemplateField>();
for(com.printable.pti.NameValuePairType nameValuePairType : nameValuePairTypes) {
toObject.put(nameValuePairType.getName(),(com.kinetic.entity.TemplateField)map(nameValuePairType,com.kinetic.entity.TemplateField.class));
}
}
}
)
.register();
Here is a good example of how to map a list of elements to Map using Orika
I have the following:
An interface I1 extends Ia, Ib, Ic
An interface I2.
A class C implements I1, I2. And this class has its own setters and getters as well.
C cInstance = new C():
//Jackson
ObjectMapper mapper = new ObjectMapper();
mapper.writeValue(new File("somefile.json"), cInstance);
//Gson
Gson gson = new Gson();
String json = gson.toJson(cInstance);
The output will be cInstance serialized according to the properties of C and what it inherited.
However, I like the properties are being serialized to be according to the setters/getters in I1 (only the cInstance properties represented in the I1 interface).
How can I do this with Jackson knowing that I have too many classes with the same problem and I can't modify the class definition or add annotations.
And the same issue applies to Deserialization (Deserializing according to an interface)
Thanks
First of all, you can always attach "mix-in annotations" even without adding annotations directly (see wiki page). With this, annotation to use would be:
#JsonSerialize(as=MyInterface.class)
but if you do not want to use mix-ins, you can force specific type to use with
objectMapper.typedWriter(MyInterface.class).writeValue(....)
Jackson's VisibilityChecker provides an easy way for filtering certain properties, especially because it allows you to test for visibility (equals "will be serialized or not") for each method/field individually.
At least this helps for the serialization phase.
Here is what I did (using Jackson version 1.9.11):
import org.codehaus.jackson.map.ObjectMapper;
import org.codehaus.jackson.map.introspect.AnnotatedMethod;
import org.codehaus.jackson.map.introspect.VisibilityChecker;
public static class InterfaceVisibilityChecker extends VisibilityChecker.Std {
private final Set<Method> visibleMethods;
public InterfaceVisibilityChecker(Class<?>... clazzes) {
super(JsonAutoDetect.Visibility.PUBLIC_ONLY);
this.visibleMethods = new HashSet<>();
for (Class<?> clz : clazzes) {
this.visibleMethods.addAll(Arrays.asList(clz.getMethods()));
}
}
#Override
public boolean isGetterVisible(Method m) {
return super.isGetterVisible(m) && isVisible(m);
}
#Override
public boolean isGetterVisible(AnnotatedMethod m) {
return isGetterVisible(m.getAnnotated());
}
private boolean isVisible(Method m) {
for (Method visiMthd : visibleMethods) {
if (isOverwriteMethod(m, visiMthd)) return true;
}
return false;
}
private boolean isOverwriteMethod(Method subMethod, Method superMethod) {
// names must be equal
if (! subMethod.getName().equals(superMethod.getName())) return false;
// return types must be assignable
if (! superMethod.getReturnType().isAssignableFrom(subMethod.getReturnType())) return false;
// parameters must be equal
if (! Arrays.equals(subMethod.getParameterTypes(), superMethod.getGenericParameterTypes())) return false;
// classes must be assignable
return superMethod.getDeclaringClass().isAssignableFrom(subMethod.getDeclaringClass());
}
}
The main idea is to use the standard VisibilityChecker and extend it by a check whether the method is declared in one of the given interfaces.
The checker is applied to an ObjectMapper instance using the following snippet:
ObjectMapper om = new ObjectMapper();
om.setVisibilityChecker(new InterfaceVisibilityChecker(
I1.class,
I2.class,
Ia.class,
Ib.class,
Ic.class
));
Some comments on the solution above:
The checker is not complete, methods like isIsGetterVisible or isFieldVisible can be handled in a similar manner if needed.
isOverwriteMethod is not optimized at all, it's checks could be cached.