I have a Flink app which keeps erroring out with this error.
com.org.ads.audience.traffic.MyClass#6eaa21d8 is not serializable. The object probably contains or references non serializable fields.
org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:140)
org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:115)
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.clean(StreamExecutionEnvironment.java:1558)
org.apache.flink.streaming.api.datastream.DataStream.clean(DataStream.java:185)
org.apache.flink.streaming.api.datastream.DataStream.flatMap(DataStream.java:611
Its proving really hard to track why this class is not serializable. I've made sure that MyClass and all other classes that it references are Serializable and have default constructors. How can I debug this further? I tried adding -Dsun.io.serialization.extendedDebugInfo=true to the command line but it does not seem to be providing me any more information.
I am running in yarn cluster mode.
One of my classes was using JDBC's BasicDataSource which contains non serializable elements. I had to mark that as transient to be able to serialize.
Related
I have the following code working in a SpringBoot application, and it does what's I'm expecting.
TypePool typePool = TypePool.Default.ofClassPath();
ByteBuddyAgent.install();
new ByteBuddy()
.rebase(typePool.describe("com.foo.Bar").resolve(), ClassFileLocator.ForClassLoader.ofClassPath())
.implement(typePool.describe("com.foo.SomeInterface").resolve())
.make()
.load(ClassLoader.getSystemClassLoader());
Its makes is so that the class com.foo.Bar implements the interface com.foo.SomeInterface (which has a default implementation)
I would like to . use the above code by referring to the class as Bar.class, not using the string representation of the name. But if I do that I get the following exception.
java.lang.UnsupportedOperationException: class redefinition failed: attempted to change superclass or interfaces
I believe due to the fact that it cause the class to be loaded, prior to the redefinition. I'm just now learning to use ByteBuddy.
I want to avoid some reflection at runtime, by adding the interface and an implementation using ByteBuddy. I've some other code that checks for this interface.
This is impossible, not because of Byte Buddy but no tool is allowed to do this on a regular VM. (There is the so-called dynamic code evolution VM which is capable of that).
If you want to avoid the problem, use redefine rather then rebase. Whenever you instrument a method, you do now however replace the original.
If this is not acceptable, have a look at the Advice class which you can use by the .visit-API to wrap logic around your original code without replacing it.
My application has this structure: there's a RepositoryFacade (that is a Singleton) that uses many other ObjectRepository that are Singleton (UserRepository, etc).
Now I'd like to test it, mocking the [Objetct]Repositiries. To do that I made the [Objetct]Repositiry implements an interface, and then i tried to:
final IUserRepository mockIUserRepository= context.mock(IUserRepository.class);
RepositoryFacade.getInstance().setUserRepository(mockIUserRepository);
final User testUser = new User("username");
// expectations
context.checking(new Expectations() {{
oneOf (mockIUserRepository).save(testUser);
}});
// execute
RepositoryFacade.getInstance().save(testUser);
And in RepositoryFacade I added:
public IUserRepository userRepository = UserRepository.getInstance();
But if I try to run the test, I obtain:
java.lang.SecurityException: class "org.hamcrest.TypeSafeMatcher"'s signer
information does not match signer information of other classes in the same
package
p.s. Originally my RepositoryFacade had not a IUserRepository variable, I used it asking always UserRepository.getInstance().what_i_want(). I introduced it to try to use JMock, so if not needed I'll be glad to remove that bad use of Singleton.
Thanks,
Andrea
The error you're getting suggests that you have a classloading issue with the org.hamcrest package rather than any issue with your singletons. See this question for more on this exception and this one for the particular problem with hamcrest and potential solutions.
Check your classpath to make sure you're not including conflicting hamcrest code from multiple jars. If you find hamcrest in multiple jars, this may be corrected by something as simple as changing their order in your classpath.
Junit itself comes in two versions - one may include an old version of hamcrest. Switching to the one not including hamcrest may also fix your problem.
If you can find a way to do it, it would be better in the long run to get rid of the singletons altogether and instead do dependency injection using something like Spring or Guice.
But what you're doing should work, once you deal with the classloading, and it's a reasonable approach to dealing with singletons in a testing context.
While doing GetComponentParts I am getting following error, to be specific it reporoduces while i do import in continuous loop while handling multiple messages in my WCF application
The composition produced a single composition error. The root cause is provided below. Review the CompositionException.Errors property for more detailed information.
1) The composition failed because it did not complete within '100' iterations. This is most likely caused by a cycle in the dependency graph of a part which is marked with a non-shared creation policy.
One more thing I cant locate CompositionException.Errors to find the root cause.
Please suggest what workaround possible, as i am not getting a single thread on the net suggesting any way...
Many Thanks
Since this is working most of the time, it is probably a threading issue which is corrupting MEF's internal state and causing this error.
When using a CompositionContainer from multiple threads, you need to create it with the isThreadSafe parameter set to true, and avoid calling methods which modify what is available - such as Compose, ComposeParts, or AddExportedValue.
Methods which are safe to call are the GetExport and SatisfyImports methods.
I've been (happily) using Ninject for a while now with some basic scenarios, and would like to give it control of my logging. I noted the existence of the Ninject.Extensions.Logging namespace, and would like to use it, but I'm running into two issues:
I want the logger to be initialized with the type of the class running it (as if I ran LogManager.GetLogger with the GetCurrentMethod().DeclaringType).
I want to be able to easily mock, or "nullify" the logger for unit testing (i.e I don't want to have the logger work), without running into NullReferenceExceptions for not initializing the logger.
Now, I know there are some questions (and even answers) around here, but I couldn't seem to find any that pointed me in the right direction.
I'll appreciate any help (even a "you bone-head" it's here! Linking to something I should have noticed).
This is the default behavior of the extension
Don't use Ninject to create the object under test in your unit tests. Create an instance manually and pass what ever you want for the logger.
Best you have a look at the unittests. https://github.com/ninject/ninject.extensions.logging/blob/master/src/Ninject.Extensions.Logging.Tests/Infrastructure/CommonTests.cs
I recently moved to using the binary serializer to send messages with NServiceBus. My messages are all defined as interfaces and are instantiated using
bus.Send<MessageType>(msg => msg.Property = someValue)
This leads to an exception being thrown from NServiceBus stating that
Cannot create an instance of an
interface
I can see from the stack trace that the SimpleMessageMapper is being used, and after looking in the source can see it's making a call to Activator.CreateInstance.
I can't find anything in the documentation stating that it's not possible to do what I'm trying to do, is there a way to fix this?
Thanks,
Matt
I only just started playing with nServiceBus, so all I can offer you is theory :).
Are you defining the implementation classes for your message interfaces, or is nServiceBus generating classes on its own? If the former, make sure you still have a default constructor and that the class and all fields/events are marked as [Serializable] or [NonSerialized]. If the latter, it's possible that nServiceBus doesn't know how to generate members which may be needed for (de)serialization. You may have to write and map the implementation class yourself.