The Hazelcast documentation shows that you can use a distributed Hazelcast as a client and mentions that if a member holding the lock gets disconnected - the lock will be released. However, if I am using the lock as a client this will not count as a "member" - will the lock still safely be released if the client gets disconnected?
The client lock will automatically be released. Try the following program and you will see yourself.
public class Main {
public static void main(String[] args) throws InterruptedException {
HazelcastInstance hz1 = Hazelcast.newHazelcastInstance();
HazelcastInstance client = HazelcastClient.newHazelcastClient();
client.getLock("clientLock").lock();
System.out.println("is locked:" + hz1.getLock("clientLock").isLocked());
client.shutdown();
System.out.println("Client is shutdown");
for(;;){
boolean isLocked = hz1.getLock("clientLock").isLocked();
if(!isLocked){
System.out.println("Client lock is released");
return;
}
System.out.println("is locked:"+ isLocked);
Thread.sleep(1000);
}
}
}
Related
As Jedis documentation state that Jedis client is not thread-safe.
A single Jedis instance is not threadsafe!
So I am using JedisPool. I want to push data to browser's WebSocket client from server. For this I am using Redis's PubSub mechanism.
#ServerEndpoint(value = "/websocket/{channelName}", configurator = GetHttpSessionConfigurator.class)
public class WSEndpoint {
private WSJedisPubSub wsJedisPubSub;
private static JedisPool jedisPool = null;
#OnOpen
public void onOpen(Session session,
#PathParam("channelName") String channelName) throws IOException,
EncodeException {
// FIXME proper synchronization is required here
if (jedisPool == null) {
initPool();
}
wsJedisPubSub = new WSJedisPubSub(session);
try (Jedis redisClient = jedisPool.getResource()) {
redisClient.subscribe(wsJedisPubSub, channelName);
}
private void initPool() {
JedisPoolConfig jedisConfiguration = new JedisPoolConfig();
jedisPool = new JedisPool(jedisConfiguration, "localhost", 6379);
}
}
full code
My application can have thousands of websockets connected to it. I have doubts about following piece of code.
try (Jedis redisClient = jedisPool.getResource()) {
redisClient.subscribe(wsJedisPubSub, channelName);
}
This redisClient should get close after try-with-resouce block, but still it is working(getting subscribed events). How ?
By default, pool size is 8. I can set to n but eventually I will have n+1 web sockets. What is the best way to deal with this ? Should I have only one Jedis instance and do the routing of message by myself ?
If Jedis client gets disconnected, then what is the recommended way for reconnect here ?
My case is rabbitmq server got out of space, just as below
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/ramonubuntu--vg-root 6299376 5956336 0 100% /
The producer publishes message to server(the message needs to be persisted), and then will be blocked forever, it will keeping waiting the response of publishing. Sure we should avoid the situation of server out of space, but is there any timeout mechanism to let producer quit the waiting?
I have tried heartbeat and SO_TIMEOUT, they both don't work, as the network works fine. Below is my producer.
protected void publish(byte[] message) throws Exception {
// ConnectionFactory can be reused between threads.
ConnectionFactory factory = new SoTimeoutConnectionFactory();
factory.setHost(this.getHost());
factory.setVirtualHost("te");
factory.setPort(5672);
factory.setUsername("amqp");
factory.setPassword("amqp");
factory.setConnectionTimeout(10 * 1000);
// doesn't help if server got out of space
factory.setRequestedHeartbeat(1);
final Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
// declare a 'topic' type of exchange
channel.exchangeDeclare(this.exchangeName, "topic", true);
channel.addReturnListener(new ReturnListener() {
#Override
public void handleReturn(int replyCode, String replyText, String exchange, String routingKey,
AMQP.BasicProperties properties, byte[] body) throws IOException {
logger.warn("[X]Returned message(replyCode:" + replyCode + ",replyText:" + replyText
+ ",exchange:" + exchange + ",routingKey:" + routingKey + ",body:" + new String(body));
}
});
channel.confirmSelect();
channel.addConfirmListener(new ConfirmListener() {
#Override
public void handleAck(long deliveryTag, boolean multiple) throws IOException {
logger.info("Ack: " + deliveryTag);
// RabbitMessagePublishMain.this.release(connection);
}
#Override
public void handleNack(long deliveryTag, boolean multiple) throws IOException {
logger.info("Nack: " + deliveryTag);
// RabbitMessagePublishMain.this.release(connection);
}
});
channel.basicPublish(this.exchangeName, RabbitMessageConsumerMain.EXCHANGE_NAME + ".-1", true,
MessageProperties.PERSISTENT_BASIC, message);
channel.waitForConfirmsOrDie(10*1000);
// now we can close connection
connection.close();
}
It will block at 'channel.waitForConfirmsOrDie(10*1000);', and the SotimeoutConnectionFactory,
public class SoTimeoutConnectionFactory extends ConnectionFactory {
#Override
protected void configureSocket(Socket socket) throws IOException {
super.configureSocket(socket);
socket.setSoTimeout(10 * 1000);
}
}
Also I captured the network between producer and rabbimq,
Please help.
You need to implement Connection Block/Unblocked.
This is basically a way of notifying the publisher that the server is running out of resources. The advantage with this is that the publisher will also be notified once it is safe to publish again.
I would recommend that you take a look at this article. A simple way of implementing this is to have a flag that indicates if it is safe to publish, if it is not wait until it is.
As an example you can take a look on how I implemented this in one of my Python examples.
I have the following planned architecture for my WCF client library:
using ChannelFactory instead of svcutil generated proxies because
I need more control and also I want to keep the client in a separate
assembly and avoid regenerating when my WCF service changes
need to apply a behavior with a message inspector to my WCF
endpoint, so each channel is able to send its
own authentication token
my client library will be used from a MVC front-end, so I'll have to think about possible threading issues
I'm using .NET 4.5 (maybe it has some helpers or new approaches to implement WCF clients in some better way?)
I have read many articles about various separate bits but I'm still confused about how to put it all together the right way. I have the following questions:
as I understand, it is recommended to cache ChannelFactory in a static variable and then get channels out of it, right?
is endpoint behavior specific to the entire ChannelFactory or I can apply my authentication behavior for each channel separately? If the behavior is specific to the entire factory, this means that I cannot keep any state information in my endpoint behavior objects because the same auth token will get reused for every channel, but obviously I want each channel to have its own auth token for the current user. This means, that I'll have to calculate the token inside of my endpoint behavior (I can keep it in HttpContext, and my message inspector behavior will just add it to the outgoing messages).
my client class is disposable (implements IDispose). How do I dispose the channel correctly, knowing that it might be in any possible state (not opened, opened, failed ...)? Do I just dispose it? Do I abort it and then dispose? Do I close it (but it might be not opened yet at all) and then dispose?
what do I do if I get some fault when working with the channel? Is only the channel broken or entire ChannelFactory is broken?
I guess, a line of code speaks more than a thousand words, so here is my idea in code form. I have marked all my questions above with "???" in the code.
public class MyServiceClient : IDisposable
{
// channel factory cache
private static ChannelFactory<IMyService> _factory;
private static object _lock = new object();
private IMyService _client = null;
private bool _isDisposed = false;
/// <summary>
/// Creates a channel for the service
/// </summary>
public MyServiceClient()
{
lock (_lock)
{
if (_factory == null)
{
// ... set up custom bindings here and get some config values
var endpoint = new EndpointAddress(myServiceUrl);
_factory = new ChannelFactory<IMyService>(binding, endpoint);
// ???? do I add my auth behavior for entire ChannelFactory
// or I can apply it for individual channels when I create them?
}
}
_client = _factory.CreateChannel();
}
public string MyMethod()
{
RequireClientInWorkingState();
try
{
return _client.MyMethod();
}
catch
{
RecoverFromChannelFailure();
throw;
}
}
private void RequireClientInWorkingState()
{
if (_isDisposed)
throw new InvalidOperationException("This client was disposed. Create a new one.");
// ??? is it enough to check for CommunicationState.Opened && Created?
if (state != CommunicationState.Created && state != CommunicationState.Opened)
throw new InvalidOperationException("The client channel is not ready to work. Create a new one.");
}
private void RecoverFromChannelFailure()
{
// ??? is it the best way to check if there was a problem with the channel?
if (((IChannel)_client).State != CommunicationState.Opened)
{
// ??? is it safe to call Abort? won't it throw?
((IChannel)_client).Abort();
}
// ??? and what about ChannelFactory?
// will it still be able to create channels or it also might be broken and must be thrown away?
// In that case, how do I clean up ChannelFactory correctly before creating a new one?
}
#region IDisposable
public void Dispose()
{
// ??? is it how to free the channel correctly?
// I've heard, broken channels might throw when closing
// ??? what if it is not opened yet?
// ??? what if it is in fault state?
try
{
((IChannel)_client).Close();
}
catch
{
((IChannel)_client).Abort();
}
((IDisposable)_client).Dispose();
_client = null;
_isDisposed = true;
}
#endregion
}
I guess better late then never... and looks like author has it working, this might help future WCF users.
1) ChannelFactory arranges the channel which includes all behaviors for the channel. Creating the channel via CreateChannel method "activates" the channel. Channel factories can be cached.
2) You shape the channel factory with bindings and behaviors. This shape is shared with everyone who creates this channel. As you noted in your comment you can attach message inspectors but more common case is to use Header to send custom state information to the service. You can attach headers via OperationContext.Current
using (var op = new OperationContextScope((IContextChannel)proxy))
{
var header = new MessageHeader<string>("Some State");
var hout = header.GetUntypedHeader("message", "urn:someNamespace");
OperationContext.Current.OutgoingMessageHeaders.Add(hout);
}
3) This is my general way of disposing the client channel and factory (this method is part of my ProxyBase class)
public virtual void Dispose()
{
CloseChannel();
CloseFactory();
}
protected void CloseChannel()
{
if (((IChannel)_client).State == CommunicationState.Opened)
{
try
{
((IChannel)_client).Close();
}
catch (TimeoutException /* timeout */)
{
// Handle the timeout exception
((IChannel)innerChannel).Abort();
}
catch (CommunicationException /* communicationException */)
{
// Handle the communication exception
((IChannel)_client).Abort();
}
}
}
protected void CloseFactory()
{
if (Factory.State == CommunicationState.Opened)
{
try
{
Factory.Close();
}
catch (TimeoutException /* timeout */)
{
// Handle the timeout exception
Factory.Abort();
}
catch (CommunicationException /* communicationException */)
{
// Handle the communication exception
Factory.Abort();
}
}
}
4) WCF will fault the channel not the factory. You can implement a re-connect logic but that would require that you create and derive your clients from some custom ProxyBase e.g.
protected I Channel
{
get
{
lock (_channelLock)
{
if (! object.Equals(innerChannel, default(I)))
{
ICommunicationObject channelObject = innerChannel as ICommunicationObject;
if ((channelObject.State == CommunicationState.Faulted) || (channelObject.State == CommunicationState.Closed))
{
// Channel is faulted or closing for some reason, attempt to recreate channel
innerChannel = default(I);
}
}
if (object.Equals(innerChannel, default(I)))
{
Debug.Assert(Factory != null);
innerChannel = Factory.CreateChannel();
((ICommunicationObject)innerChannel).Faulted += new EventHandler(Channel_Faulted);
}
}
return innerChannel;
}
}
5) Do not re-use channels. Open, do something, close is the normal usage pattern.
6) Create common proxy base class and derive all your clients from it. This can be helpful, like re-connecting, using pre-invoke/post invoke logic, consuming events from factory (e.g. Faulted, Opening)
7) Create your own CustomChannelFactory this gives you further control how factory behaves e.g. Set default timeouts, enforce various binding settings (MaxMessageSizes) etc.
public static void SetTimeouts(Binding binding, TimeSpan? timeout = null, TimeSpan? debugTimeout = null)
{
if (timeout == null)
{
timeout = new TimeSpan(0, 0, 1, 0);
}
if (debugTimeout == null)
{
debugTimeout = new TimeSpan(0, 0, 10, 0);
}
if (Debugger.IsAttached)
{
binding.ReceiveTimeout = debugTimeout.Value;
binding.SendTimeout = debugTimeout.Value;
}
else
{
binding.ReceiveTimeout = timeout.Value;
binding.SendTimeout = timeout.Value;
}
}
I was trying to create a simple parent child process with IPC between them using Hadoop IPC. It turns out that program executes and prints the results but it doesn't exit. Here is the code for it.
interface Protocol extends VersionedProtocol{
public static final long versionID = 1L;
IntWritable getInput();
}
public final class JavaProcess implements Protocol{
Server server;
public JavaProcess() {
String rpcAddr = "localhost";
int rpcPort = 8989;
Configuration conf = new Configuration();
try {
server = RPC.getServer(this, rpcAddr, rpcPort, conf);
server.start();
} catch (IOException e) {
e.printStackTrace();
}
}
public int exec(Class klass) throws IOException,InterruptedException {
String javaHome = System.getProperty("java.home");
String javaBin = javaHome +
File.separator + "bin" +
File.separator + "java";
String classpath = System.getProperty("java.class.path");
String className = klass.getCanonicalName();
ProcessBuilder builder = new ProcessBuilder(
javaBin, "-cp", classpath, className);
Process process = builder.start();
int exit_code = process.waitFor();
server.stop();
System.out.println("completed process");
return exit_code;
}
public static void main(String...args) throws IOException, InterruptedException{
int status = new JavaProcess().exec(JavaProcessChild.class);
System.out.println(status);
}
#Override
public IntWritable getInput() {
return new IntWritable(10);
}
#Override
public long getProtocolVersion(String paramString, long paramLong)
throws IOException {
return Protocol.versionID;
}
}
Here is the child process class. However I have realized that it is due to RPC.getServer() on the server side that it the culprit. Is it some known hadoop bug, or I am missing something?
public class JavaProcessChild{
public static void main(String...args){
Protocol umbilical = null;
try {
Configuration defaultConf = new Configuration();
InetSocketAddress addr = new InetSocketAddress("localhost", 8989);
umbilical = (Protocol) RPC.waitForProxy(Protocol.class, Protocol.versionID,
addr, defaultConf);
IntWritable input = umbilical.getInput();
JavaProcessChild my = new JavaProcessChild();
if(input!=null && input.equals(new IntWritable(10))){
Thread.sleep(10000);
}
else{
Thread.sleep(1000);
}
} catch (Throwable e) {
e.printStackTrace();
} finally{
if(umbilical != null){
RPC.stopProxy(umbilical);
}
}
}
}
We sorted that out via mail. But I just want to give my two cents here for the public:
So the thread that is not dying there (thus not letting the main thread finish) is the org.apache.hadoop.ipc.Server$Reader.
The reason is, that the implementation of readSelector.select(); is not interruptable. If you look closely in a debugger or threaddump, it is waiting on that call forever, even if the main thread is already cleaned up.
Two possible fixes:
make the reader thread a deamon (not so cool, because the selector
won't be cleaned up properly, but the process will end)
explicitly close the "readSelector" from outside when interrupting the threadpool
However, this is a bug in Hadoop and I have no time to look through the JIRAs. Maybe this is already fixed, in YARN the old IPC is replaced by protobuf and thrift anyways.
BTW also this is platform dependend on the implementation of the selectors, I observed these zombies on debian/windows systems, but not on redhat/solaris.
If anyone is interested in a patch for Hadoop 1.0, email me. I will sort out the JIRA bug in the near future and edit this here with more information. (Maybe this is fixed in the meanwhile anyways).
I want the server to constantly track for available clients using WCF Discovery.
public void Start()
{
findCriteria = new FindCriteria(typeof(ITestRunnerAgent))
{
Scopes = {new Uri(scope)},
Duration = TimeSpan.FromMilliseconds(DiscoveryIntervalInMiliseconds)
};
discoveryClient = GetInitilizedDisoveryClient();
discoveryClient.FindAsync(findCriteria);
}
private DiscoveryClient GetInitilizedDisoveryClient()
{
var client = new DiscoveryClient(new UdpDiscoveryEndpoint());
client.FindProgressChanged += OnFindProgressChanged;
client.FindCompleted += OnFindCompleted;
return client;
}
private void OnFindCompleted(object sender, FindCompletedEventArgs e)
{
if (!e.Cancelled)
{
// HERE! Sometimes e.Error is not null, but as described in question
discoveryClient.FindAsync(findCriteria);
}
}
Unfortunately, sometimes at the point specified by comment i get an aborted Udp channel:
The communication object,
System.ServiceModel.Channels.UdpChannelFactory+ClientUdpDuplexChannel,
cannot be used for communication
because it has been Aborted.
Has anyone ideas why?
It could be that some network infrastructure at your office is droping the connections.
You should write your code to check for aborted communication, and recover from it.
To recover you could close down the aborted channel and create a new one.
Well, this doesn't answer your question, but I feel a little wary about your code. It seems fundamentally correct, but it feels like your discovery could be running very fast. I would implement recurring discovery in a separate thread with some sleep time just to make the network happier. Just a thought to clean up the code. Sorry if this doesn't help.
public void Start()
{
var bw = new System.ComponentModel.BackgroundWorker();
bw.DoWork += new System.ComponentModel.DoWorkEventHandler(DiscoveryThread);
bw.RunWorkerAsync();
}
private void DiscoveryThread(object sender, System.ComponentModel.DoWorkEventArgs e)
{
var client = new DiscoveryClient(new UdpDiscoveryEndpoint());
var findCriteria = new FindCriteria(typeof(ITestRunnerAgent))
{
Scopes = {new Uri(scope)},
Duration = TimeSpan.FromMilliseconds(DiscoveryIntervalInMiliseconds)
};
while(true)
{
client.Find(findCriteria);
// lock, clear, and add discovered endpoints to a global List of some sort
System.Threading.Thread.Sleep(3000);
}
}
As it is asynchronous operation the thread terminates after executing FindAsync(criteria) method. just wrote Console.Readline() after method call or use Autoreset event hold the thread.