JedisMovedDataException (When using sadd with pipeline) - redis

Question

I have a 3 node EC2 redis cluster setup and I am trying to add records to redis (using sadd) with pipeline mode.
I get the following error after adding about 70/82 and 81 keys in 3 nodes:
Exception in thread "main" redis.clients.jedis.exceptions.JedisMovedDataException: MOVED 1539 172.31.59.103:6379
at redis.clients.jedis.Protocol.processError(Protocol.java:93)
at redis.clients.jedis.Protocol.process(Protocol.java:122)
at redis.clients.jedis.Protocol.read(Protocol.java:191)
at redis.clients.jedis.Connection.getOne(Connection.java:258)
at redis.clients.jedis.ShardedJedisPipeline.sync(ShardedJedisPipeline.java:44)
at org.hu.e63.MovieLens21MPipeline.push(MovieLens21MPipeline.java:47)
at org.hu.e63.MovieLens21MPipeline.main(MovieLens21MPipeline.java:53
I have looked at this thread, my code pretty much looks like that:
Input file is from here(ratings.csv): http://files.grouplens.org/datasets/movielens/ml-latest-small.zip
Here's the code:
import java.io.File;
import java.util.ArrayList;
import java.util.List;
import java.util.Scanner;
import redis.clients.jedis.JedisShardInfo;
import redis.clients.jedis.ShardedJedis;
import redis.clients.jedis.ShardedJedisPipeline;
public class MovieLens21MPipeline {
ShardedJedis jedis;
public MovieLens21MPipeline() {
JedisShardInfo si = new JedisShardInfo("172.31.59.103", 6379, 5000);
List<JedisShardInfo> list = new ArrayList<JedisShardInfo>();
list.add(si);
list.add(new JedisShardInfo("172.31.59.104", 6379, 5000));
list.add(new JedisShardInfo("172.31.59.105", 6379, 5000));
jedis = new ShardedJedis(list);
}
public void push() {
ShardedJedisPipeline pipeline = jedis.pipelined();
Scanner s;
try {
s = new Scanner(new File("input/ratings.csv"));
StringBuilder key = new StringBuilder();
String s1 = s.nextLine(); // Skip first line
while (s.hasNextLine()) {
s1 = s.nextLine();
String[] spl = s1.split(",");
// key="u:"+spl[0]+":m";
key.append("u:").append(spl[0]).append(":m");
pipeline.sadd(key.toString(), spl[1]);
key.setLength(0);
}
} catch (Exception e) {
// TODO Auto-generated catch block
System.out.println("bye" + e.getMessage());
}
pipeline.sync();
}
public static void main(String[] args) {
MovieLens21MPipeline obj = new MovieLens21MPipeline();
long startTime = System.currentTimeMillis();
obj.push();
long endTime = System.currentTimeMillis();
double d = 0.0;
d = (double) (endTime - startTime);
System.out.println("Throughput: " + d);
}
}
Any help appreciated.

Answer 1

MOVED indicates that you're using Redis Cluster. ShardedJedis is not for Redis Cluster, so you should use JedisCluster instead.
Please note that JedisCluster doesn't have pipeline mode, so you may want to send your operation one by one.
Hope this helps.

Answer 2

Just like Lim said, you are using Redis in cluster mode. Use JedisCluster can solve this problem. In addition, as a supplement, why JedisCluster works?
We can known that from JedisCluster source code.
private T runWithRetries(byte[] key, int attempts, boolean tryRandomNode, boolean asking) {
if (attempts <= 0) {
throw new JedisClusterMaxRedirectionsException("Too many Cluster redirections?");
}
Jedis connection = null;
try {
// omit ... get connection
return execute(connection);
} catch (JedisNoReachableClusterNodeException jnrcne) {
throw jnrcne;
} catch (JedisConnectionException jce) {
// release current connection before recursion
// omit
return runWithRetries(key, attempts - 1, tryRandomNode, asking);
} catch (JedisRedirectionException jre) {
// if MOVED redirection occurred,
if (jre instanceof JedisMovedDataException) {
// here is the key point
// it rebuilds cluster's slot cache
// recommended by Redis cluster specification
// Jedis or ShardedJedis won't run this code
this.connectionHandler.renewSlotCache(connection);
}
// omit ...
return runWithRetries(key, attempts - 1, false, asking);
} finally {
releaseConnection(connection);
}
}