Search and Transactions in Sharded Redis - redis

Question

the question refers to the sharded configuration of redis. I have implemented a small test application in Java which creates 100.000 user hashes over Jedis in the form of user:userID. Each hash has the elements: name, phone, department, userID. I also created simple key-value pairs with the key phone:phone number which contains the userID whose phone number is the ID and sets for each department with the userIDs who work for that particular department. The two latter types I use only for seaching. These structures and the search are similar to Searching in values of a redis db.
In short, the data structures:
user:userID->{name, department, phone, userID}
department:department->([userID1, userID2,....])
phone:phone->userID
Use cases for the search:
access to user-hashes based on key i.e. userID
search for users with a phone number
search for all users of a department
Everything works all right in the single instance and sharded configuration but I would have the following questions:
In the single instance configuration it is possible to look for phone
number with a wide card e.g. with the KEY method but this is not
available in the sharded configuration. How would it be possible to
look for keys whose first part is known?
The user ID is generated from a zset whose score is increased for
userID. This can be doen in a transaction for the single instance
configuration but transactions seem not to be supported for sharding
configurations over jedis even if the participating keys are on the
same instance. How would it be possible to solve this problem if
multiple client threads can also do the user creation?
Thank you for your responses also in advance.

Answer 1

For the 1st part of your question:
There is no magic here, if you want to search across all your shards, you have to iterate over all shards. Jedis don't have this method, but you could extend ShardedJedis to add it (untested):
public Set<String> keys(String pattern) {
HashSet<String> found = new HashSet<String>();
for (Jedis jedis : getAllShards()) {
found.addAll(jedis.keys(pattern));
}
return found;
}
For the 2nd part of your question:
AFAIK, Jedis doesn't support transactions when using Shards, even if you do force the related keys to be on the same shard (see Jedis Advanced Usage).
This same link suggest a workaround that may apply for a few scenarios:
Mixed approach
If you want easy load distribution of ShardedJedis, but still need
transactions/pipelining/pubsub etc, you can also mix the normal and
the sharded approach: define a master as normal Jedis, the others as
sharded Jedis. Then make all the shards slaveof master. In your
application, direct your write requests to the master, the read
requests to ShardedJedis. Your writes don't scale anymore, but you
gain good read distribution, and you have
transactions/pipelining/pubsub simply using the master. Dataset should
fit in RAM of master. Remember that you can improve performance of the
master a lot, if you let the slaves do the persistance for the master!