How do I push integers with lpush in a redis list type?
I want to test my finagle-redis client if it works correctly and insert manual sample data into redis like this
127.0.0.1:6379> rpush key:214 1 1 1
(integer) 3
127.0.0.1:6379> LRANGE key:214 0 -1
1) "1"
2) "1"
3) "1"
Redis already displays the numbers as char. When I extract them, I also get chars:
val data: List[ChannelBuffer] = Await.result(redisClient.lRange(key, 0, -1))
val buffer: ChannelBuffer = data(0)
buffer.readChar() // 1
buffer.readInt() // 49
Is writing integers to a list possible with the cli client? And if not, will the following even work?
val key = ChannelBuffers.copiedBuffer("listkey", utf8)
val list: List[ChannelBuffer] = List(1,1,1).map { number =>
val buffer = ChannelBuffers.buffer(4) // int sized buffer
buffer.writeInt(number)
buffer
}
// will this store the int's correctly??
redisClient.lpush(key, list)
In redis most everything is a string. This is because of the protocol used. It's text-based, so it's impossible to tell number 1234 from string "1234".
It might be storing integers internally, but you'll get strings anyway. You should cast your numbers in the app.
Related
I want to keep some user feedbacks in redis. Some users may give multiple feedbacks. The users are assigned numerical user id
Here is an example:
zadd feedbacks 1 feedback1 2 feedback2 3 feedback3 1 feedback4
In this case user #1 gave feedbacks feedback1 and feedback4, #2 feedback2 and #3 feedback3.
If I use ZRANGEBYSCORE feedbacks 1 1
I will be able to see the feedback from user #1:
1) "feedback1"
2) "feedback4"
However I want to store more than just the text. I want to be able to retrieve the timestamp for example. Is there any way I can insert a hash vale to the key feedbacks above?
Something likes zadd feedbacks 1 text:feedback1 timestamp:123456
No, you cannot. Lists, sets, hashes and sorted sets only support Redis' string data type for the values.
You can stringify your field-value pairs though, using JSON or your preferred format.
ZADD feedbacks 1 "{\"text\":\"feedback1\",\"timestamp\":\"123456\""
Unless you need to modify a given field atomically, this approach should do.
And even in that case, you can use Lua scripts to achieve server-side JSON manipulation and updates. See How to nest a list into a structure in Redis to reduce top level? for a similar solution.
Secondary indexes
But you may want to query multiple ways: by user id, by timestamp, etc.
In that case, consider using regular keys to store the feedback object, say as a hash.
HSET feedbacks:feedback1 text feedback1 timestamp 123456 user 1 ...
And your indexes:
ZADD feedbacks-by-user 1 feedback1
ZADD feedbacks-by-timestamp 123456 feedback1
...
Say you want all feedbacks of a given user:
ZRANGEBYSCORE feedbacks-by-user 1 1
Then you go get the values for the returned keys. Of course, you may want to avoid the two round trips. Again, Lua script.
The script:
local keys = redis.call('ZRANGEBYSCORE', KEYS[1], ARGV[1], ARGV[1])
for i, v in ipairs(keys) do
local k = {}
k[1] = v
k[2] = redis.call('HGETALL', 'feedbacks:'..v)
keys[i] = k
end
return keys
Usage:
> EVAL "local keys = redis.call('ZRANGEBYSCORE', KEYS[1], ARGV[1], ARGV[1]) \n for i, v in ipairs(keys) do \n local k = {} \n k[1] = v \n k[2] = redis.call('HGETALL', 'feedbacks:'..v) \n keys[i] = k \n end \n return keys" 1 feedbacks-by-user 1
1) 1) "feedback1"
2) 1) "text"
2) "feedback1"
3) "timestamp"
4) "123456"
5) "user"
6) "1"
2) 1) "feedback4"
2) 1) "text"
2) "feedback4"
3) "timestamp"
4) "465465"
5) "user"
6) "1"
You can similarly query for a range of timestamps.
You can mix and match your queries using ZINTERSTORE or ZUNIONSTORE.
You may be interested in How to store in Redis sorted set with server-side timestamp as score?. You can make a nice Lua script to take care of creating the hash, and the secondary index entries, all in one go with redis-server-side timestamping.
Finally, whenever using Lua on Redis, consider to load the script and use EVALSHA.
I got several sorted sets with a common prefix (itemmovements:) in Redis.
I know we can use ZCOUNT to get the number of items for a single (sorted set) key like this:
127.0.0.1:6379> zcount itemmovements:8 0 1000000000
(integer) 23
(I am able to do this, since I know the range of the item scores.)
How to run this in a loop for all keys prefixed itemmovements:?
Taking hint from How to atomically delete keys matching a pattern using Redis I tried this:
127.0.0.1:6379> EVAL "return redis.call('zcount', unpack(redis.call('keys', ARGV[1])), 0, 1000000000)" 0 itemmovements:*
(integer) 150
but as you can see it just returns a single number (which happens to be the size of itemmovements:0, the first value returned by keys).
I realized I did not understand what that lua code in EVAL was doing. The code below works fine:
eval "local a = {}; for _,k in ipairs(redis.call('keys', 'itemmovements:*')) do table.insert(a, k); table.insert(a, redis.call('zcount', k, 0, 1000000000)); end; return a" 0
I have around 3 million keys of structure
with following format as an example
3000000:60
Each key has a hset of string converted values of integer and double
Each key has around 10 such entries in map.
Key: String = 3000000:60
Value : field(String) = abcc , Value = 1000:-123.456:1001:234.57:.....
When I read the data using hgetAll or hget or hmget it takes 2-3 minutes
When I change it to pipeline , I could finish in 20 seconds but I am not sure how to process the data in batches.
My code snippet is like below
readPipeline.multi();
for (long i = start; i < end; ++i)
if (pipelineBatch == currentBatchCount)
readPipeline.exec();
List<Object> allObjects = readPipeline.syncAndReturnAll();
Here when I read sometimes the Objects have value as Queued. So, how do I process the pipelined data?
I have an NS_OPTION that I'm defining as such :
typedef NS_OPTIONS(NSInteger, PermittedSize) {
SmallSize = 1 << 0,
MediumSize = 1 << 1,
LargeSize = 1 << 2
};
And later I set the values I need :
PermittedSize size = SmallSize | MediumSize;
I'm using it to randomly generate an various objects of small and medium sizes(duh) for a particular level of a game.
What is the best way to go about selecting which size of an object to generate? Meaning, I'd like to choose randomly for each object I'm generating whether it will be one of the 2 options allowed (small and medium in this case). Normally I would use an arc4random function with the range of numbers I need - but in this case, how can it be done with bits? (and then mapped back to the values of the PermittedSize type?
Use the result from arc4random to determine the amount of bit shifting you want to do. Something like this:
int bitShiftAmount = arc4random_uniform(numberOfPermittedSizes);
PermittedSize size = 1 << bitShiftAmount;
You are still working with integers. SmallSize is 1. MediumSize is 2. And LargeSize is 4.
So pick a random number from 1 to 3. 1 is small, 2 is medium, 3 is both.
Once you have a random number, assign it.
NSInteger val = arc4random_uniform(3) + 1; // give 1-3
PermittedSize size = (PermittedSize)val;
I'm planning to start using hashes insead of regular keys. But I can't find any information about multi get for hash-keys in Redis wiki. Is this kind of command is supported by Redis?
Thank you.
You can query hashes or any keys in pipeline, i.e. in one request to your redis instance. Actual implementation depends on your client, but with redis-py it'd look like this:
pipe = conn.pipeline()
pipe.hgetall('foo')
pipe.hgetall('bar')
pipe.hgetall('zar')
hash1, hash2, hash3 = pipe.execute()
Client will issue one request with 3 commands. This is the same technique that is used to add multiple values to a set at once.
Read more at http://redis.io/topics/pipelining
No MHGETALL but you can Lua it:
local r = {}
for _, v in pairs(KEYS) do
r[#r+1] = redis.call('HGETALL', v)
end
return r
If SORT let you use multiple GETs with the -> syntax, and all your hashes had the same fields, you could get them in a bulk reply by putting their names into a set and sorting that.
SORT names_of_hashes GET *->field1 *->field2 *->field3 *->etc
But it doesn't look like you can do that with the hash access. Plus you'd have to turn the return list back into hashes yourself.
UPDATE: Redis seems to let you fetch multiple fields if you name your hashes nicely:
redis> hset hash:1 name fish
(integer) 1
redis> hset hash:2 name donkey
(integer) 1
redis> hset hash:3 name horse
(integer) 1
redis> hset hash:1 type fish
(integer) 1
redis> hset hash:2 type mammal
(integer) 1
redis> hset hash:3 type mammal
(integer) 1
redis> sadd animals 1
(integer) 1
redis> sadd animals 2
(integer) 1
redis> sadd animals 3
(integer) 1
redis> sort animals get # get hash:*->name get hash:*->type
1. "1"
2. "fish"
3. "fish"
4. "2"
5. "donkey"
6. "mammal"
7. "3"
8. "horse"
9. "mammal"
There is no command to do it on one shot, but there is a way to do it "nicely", using a list (or sorted set) where you would store you hashKeys, and then retrieve them as bulk using multi.
In PHP:
$redis->zAdd("myHashzSet", 1, "myHashKey:1");
$redis->zAdd("myHashzSet", 2, "myHashKey:2");
$redis->zAdd("myHashzSet", 3, "myHashKey:3");
$members = $redis->zRange("myHashzSet", 0, -1);
$redis->multi();
foreach($members as $hashKey) {
$redis->hGetAll($hashKey);
}
$results = $redis->exec();
I recommand using a sorted set, where you use the score as an ID for your hash, it allows to take advantages of all score based command.
Redis has a HMGET command, which returns the values of several hash keys with one command.