I'm trying to implement the redis solution suggested here: http://www.slideshare.net/cacois/cois-palkostrata2013: it is the best I have found so far.
I have the following data structure "hash"
{'user': username, 'text': sometext, 'time': 1400543375}
I want to save the data in a log, the slides suggest saving it in a sorted list. When I try this:
zadd mysortedset 1400543375 {'user': username, 'text': sometext, 'time': 1400543375}
I get
Invalid argument(s)
so I thought I should create the hash first and then add it to the set
127.0.0.1:6379> HMSET setmember:1400543375 user username text sometext time 1400543375
OK
and then
127.0.0.1:6379> zadd mysortedset 1400543375 setmember:1400543375
(integer) 1
Is this the way it is supposed to be done?
What’s simply missing are the quotes around your JSON. Otherwithe it tries to interpret your zadd as scove value score value. And „username,“ is no valid score. ;) I tried it with
zadd mysortedset 1234 ”{’user’: foo …}”
and it works.
A log suggest you get the entries in the right order to begin with. In which case use the l* commands (lpush,lpop etc)
Related
I have multiple hash-keys organized in "folders" like MOC273,
127.0.0.1:6379> HMSET MOC273:123654789 "H_W" "json->H_W->123654789" "H_Y" "json->H_Y->123654789"
OK
127.0.0.1:6379> HMSET MOC273:987654321 "H_W" "json->H_W->987654321" "H_Y" "json->H_Y->987654321"
OK
With HGETALL I can get all the data from a hash
127.0.0.1:6379> HGETALL MOC273:123654789
1) "H_W"
2) "json->H_W->123654789"
3) "H_Y"
4) "json->H_Y->123654789"
How can I get all the hashes contained in MOC273(where the name starts with MOC273) Something like.
127.0.0.1:6379> HGETALL MOC273*
It's possible ?
Current structure of the redis
So you can use Redisearch to create a secondary index around the HASH data structure. This will allow you to search through HASH data more efficiently.
https://oss.redis.com/redisearch/ - Quick start should help you to get started
I am learning HyperLogLogs examples with redis-cli
The redis-cli examples show how you can use HyperLogLog commands to record and count unique user visits to a website.
The command PFADD adds one or many strings to a HyperLogLog. PFADD returns 1 if the cardinality was changed and 0 if it remains the same:
Nonetheless, It report error when I follow the instructions:
127.0.0.1:6379> PFADD visits:2015-01-01 "carl" "max" "hugo" "arthur"
(error) WRONGTYPE Key is not a valid HyperLogLog string value.
The error is, even if terse, quite informative - you are trying to use an existing key (i.e. visits:2015-01-01) that is not an HLL.
The existing key is possibly a string of some kind, but if you DEL visits:2015-01-01, you should be able to PFADD to it.
I'm iterating through data and dumping some to a Redis DB. Here's an example:
hmset id:1 username "bsmith1" department "accounting"
How can I increment the unique ID on the fly and then use that during the next hmset command? This seems like an obvious ask but I can't quite find the answer.
Use another key, a String, for storing the last ID. Before calling HMSET, call INCR on that key to obtain the next ID. Wrap the two commands in a MULTI/EXEC block or a Lua script to ensure the atomicity of the transaction.
Like Itamar mentions you can store your index/counter in a separate key. In this example I've chosen the name index for that key.
Python 3
KEY_INDEX = 'index'
r = redis.from_url(host)
def store_user(user):
r.incr(KEY_INDEX, 1) # If key doesn't exist it will get created
index = r.get(KEY_INDEX).decode('utf-8') # Decode from byte to string
int_index = int(index) # Convert from string to int
result = r.set('user::%d' % int_index, user)
...
Note that user::<index> is an arbitrary key chosen by me. You can use whatever you want.
If you have multiple machines writing to the same DB you probably want to use pipelines.
I've been experimenting with Redis today. I've managed to store cached values from Drupal, but I'm looking to investigate a bit further and view the value stored in cache_my_custom_cache..
127.0.0.1:6379> keys *
1) "ff3169bd93659dc31322abc32835ef3e:cache_bootstrap:lookup_cache"
2) "ff3169bd93659dc31322abc32835ef3e:cache_bootstrap:module_implements"
3) "ff3169bd93659dc31322abc32835ef3e:cache_bootstrap:bootstrap_modules"
4) "ff3169bd93659dc31322abc32835ef3e:cache_bootstrap:system_list"
5) "ff3169bd93659dc31322abc32835ef3e:cache_bootstrap:variables"
6) "ff3169bd93659dc31322abc32835ef3e:path:a:und"
7) "myhash"
8) "ff3169bd93659dc31322abc32835ef3e:path:s:und"
9) "ff3169bd93659dc31322abc32835ef3e:cache_my_custom_cache:custom_cache_markup"
10) "ff3169bd93659dc31322abc32835ef3e:cache_bootstrap:hook_info"
127.0.0.1:6379> type ff3169bd93659dc31322abc32835ef3e:cache_my_custom_cache:custom_cache_markup
hash
this reveals it's of type hash.. But when I run:
127.0.0.1:6379> HGET ff3169bd93659dc31322abc32835ef3e:cache_qbe:qbe_markup
(error) ERR wrong number of arguments for 'hget' command
It doesn't like it! Completely new to this, can someone offer a solution?
Besides the key you will need to specify the field in the HGET command.
To get a list of all fields in the hash you can run this:
hkeys ff3169bd93659dc31322abc32835ef3e:cache_my_custom_cache:custom_cache_markup
Then you can do:
hget ff3169bd93659dc31322abc32835ef3e:cache_my_custom_cache:custom_cache_markup FIELD
You can also get all the values in the hash like this:
hvals ff3169bd93659dc31322abc32835ef3e:cache_my_custom_cache:custom_cache_markup
HGET expects an additional parameter after the key name that enumerates which field of your hash you would like returned. Something like this:
HGET my_hash_key my_hash_field
If you're trying to retrieve all fields of your hash at once, you should use HGETALL:
HGETALL my_hash_key
Documentation for HGET: here
Documentation for HGETALL: here
Discussion of Redis types (including hashes): here
I'm currently playing around with Redis and i've got a few questions. Is it possible to get values from an array of keys?
Example:
users:1:name "daniel"
users:1:age "24"
users:2:name "user2"
users:2:age "24"
events:1:attendees "users:1", "users:2"
When i redis.get events:1:attendees it returns "users:1", "users:2". I can loop through this list and get users:1, get users:2. But this feels wrong, is there a way to get all the attendees info on 1 get?!
In rails i would do something like this:
#event.attendees.each do |att|
att.name
end
But in redis i can't because it returns the keys and not the actual object stored at that key.
thanks :)
Doing a loop on the items and synchronously accessing each element is not very efficient. With Redis 2.4, there are various ways to do what you want:
by using the sort command
by using pipelining
by using variadic parameter commands
With Redis 2.6, you can also use Lua scripting, but this is not really required here.
By the way, the data structure you described could be improved by using hashes. Instead of storing user data in separate keys, you could group them in a hash object.
Using the sort command
You can use the Redis sort command to retrieve the data in one roundtrip.
redis> set users:1:name "daniel"
OK
redis> set users:1:age 24
OK
redis> set users:2:name "user2"
OK
redis> set users:2:age 24
OK
redis> sadd events:1:attendees users:1 users:2
(integer) 2
redis> sort events:1:attendees by nosort get *:name get *:age
1) "user2"
2) "24"
3) "daniel"
4) "24"
Using pipelining
The Ruby client support pipelining (i.e. the capability to send several queries to Redis and wait for several replies).
keys = $redis.smembers("events:1:attendees")
res = $redis.pipelined do
keys.each do |x|
$redis.mget(x+":name",x+":age")
end
end
The above code will retrieve the data in two roundtrips only.
Using variadic parameter command
The MGET command can be used to retrieve several data in one shot:
redis> smembers events:1:attendees
1) "users:2"
2) "users:1"
redis> mget users:1:name users:1:age users:2:name users:2:age
1) "daniel"
2) "24"
3) "user2"
4) "24"
The cost here is also two roundtrips. This works if you can guarantee that the number of keys to retrieve is limited. If not, pipelining is a much better solution.
You can use Redis' EVAL command to send it a Lua script that runs a loop "server side" and return the results in a block.