I have a redis queue and node app that is inserting items into the queue. They are just JSON strings. I'm using the bee-queue npm package for this.
For the purposes of debugging, it would be nice if there was way to see the items currently in the queue waiting to be processed using the redis cli?
127.0.0.1:6379> keys *
1) "bq:myqueue:waiting"
2) "bq:myqueue:jobs"
3) "bq:myqueue:id"
I can see there are items in there
127.0.0.1:6379> llen bq:myqueue:waiting
(integer) 2
but I can't seem to actually view them:
127.0.0.1:6379> get bq:myqueue:waiting
(error) WRONGTYPE Operation against a key holding the wrong kind of value
When I use LRANGE to list the data, I get this:
127.0.0.1:6379> LRANGE bq:myqueue:waiting 0 3
1) "9"
2) "8"
3) "7"
Based on the data I'm seeing when using the node.js client to process these items, those are the ids that redis is assigning to the items in the queue, but it's not the JSON data I am actually putting in the queue.
Looks like Adam Marshall had the right idea in checking the type. It turns out that bq:myqueue:jobs is a hash so you can use the hash commands for exploring the contents. For example
127.0.0.1:6379> TYPE bq:myqueue:jobs
hash
127.0.0.1:6379> HGKEYS bq:myqueue:jobs
1) "11"
2) "9"
3) "4"
127.0.0.1:6379> HGET bq:myqueue:jobs "4"
"{"data":{"a":1001, "b": "four"},"options":{"timestamp":1632162180724,"stacktraces":[]},"status":"created","progress":0}"
127.0.0.1:6379> HGETALL bq:myqueue:jobs
1) "11"
2) "{"data":{"a":1, "b": "two"},"options":{"timestamp":1632162260037,"stacktraces":[]},"status":"created","progress":0}"
3) "9"
4) "{"data":{"a":99, "b": "three"},"options":{"timestamp":1632189454151,"stacktraces":[]},"status":"created","progress":0}"
5) "4"
6) "{"data":{"a":1001, "b": "four"},"options":{"timestamp":1632162180724,"stacktraces":[]},"status":"created","progress":0}"
The formatting leaves a bit to be desired but I think that basically works.
Related
I've recently discovered that redis has a property graph model implementation called redis graph and it's amazing.
One thing that I really miss for my use-case though, is the ability to "watch" the data. In typical redis data structures I can enable Keyspace notifications or client tracking and be notified on the data mutations I'm interested in, pull data from the server or mark my local cache as "dirty".
I don't know how that would work for a property graph since relations are much more complex (and the key feature for that matter), but is there a way to watch or synchronize with data stored in redis graph?
Keyspace notifications can be enabled for modules like this:
redis.cloud> CONFIG SET notify-keyspace-events AKE
The 'A' part includes modules—if the module publishes anything. Unfortunately, I tried this with RedisGraph, and it doesn't.
You can reproduce my test below. In one terminal I launched redis-cli and did this:
127.0.0.1:6379> PSUBSCRIBE *
Reading messages... (press Ctrl-C to quit)
1) "psubscribe"
2) "*"
3) (integer) 1
In another I did this:
127.0.0.1:6379> GRAPH.QUERY test 'CREATE (n:foo { alfa: 12, bravo: "test" })'
1) 1) "Labels added: 1"
2) "Nodes created: 1"
3) "Properties set: 2"
4) "Cached execution: 0"
5) "Query internal execution time: 0.204701 milliseconds"
127.0.0.1:6379> GRAPH.QUERY test 'MATCH (n) RETURN n'
1) 1) "n"
2) 1) 1) 1) 1) "id"
2) (integer) 0
2) 1) "labels"
2) 1) "foo"
3) 1) "properties"
2) 1) 1) "alfa"
2) (integer) 12
2) 1) "bravo"
2) "test"
3) 1) "Cached execution: 0"
2) "Query internal execution time: 1.106191 milliseconds"
127.0.0.1:6379> GRAPH..DELETE test
"Graph removed, internal execution time: 0.064498 milliseconds"
The first terminal returned nothing in response to this.
No. Unfortunately, currently, there is no way to trigger a keyspace notification whenever nodes or relationships are created, removed, or updated.
We plan to add such functionality in the future, but there is no specific date we can share right now.
I'm new to redis, but from reading the docs I couldn't find if it is possible to update a value with a query (like mongo or postgres).
For example, I want to execute some logic for players in the lobby. I want to update the user status to lobby only if it was in home.
Applying this logic in the app won't work since I may encounter a race condition between 2 processes.
I would do something like this in mongo:
update({id:1, status:home}, {status: lobby})
This way I will receive the result (success or fail) if the update was successful, and it is atomic.
How can I do it in redis?
There's no update-by-query in Redis, like in other document stores/RDBMSs
You can either go with MULTI-EXEC transactions or EVAL / EVAL SHA to perform atomic updates and compose a batch of commands together
Redis uses a single thread for command processing (as of this moment), so evaluating a script would be performed atomically, and would probably be more straight forward approach than multi-exec.
You can do it in a LUA procedure:
eval "local ks=redis.call('keys','user:*') for k=1,#ks do print(k) if (redis.call('hget',ks[k],'status') == KEYS[1]) then redis.call('hset',ks[k],'status',KEYS[2]) end end" 2 home lobby
127.0.0.1:6370> hset user:1 id 1 (integer) 1
127.0.0.1:6370> hset user:1 status outside (integer) 1
127.0.0.1:6370> hset user:2 id 2 (integer) 1
127.0.0.1:6370> hset user:2 status home (integer) 1
127.0.0.1:6370> eval "local ks=redis.call('keys','user:*') for k=1,#ks do print(k) if (redis.call('hget',ks[k],'status') == KEYS[1]) then
redis.call('hset',ks[k],'status',KEYS[2]) end end" 2 home lobby
(nil)
127.0.0.1:6370> hgetall user:2 1) "id" 2) "2" 3) "status" 4) "lobby"
127.0.0.1:6370> hgetall user:1 1) "id" 2) "1" 3) "status" 4) "outside"
I am pushing data into redis from python like this:
ts = datetime.datetime.now().timestamp()
if msg.field == 2:
seq = [ts, 'ask', msg.price]
r.rpush(contractTuple[0], *seq)
I expect the inserted data (seq) to be one object in redis. However, when I look at the data from the reds-clithe fields of the python list are on separate lines:
127.0.0.1:6379> lrange ES 0 -13
406) "1523994426.496158"
407) "ask"
408) "2699.5"
127.0.0.1:6379>
Is this the way redis-cli displays data (strange if true imo), or am I pushing data into redis incorrectly?
See: http://redis-py.readthedocs.io/en/latest/index.html#redis.StrictRedis.rpush:
rpush(name, *values)
Push values onto the tail of the list name
Redis doesn't have a concept of "objects". If you want these values to be grouped, you'll have to implement your own methods to (de)serialize them into strings.
To add keys to redis I did the following via the redis CLI:
127.0.0.1:6379> KEYS *
1) "key1"
2) "key2"
3) "key3"
127.0.0.1:6379> SET name "rahul"
OK
127.0.0.1:6379> KEYS *
1) "key1"
2) "name"
3) "key2"
4) "key3"
127.0.0.1:6379>
To validate the persistence of the data in my redis data store, I re-started the server, upon checking the keys, I found few keys to be missing :
127.0.0.1:6379> KEYS *
1) "key3"
2) "key2"
3) "key1"
127.0.0.1:6379>
Are there any specific naming conventions for redis keys. I was using a Windows system. Any idea of what has gone wrong. TIA.
If you do a graceful shutdown values will be written to disk before the service is shutdown. If it's a abrupt shutdown or power failure values will be lost. For that you can enable persistance (RDB or AOF). By default redis follows RDB snapshotting, by default it takes snapshot based on three conditions
1) atleast one keys changed for 15 mins.
2) atleast ten keys changed for 5 mins.
3) atleast 10,000 keys changed for 1 min.
You can change these values in redis.conf file under SNAPSHOTTING.
Try reading the redis.conf file fully, it will give you more detailed explanations.
Explored spring session and redis it looks really good.
Trying to solve one question for a long time , how to retrieve list of session token from redis db based on the spring session token value in the hash .
I know its not a relational database and there is no straightforward way to achieve but is that a way to figure this out which is really important for us to solve problems
I read in blogs we need to keep a set to track , are there any ways to acheive this when using spring session. i am not even sure how to do this
Any help is highly appreciated .
Thank you
Useful Commands:
redis-cli : To enter into redis console
Example:
root#root>redis-cli
127.0.0.1:6379> _
keys * :Shows all keys stored in redis DB
Example:
127.0.0.1:6379>keys *
“spring:session:expirations:1440354840000“
“spring:session:sessions:3b606f6d-3d30-4afb-bea6-ef3a4adcf56b“
monitor : To monitor the redis DB
127.0.0.1:6379> monitor
OK
1441273902.701071 [0 127.0.0.1:49137] "PING"
1441273920.000888 [0 127.0.0.1:49137] "SMEMBERS"
hgetall SESSION_ID :To check all the keys stored inside a session
example: :
127.0.0.1:6379>hgetall spring:session:sessions:3b606f6d-3d30-4afb-bea6-ef3a4adcf56b
flushall Remove all keys from the DB.
Example :
127.0.0.1:6379> flushall
ok
Open redis-cli then run
127.0.0.1:6379> keys *
1) "spring:session:expirations:1435594380000"
2) "spring:session:sessions:05adb1d7-c7db-4ffb-99f7-47d7bd1867ee"
127.0.0.1:6379> type spring:session:sessions:05adb1d7-c7db-4ffb-99f7-47d7bd1867ee
hash
127.0.0.1:6379> hgetall spring:session:sessions:05adb1d7-c7db-4ffb-99f7-47d7bd1867ee
1) "sessionAttr:SPRING_SECURITY_CONTEXT"
2) ""
3) "sessionAttr:javax.servlet.jsp.jstl.fmt.request.charset"
4) "\xac\xed\x00\x05t\x00\x05UTF-8"
5) "creationTime"
6) "\xac\xed\x00\x05sr\x00\x0ejava.lang.Long;\x8b\xe4\x90\xcc\x8f#\xdf\x02\x00\x01J\x00\x05valuexr\x00\x10java.lang.Number\x86\xac\x95\x1d\x0b\x94\xe0\x8b\x02\x00\x00xp\x00\x00\x01N?\xfb\xb6\x83"
7) "maxInactiveInterval"
8) "\xac\xed\x00\x05sr\x00\x11java.lang.Integer\x12\xe2\xa0\xa4\xf7\x81\x878\x02\x00\x01I\x00\x05valuexr\x00\x10java.lang.Number\x86\xac\x95\x1d\x0b\x94\xe0\x8b\x02\x00\x00xp\x00\x00\a\b"
9) "lastAccessedTime"
10) "\xac\xed\x00\x05sr\x00\x0ejava.lang.Long;\x8b\xe4\x90\xcc\x8f#\xdf\x02\x00\x01J\x00\x05valuexr\x00\x10java.lang.Number\x86\xac\x95\x1d\x0b\x94\xe0\x8b\x02\x00\x00xp\x00\x00\x01N?\xfb\xb6\xa6"
127.0.0.1:6379>