When I directly run the below GET command in my redis cloud,
GET 1000:125:1603875000
I am getting error
WRONGTYPE Operation against a key holding the wrong kind of value redis
When I check
type 1000:125:1603875000
gives me
Hash
But if I execute SET before Get, like this
SET 1000:125:1603875000 11
I get "11" on executing GET command.
Why does the string is considered as Hash? How can I execute GET with the specified string.
That's because when you RUN 'SET 1000:125:1603875000' to 11, you are overwriting the initial '1000:125:1603875000' which was a hash and once you set '1000:125:1603875000' as '11' you can run a GET command to get the value of the key.
To get the value of a redis hash you can run HGETALL to get all the values in the hash or HGET KEYNAME to get a specific key of the hash.
To illustrate the use of these commands:
127.0.0.1:6379> HSET employee name Ankit
(integer) 1
127.0.0.1:6379> GET employee
(error) WRONGTYPE Operation against a key holding the wrong kind of value
127.0.0.1:6379> HGETALL employee
1) "name"
2) "Ankit"
127.0.0.1:6379> HGET employee name
"Ankit"
127.0.0.1:6379> SET employee Ankit
OK
127.0.0.1:6379> GET employee
"Ankit"
127.0.0.1:6379>
I'd like to periodically scan thru a redis instance for keys that changed since the last scan. in between the scans i don't want to process the keys.
eg one key could get a thousand updates between scans. i care for the most recent value only when doing the next periodic scan.
There is no built-in way in Redis to achieve that (yet).
You could, for example, recode your app and add some sort of a way to track updates. For example, wherever you're calling SET foo bar, also call ZADD updated <timestamp> foo. Then, you can use the 'updated' Sorted Set to retrieve updated keys.
Alternatively, you can try using RedisGears to automate the tracking part (for starters). Assuming that you have RedisGears running (i.e. docker run -it -p 6379:6379 redislabs/redisgears), you can do something like the following:
$ cat gear.py
def addToUpdatedZset(x):
import time
now = time.time()
execute('ZADD', 'updated', now, x['key'])
return x
GB().filter(lambda x: x['key'] != 'updated').foreach(addToUpdatedZset).register('*')
$ redis-cli RG.PYEXECUTE "$(cat gear.py)"
OK
$ redis-cli
127.0.0.1:6379> KEYS *
(empty list or set)
127.0.0.1:6379> SET foo bar
OK
127.0.0.1:6379> KEYS *
1) "updated"
2) "foo"
127.0.0.1:6379> ZRANGE updated 0 -1 WITHSCORES
1) "foo"
2) "1559339877.1392548"
127.0.0.1:6379> SET baz qux
OK
127.0.0.1:6379> KEYS *
1) "updated"
2) "baz"
3) "foo"
127.0.0.1:6379> ZRANGE updated 0 -1 WITHSCORES
1) "foo"
2) "1559339877.1392548"
3) "baz"
4) "1559339911.5493586"
In Redis, keys user* will print all keys starting with user.
For example:
keys user*
1) "user2"
2) "user1"
Now, I want all keys that don't start with user to be printed.
How could I do that?
IMPORTANT: always use SCAN instead of (the evil) KEYS
Redis' pattern matching is somewhat functionally limited (see the implementation of stringmatchlen in util.c) and does not provide that which you seek ATM. That said, consider the following possible routes:
Extend stringmatchlen to match your requirements, possibly submitting it as a PR.
Consider what you're trying to do - fetching a subset of keys is always going to be inefficient unless you index them, consider tracking the names of all non-user keys (i.e.g. in a Redis Set) instead.
If you are really insistent on scanning the entire keyspace and match against negative patterns, one way to accomplish that is with a little bit of Lua magic.
Consider the following dataset and script:
127.0.0.1:6379> dbsize
(integer) 0
127.0.0.1:6379> set user:1 1
OK
127.0.0.1:6379> set use:the:force luke
OK
127.0.0.1:6379> set non:user a
OK
Lua (save this as scanregex.lua):
local re = ARGV[1]
local nt = ARGV[2]
local cur = 0
local rep = {}
local tmp
if not re then
re = ".*"
end
repeat
tmp = redis.call("SCAN", cur, "MATCH", "*")
cur = tonumber(tmp[1])
if tmp[2] then
for k, v in pairs(tmp[2]) do
local fi = v:find(re)
if (fi and not nt) or (not fi and nt) then
rep[#rep+1] = v
end
end
end
until cur == 0
return rep
Output - first time regular matching, 2nd time the complement:
foo#bar:~$ redis-cli --eval scanregex.lua , "^user"
1) "user:1"
foo#bar:~$ redis-cli --eval scanregex.lua , "^user" 1
1) "use:the:force"
2) "non:user"
#Karthikeyan Gopall you nailed it in your comment above and this saved me a bunch of time. Thanks!
Here's how you can use it in various combinations to get what you want:
redis.domain.com:6379[1]> set "hello" "foo"
OK
redis.domain.com:6379[1]> set "hillo" "bar"
OK
redis.domain.com:6379[1]> set "user" "baz"
OK
redis.domain.com:6379[1]> set "zillo" "bash"
OK
redis.domain.com:6379[1]> scan 0
1) "0"
2) 1) "zillo"
2) "hello"
3) "user"
4) "hillo"
redis.domain.com:6379[1]> scan 0 match "[^u]*"
1) "0"
2) 1) "zillo"
2) "hello"
3) "hillo"
redis.domain.com:6379[1]> scan 0 match "[^u^z]*"
1) "0"
2) 1) "hello"
2) "hillo"
redis.domain.com:6379[1]> scan 0 match "h[^i]*"
1) "0"
2) 1) "hello"
According to redis keys documentation the command supports glob style patterns, not regular expressions.
and if you look at the documentation, you'll see that the "!" character is not special as opposites to regular expressions.
Here is a simple test I ran in my own db:
redis 127.0.0.1:6379> set a 0
OK
redis 127.0.0.1:6379> set b 1
OK
redis 127.0.0.1:6379> keys *
1) "a"
2) "b"
redis 127.0.0.1:6379> keys !a
(empty list or set) // I expected "b" here
redis 127.0.0.1:6379> keys !b
(empty list or set) // I expected "a" here
redis 127.0.0.1:6379> keys [!b]
1) "b"
redis 127.0.0.1:6379> keys [b]
1) "b"
redis 127.0.0.1:6379> keys [ab]
1) "a"
2) "b"
redis 127.0.0.1:6379> keys ![b]
(empty list or set)
So I just don't think what you are trying to achieve is possible via the keys command.
Besides, the keys command is not very suitable for production environment as it locks your whole redis database.
I would recommend getting all the keys with the scan command, store them locally, and then remove them using LUA
Here's a trick to achieve this with native redis commands (no need for Lua scripts or anything).
If you are able to control the timing of when you insert the new keys (the ones you want to keep, deleting all other stuff like in your question), you can:
Before setting the new keys, set the expiration to all existing keys (by pattern or everything) to expire right now (see how)
Load the new keys
Redis will automatically delete all the older keys and you will be left just with the new ones you want.
You also can print all keys and pass it to grep. For example:
redis-cli -a <password> keys "*" | grep -v "user"
I have a table with columns of data that use in SQL to sort a list of events by Id. We use this to show activity feeds of these events on our site but would like to move all this to redis. We've been over the many ways to do something like this, but I want to bounce them off the community here to get some feed back. Here is the current colum.
There will be another hash table with all the events data in it for sure. However where things get sticky is the timing, and mulitple sorts needed. The first issue is that event in our system are both future, current, and past. So we need to be able to sort by taking the current time and seeing what is happing now, in the future, and already done. However on top of that we have other colums to sort by as well. The multi sorting is where things really get hairy. Here are some of the multi sorts:
Is Feature? (boolean) = Is it a featured event, if so it should display at the top of our feed regardless of time.
Is it happening now? (unixtimestamp) = Check against current time and get list of all things going on now and list IDs.
Is it in the past? (unixtimestamp) = Check current time to see if its already done and list IDs
Is it in the future (unixtimestamp) = Check current time to see if its happening later and list IDs
Things get even more crazy when you take into account that events belong to other users. So we also have to take these ideas and filter them out by USER ID and then sort them. We dont like the idea of having thousands of sorted sets for each user either. So how would you guys go about a complex stream like this?
Visualizing relational data in Redis is not straight forward. Like in the example in discussion storing data with different sorting options is a challenge. Solution possible might not look efficient at 1st instance but performance testing should be done to arrive on some final conclusion.
My suggestion for implementing such solution would:
Every column should be mapped to a LIST in Redis. LIST is selected because duplicate entries are possible, else SET can be used.
In above example we can have following LISTs:
ID, ACTIVITY_TYPE, ACTIVITY_ID, LISTING_TYPE, LISTING_ID, IS_PUBLIC, CREATED_AT, UPDATED_AT, ACTIVITY_AT
DATA IS STORED in these lists in following manner:
//first row
//Column ID
$ redis-cli lpush ID 10730268
1
//If we want to search by ID then create a separate HASH for ID
$ redis-cli HSET ID_HASH 10730268 1
1
//If we want to search by sorted ID then create a separate LIST which stores IDs
$ redis-cli lpush SORTED_ID 10730268
//Column ACTIVITY_TYPE
$ redis-cli lpush ACTIVITY_TYPE "created_event_highlight"
1
//Column ACTIVITY_ID
$ redis-cli lpush ACTIVITY_ID 707003
1
//Column LISTING_TYPE
$ redis-cli lpush LISTING_TYPE "CHANNEL"
1
//Column LISTING_ID
$ redis-cli lpush LISTING_ID 51670
1
//Column IS_PUBLIC
$ redis-cli lpush IS_PUBLIC 1
1
//Column CREATED_AT
$ redis-cli lpush CREATED_AT "2013-04-21 13:34:09"
1
//If we want to search by sorted CREATED_AT then create a separate
//LIST which maintains for CREATED_AT
$ redis-cli lpush SORTED_CREATED_AT "2013-04-21 13:34:09"
1
//If we want to search by CREATED_AT then create a separate HASH
$ redis-cli HSET CREATED_AT_HASH "2013-04-21 13:34:09" 1
1
//Column UPDATED_AT
$ redis-cli lpush UPDATED_AT "2013-04-02 11:34:09"
1
//Column ACTIVITY_AT
$ redis-cli lpush ACTIVITY_AT "2014-01-01 03:04:01"
1
The HASH ID_HASH is used to search data using ID as KEY. If data is to be retrieved using SORTED ID's then, SORTED_ID list could be sort using SORT command. ID_HASH could be searched for getting index using sorted IDs returned from SORTED_ID. Using this INDEX we can get values of all the columns.
To fetch data you can do something like :
$ redis-cli hget ID_HASH 10730268
1
Using these to values as index query columns as
$redis-cli lindex ID 1
10730268
$redis-cli lindex ACTIVITY_TYPE 1
"created_event_highlight"
$redis-cli lindex ACTIVITY_ID 1
707003
$redis-cli lindex LISTING_TYPE 1
"CHANNEL
$redis-cli lindex IS_PUBLIC 1
1
$redis-cli lindex CREATED_AT 1
"2013-04-21 13:34:09"
$redis-cli lindex UPDATED_AT 1
"2013-04-02 11:34:09"
$redis-cli lindex ACTIVITY_AT 1
"2014-01-01 03:04:01"
Similarly if data is to be retrieved using SORTED CREATED_AT then, SORTED_CREATED_AT list could be sort using SORT command. CREATED_AT returned could be searched in HASH CREATED_AT_HASH for getting index. Using this INDEX we can get values of all the columns. This is assuming id time-stamps are unique. If time-stamps are common then we would have to create separate LISTS for each time-stamps and store all Indexes mapped to a time-stamp.
Other approach you could take is map your data to be stored to a datastructre and create its objects. Store these created objects in serialized string form in REDIS against IDs or timestamps.
There does not exist a straight forward way for mapping relational databases to REDIS. May be you could explore other NoSql options. But if REDIS is your then try above approaches or innovate using possible data structures(string, list, set, hash) in REDIS.
===========================================
SERIALIZED OBJECTS
Considering issues that above solution would have, creating a datatstructre which maps the given data and storing them in string serialized format against different keys seems more reasonable.
(An example of C++ struct)
struct EventInformation{
friend class boost::serialization::access;
long id; //Primary key, you could have primary key as
//combination of members, accordingly KEY in
//HASH TABLE and values being stored in LISTS would change
long activityId;
long listingId;
bool isPublic;
std::string activityType;
std::string listingType;
std::string createdAt;
std::string updatedAt;
std::string activityAt;
template<class Archive>
void serialize(Archive &ar, const unsigned int version){
ar & id
& activityId
& listingId
& isPublic
& activityType
& listingType
& createdAt
& updatedAt
& activityAt;
};
In REDIS we can store the serialized object of above struct against ID in a HASH table.
Create a separate LISTS for CREATED_AT (i.e. the name of LIST would be like "2013-04-21 13:34:09") which store LIST of IDs which are mapped to them. This means if there are n timestamps we will have n LISTS.
//created at: we append string "created" in beginning of
//List names to segregate UPDATE, CREATE and ACTIVITY,
//although in given example all timestamps are same for a single row
//I think they might be different as well hence the addition of string literal
$ redis-cli lpush "created-2013-04-21 13:34:09" 10730268
$ redis-cli lpush "created-2013-04-21 13:34:09" 10730267
$ redis-cli lpush "created-2013-04-21 13:40:26" 10730266
$ redis-cli lpush "created-2013-04-21 13:40:26" 10730265
$ redis-cli lpush "created-2013-04-21 13:38:41" 10730264
$ redis-cli lpush "created-2013-04-21 13:38:41" 10730263
All the names of CREATED_AT timestamp LIST are stored in master LIST called CREATED_AT_LIST. This enables us to sort the CREATED_AT timetstamps.
$ redis-cli lpush CREATED_AT_LIST "2013-08-23 09:34:09"
$ redis-cli lpush CREATED_AT_LIST "2013-08-23 09:34:09"
$ redis-cli lpush CREATED_AT_LIST "2013-08-23 09:40:26"
$ redis-cli lpush CREATED_AT_LIST "2013-08-23 09:40:26"
$ redis-cli lpush CREATED_AT_LIST "2013-08-23 09:38:41"
$ redis-cli lpush CREATED_AT_LIST "2013-08-23 09:38:41"
Above LIST has to be retrieved in client code and strings have to be converted to
time format and then sort.
Similarly LISTS of UPDATED_AT and ACTIVITY_AT could be created which will have LIST of IDs which are mapped to them. Also their master lists created.
//update at
$ redis-cli lpush "updated-2013-04-21 09:44:19" 10730268
$ redis-cli lpush "updated-2013-04-21 09:44:19" 10730267
$ redis-cli lpush "updated-2013-04-21 09:40:26" 10730266
$ redis-cli lpush UPDATED_AT_LIST "2013-08-23 09:44:19"
$ redis-cli lpush UPDATED_AT_LIST "2013-08-23 09:44:19"
$ redis-cli lpush UPDATED_AT_LIST "2013-08-23 09:40:26"
//activity at
$ redis-cli lpush "activity-2013-04-21 09:44:19" 10730265
$ redis-cli lpush "activity-2013-04-21 09:44:19" 10730264
$ redis-cli lpush "activity-2013-04-21 09:40:26" 10730263
$ redis-cli lpush ACTIVITY_AT_LIST "2013-08-23 09:44:19"
$ redis-cli lpush ACTIVITY_AT_LIST "2013-08-23 09:44:19"
$ redis-cli lpush ACTIVITY_AT_LIST "2013-08-23 09:40:26"
The master lists created could be sorted and ID could be retrieved accordingly. Once we have the ID we can query the HASH table to get back the serialized object which could be de-serialized.
I would try adding a working example of above.
Is there a Redis command for fetching all keys in the database? I have seen some python-redis libraries fetching them. But was wondering if it is possible from redis-client.
Try to look at KEYS command. KEYS * will list all keys stored in redis.
EDIT: please note the warning at the top of KEYS documentation page:
Time complexity: O(N) with N being the number of keys in the database, under the assumption that the key names in the database and the given pattern have limited length.
UPDATE (V2.8 or greater): SCAN is a superior alternative to KEYS, in the sense that it does not block the server nor does it consume significant resources. Prefer using it.
Updated for Redis 2.8 and above
As noted in the comments of previous answers to this question, KEYS is a potentially dangerous command since your Redis server will be unavailable to do other operations while it serves it. Another risk with KEYS is that it can consume (dependent on the size of your keyspace) a lot of RAM to prepare the response buffer, thus possibly exhausting your server's memory.
Version 2.8 of Redis had introduced the SCAN family of commands that are much more polite and can be used for the same purpose.
The CLI also provides a nice way to work with it:
$ redis-cli --scan --pattern '*'
It can happen that using redis-cli, you connect to your remote redis-server, and then the command:
KEYS *
is not showing anything, or better, it shows:
(empty list or set)
If you are absolutely sure that the Redis server you use is the one you have the data, then maybe your redis-cli is not connecting to the Redis correct database instance.
As it is mentioned in the Redis docs, new connections connect as default to the db 0.
In my case KEYS command was not retrieving results because my database was 1. In order to select the db you want, use SELECT.
The db is identified by an integer.
SELECT 1
KEYS *
I post this info because none of the previous answers was solving my issue.
-->Get all keys from redis-cli
-redis 127.0.0.1:6379> keys *
-->Get list of patterns
-redis 127.0.0.1:6379> keys d??
This will produce keys which start by 'd' with three characters.
-redis 127.0.0.1:6379> keys *t*
This wil get keys with matches 't' character in key
-->Count keys from command line by
-redis-cli keys * |wc -l
-->Or you can use dbsize
-redis-cli dbsize
Get All Keys In Redis
Get all keys using the --scan option:
$ redis-cli --scan --pattern '*'
List all keys using the KEYS command:
$ redis-cli KEYS '*'
Take a look at following Redis Cheat Sheet.
To get a subset of redis keys with the redis-cli i use the command
KEYS "prefix:*"
Yes, you can get all keys by using this
var redis = require('redis');
redisClient = redis.createClient(redis.port, redis.host);
redisClient.keys('*example*', function (err, keys) {
})
SCAN doesn't require the client to load all the keys into memory like KEYS does. SCAN gives you an iterator you can use. I had a 1B records in my redis and I could never get enough memory to return all the keys at once.
Here is a python snippet to get all keys from the store matching a pattern and delete them:
import redis
r = redis.StrictRedis(host='localhost', port=6379, db=0)
for key in r.scan_iter("key_pattern*"):
print key
redis-cli -h <host> -p <port> keys *
where * is the pattern to list all keys
KEYS pattern
Available since 1.0.0.
Time complexity: O(N) with N being the number
of keys in the database, under the assumption that the key names in
the database and the given pattern have limited length.
Returns all keys matching pattern.
Warning : This command is not recommended to use because it may ruin performance when it is executed against large databases instead of KEYS you can use SCAN or SETS.
Example of KEYS command to use :
redis> MSET firstname Jack lastname Stuntman age 35
"OK"
redis> KEYS *name*
1) "lastname"
2) "firstname"
redis> KEYS a??
1) "age"
redis> KEYS *
1) "lastname"
2) "age"
3) "firstname"
In order to get all the keys available in redis server, you should open redis-cli and type:
KEYS *
In order to get more help please visit this page:
This Link
If your redis is a cluster,you can use this script
#!/usr/bin/env bash
redis_list=("172.23.3.19:7001,172.23.3.19:7002,172.23.3.19:7003,172.23.3.19:7004,172.23.3.19:7005,172.23.3.19:7006")
arr=($(echo "$redis_list" | tr ',' '\n'))
for info in ${arr[#]}; do
echo "start :${info}"
redis_info=($(echo "$info" | tr ':' '\n'))
ip=${redis_info[0]}
port=${redis_info[1]}
echo "ip="${ip}",port="${port}
redis-cli -c -h $ip -p $port set laker$port '湖人总冠军'
redis-cli -c -h $ip -p $port keys \*
done
echo "end"
For the ones that wants a typescript helper (using ioredis)
import Redis from 'ioredis';
import { from, Observable, of } from 'rxjs';
import { first, mergeMap } from 'rxjs/operators';
export function scanKeysFromRedis(redisStore: Redis.Redis, key: string,
target: number = 0, keys: string[] = []): Observable<string[]> {
return from(redisStore.scan(target, 'MATCH', key)).pipe(
first(),
mergeMap((_keys) => {
const _target = Number(_keys[0]);
if (_target !== 0) {
return scanKeysFromRedis(redisStore, key, _target, [...keys, ..._keys[1]]);
}
return of([...keys, ..._keys[1]]);
}),
);
}
and call it with: scanKeysFromRedis(store, 'hello');
If you are using Laravel Framework then you can simply use this:
$allKeyList = Redis::KEYS("*");
print_r($allKeyList);
In Core PHP:
$redis = new Redis();
$redis->connect('hostname', 6379);
$allKeyList = $redis->keys('*');
print_r($allKeyList);
You can simply connect to your redis server using redis-cli, select your database and type KEYS *, please remember it will give you all the keys present in selected redis database.
We should be using --scan --pattern with redis 2.8 and later.
You can try using this wrapper on top of redis-cli.
https://github.com/VijayantSoni/redis-helper