Related
Hope you all are safe and well !!
I am running a script that gives me Azure AD apps with its secret end date property. The property name which gives me all details in Azure AD is “PasswordCredentials” and I am using get-azureadapplication cmdlet.
What is the best way to check If the app has end date value within a month and filter on it, I tried where-object with get-date.adddays(30) and tried to compare with -lt operators.
Appreciate your support here.
Well, in your case, you should note there may be several PasswordCredentials for one app, so you need to use a loop to check every PasswordCredential.
And the EndDate you got from the Get-AzureADApplication command is a UTC time, but Get-Date returns the local time, so you need to use (Get-Date).AddDays(30).ToUniversalTime() instead of (Get-Date).AddDays(30). Also, if you just want to check the expiring ones, you must exclude the ones that have expired, so use $PasswordCredential.EndDate -gt $nowdate like below.
As you did not give your script, I can just give a sample for you, my sample is for a single app, if you want to check all the apps in your tenant, use a loop to do that.
$PasswordCredentials = (Get-AzureADApplication -ObjectId <object-id>).PasswordCredentials
$nowdate = (Get-Date).ToUniversalTime()
$wantdate = (Get-Date).AddDays(30).ToUniversalTime()
foreach($PasswordCredential in $PasswordCredentials){
if($PasswordCredential.EndDate -lt $wantdate -and $PasswordCredential.EndDate -gt $nowdate){
$keyid = $PasswordCredential.KeyId
Write-Output "The key with KeyId $keyid will expire"
}
}
Update:
If you want to get the AppId and AppName, you could use the script below.
$apps = Get-AzureADApplication -All $true
$nowdate = (Get-Date).ToUniversalTime()
$wantdate = (Get-Date).AddDays(30).ToUniversalTime()
foreach($app in $apps){
$PasswordCredentials = $app.PasswordCredentials
$appid = $app.AppId
$displayname = $app.DisplayName
foreach($PasswordCredential in $PasswordCredentials){
if($PasswordCredential.EndDate -lt $wantdate -and $PasswordCredential.EndDate -gt $nowdate){
$a = $app | select #{Name="AppId"; Expression={$appid}}, #{Name="DisplayName"; Expression={$displayname}}, #{Name="EndDate"; Expression={$PasswordCredential.EndDate}}
Write-Output $a
}
}
}
In my setup, the info command shows me the following:
[keys] => 1128
[expires] => 1125
I'd like to find those 3 keys without an expiration date. I've already checked the docs to no avail. Any ideas?
Modified from a site that I can't find now.
redis-cli keys "*" | while read LINE ; do TTL=`redis-cli ttl "$LINE"`; if [ $TTL -eq -1 ]; then echo "$LINE"; fi; done;
edit: Note, this is a blocking call.
#Waynn Lue's answer runs but uses the Redis KEYS command which Redis warns about:
Warning: consider KEYS as a command that should only be used in production environments with extreme care. It may ruin performance when it is executed against large databases.
Redis documentation recommends using SCAN.
redis-cli --scan | while read LINE ; do TTL=`redis-cli ttl "$LINE"`; if [ $TTL -eq -1 ]; then echo "$LINE"; fi; done;
If you want to scan for a specific key pattern, use:
redis-cli --scan --pattern "something*"
In case somebody is getting bad arguments or wrong number of arguments error,
put double quotes around $LINE.
So,it would be
redis-cli keys "*" | while read LINE ; do TTL=`redis-cli ttl "$LINE"`; if [ $TTL -eq -1 ]; then echo "$LINE"; fi; done;
This happens when there are spaces in the key.
To me the accepted answer appears unusable for a medium-sized dataset, as it will run a redis-cli command for each and every key.
Instead I used this lua script to filter the keys inside the redis server:
local show_persistent = ARGV[1] ~= "expiring"
local keys = {}
for i, name in ipairs(redis.call("keys", "*")) do
local persistent = redis.call("pttl", name) < 0
if persistent == show_persistent then
table.insert(keys, name)
end
end
return keys
This can be called as
$ redis-cli --eval show-persistent-keys.lua
to get all keys without an expiration time. It also can be called as
$ redis-cli --eval show-persistent-keys.lua , expiring
to find the opposite key set of all keys with an expiration time set.
On the downside this may block for too long (appears fine for 1 M keys). I'd use scan instead but I happen to have to run this against a legacy Redis at version 2.6, which does not have scan available.
I needed to extract non-expiring keys from bigger (40GB) dataset, so using keys command was not suitable for me. So in case someone is looking for offline/non-blocking solution, you can use https://github.com/sripathikrishnan/redis-rdb-tools for extraction of non-expiring keys from redis rdb dump.
You can just install libraries via pip:
pip install rdbtools python-lzf
Then create simple parser which extracts keys and values which has expiration set to None:
from rdbtools import RdbParser, RdbCallback
from rdbtools.encodehelpers import bytes_to_unicode
class ParserCallback(RdbCallback):
def __init__(self):
super(ParserCallback, self).__init__(string_escape=None)
def encode_key(self, key):
return bytes_to_unicode(key, self._escape, skip_printable=True)
def encode_value(self, val):
return bytes_to_unicode(val, self._escape)
def set(self, key, value, expiry, info):
if expiry is None:
print('%s = %s' % (self.encode_key(key), self.encode_value(value)))
callback = ParserCallback()
parser = RdbParser(callback)
parser.parse('/path/to/dump.rdb')
#!/usr/bin/env python
import argparse
import redis
p = argparse.ArgumentParser()
p.add_argument("-i", '--host', type=str, default="127.0.0.1", help="redis host", required=False)
p.add_argument("-p", '--port', type=int, default=6379, help="redis port", required=False)
p.add_argument("-n", '--db', type=int, default=0, help="redis database", required=False)
args = p.parse_args()
r = redis.Redis(host=args.host, port=args.port, db=args.db)
try:
keys = r.keys()
for key in keys:
if r.ttl(key) < 0:
print(key)
except KeyboardInterrupt:
pass
Is there a way to print the number of keys in Redis?
I am aware of
keys *
But that seems slightly heavy weight. - Given that Redis is a key value store maybe this is the only way to do it. But I would still like to see something along the lines of
count keys *
You can issue the INFO command, which returns information and statistics about the server. See here for an example output.
As mentioned in the comments by mVChr, you can use info keyspace directly on the redis-cli.
redis> INFO
# Server
redis_version:6.0.6
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:b63575307aaffe0a
redis_mode:standalone
os:Linux 5.4.0-1017-aws x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:9.3.0
process_id:2854672
run_id:90a5246f10e0aeb6b02cc2765b485d841ffc924e
tcp_port:6379
uptime_in_seconds:2593097
uptime_in_days:30
hz:10
configured_hz:10
lru_clock:4030200
executable:/usr/local/bin/redis-server
DBSIZE returns the number of keys and it's easier to parse.
Downside: if a key has expired it may still count.
http://redis.io/commands/dbsize
The DBSIZE command returns the number of keys
> DBSIZE
WARNING: Do not run this on a production machine.
On a Linux box:
redis-cli KEYS "*" | wc -l
Note: As mentioned in comments below, this is an O(N) operation, so on a large DB with many keys you should not use this. For smaller deployments, it should be fine.
Since Redis 2.6, lua is supported, you can get number of wildcard keys like this
eval "return #redis.call('keys', 'prefix-*')" 0
see eval command
Go to redis-cli and use below command
info keyspace
It may help someone
Displays the DB's name along with keys count:
redis-cli info keyspace
# Keyspace
db0:keys=12995,expires=0,avg_ttl=0
db12:keys=5524396,expires=5,avg_ttl=45201
Display the no. of keys in the selected database:
redis-cli dbsize
(integer) 12995
Displays whole Redis's stats:
redis-cli info
dbsize() returns the total number of keys.
You can quickly estimate the number of keys matching a given pattern by sampling keys at random, then checking what fraction of them matches the pattern.
Example in python; counting all keys starting with prefix_:
import redis
r = redis.StrictRedis(host = 'localhost', port=6379)
iter=1000
print 'Approximately', r.dbsize() * float(sum([r.randomkey().startswith('prefix_') for i in xrange(iter)])) / iter
Even iter=100 gives a decent estimate in my case, yet is very fast, compared to keys prefix_.
An improvement is to sample 1000 keys on every request, but keep the total count, so that after two requests you'll divide by 2000, after three requests you'll divide by 3000. Thus, if your application is interested in the total number of matching keys fairly often, then every time it will get closer and closer to the true value.
After Redis 2.6, the result of INFO command are splitted by sections. In the "keyspace" section, there are "keys" and "expired keys" fields to tell how many keys are there.
eval "local count = redis.call('scan', 0, 'match', 'key:*:key', 'count', 10000) if count ~= 0 then return #count[2] end " 0
eval "local count = redis.call('sscan', 'key.key:all', 0, 'match', '*', 'count', 1000000) if count ~= 0 then return #count[2] end " 0
I'm writing a Powershell script and in it I'm using a hash table to store information about database checks. The table has 5 keys (host, check, last execution time, last rep, status) and I want to search in my table for values where:
$s = $table where $host -eq $hostname -and check -eq $check
Does anyone have any idea how this is done? And if it makes any difference, the script cannot rely on .NET framework higher than 2.0
I´m new to Powershell and scripting in general so this might be very obvious but I still can't seem to find an answer on Google. Also if someone knows a good reference page for Powershell scripting I would really appreciate a link.
Gísli
EDIT: Don't see how it matters but here is a function I use to create a hash table:
function read_saved_state{
$state = #{}
$logpos = #{}
$last_log_rotate = 0
foreach($s in Get-Content $saved_state_file){
$x = $s.split('|')
if($x[0] -eq 'check'){
$state.host = $x[1]
$state.check = $x[2]
$state.lastexec = $x[3]
$state.lastrep = $x[4]
$state.status = $x[5]
}
elseif($x[0] -eq 'lastrotate'){
$last_log_rotate = $x[1]
}
elseif($x[0] -eq 'log'){
$logpos.lastpos = $x[3]
}
}
$saved_state_file has one line for each check run and can also have a line for last log rotate and last log position. There can be as many as 12 checks for one host.
I'm trying to extract a particular check, run at a particular host, and changing the lastexec_time, last_rep and status.
return $state,$logpos,$last_log_rotate
}
Assuming you have an array or list of hashtables (not entirely clear from the question), your syntax is pretty close:
$s = $tables | where {($_.host -eq $hostname) -and ($_.check -eq $check)}
Is there a Redis command for fetching all keys in the database? I have seen some python-redis libraries fetching them. But was wondering if it is possible from redis-client.
Try to look at KEYS command. KEYS * will list all keys stored in redis.
EDIT: please note the warning at the top of KEYS documentation page:
Time complexity: O(N) with N being the number of keys in the database, under the assumption that the key names in the database and the given pattern have limited length.
UPDATE (V2.8 or greater): SCAN is a superior alternative to KEYS, in the sense that it does not block the server nor does it consume significant resources. Prefer using it.
Updated for Redis 2.8 and above
As noted in the comments of previous answers to this question, KEYS is a potentially dangerous command since your Redis server will be unavailable to do other operations while it serves it. Another risk with KEYS is that it can consume (dependent on the size of your keyspace) a lot of RAM to prepare the response buffer, thus possibly exhausting your server's memory.
Version 2.8 of Redis had introduced the SCAN family of commands that are much more polite and can be used for the same purpose.
The CLI also provides a nice way to work with it:
$ redis-cli --scan --pattern '*'
It can happen that using redis-cli, you connect to your remote redis-server, and then the command:
KEYS *
is not showing anything, or better, it shows:
(empty list or set)
If you are absolutely sure that the Redis server you use is the one you have the data, then maybe your redis-cli is not connecting to the Redis correct database instance.
As it is mentioned in the Redis docs, new connections connect as default to the db 0.
In my case KEYS command was not retrieving results because my database was 1. In order to select the db you want, use SELECT.
The db is identified by an integer.
SELECT 1
KEYS *
I post this info because none of the previous answers was solving my issue.
-->Get all keys from redis-cli
-redis 127.0.0.1:6379> keys *
-->Get list of patterns
-redis 127.0.0.1:6379> keys d??
This will produce keys which start by 'd' with three characters.
-redis 127.0.0.1:6379> keys *t*
This wil get keys with matches 't' character in key
-->Count keys from command line by
-redis-cli keys * |wc -l
-->Or you can use dbsize
-redis-cli dbsize
Get All Keys In Redis
Get all keys using the --scan option:
$ redis-cli --scan --pattern '*'
List all keys using the KEYS command:
$ redis-cli KEYS '*'
Take a look at following Redis Cheat Sheet.
To get a subset of redis keys with the redis-cli i use the command
KEYS "prefix:*"
Yes, you can get all keys by using this
var redis = require('redis');
redisClient = redis.createClient(redis.port, redis.host);
redisClient.keys('*example*', function (err, keys) {
})
SCAN doesn't require the client to load all the keys into memory like KEYS does. SCAN gives you an iterator you can use. I had a 1B records in my redis and I could never get enough memory to return all the keys at once.
Here is a python snippet to get all keys from the store matching a pattern and delete them:
import redis
r = redis.StrictRedis(host='localhost', port=6379, db=0)
for key in r.scan_iter("key_pattern*"):
print key
redis-cli -h <host> -p <port> keys *
where * is the pattern to list all keys
KEYS pattern
Available since 1.0.0.
Time complexity: O(N) with N being the number
of keys in the database, under the assumption that the key names in
the database and the given pattern have limited length.
Returns all keys matching pattern.
Warning : This command is not recommended to use because it may ruin performance when it is executed against large databases instead of KEYS you can use SCAN or SETS.
Example of KEYS command to use :
redis> MSET firstname Jack lastname Stuntman age 35
"OK"
redis> KEYS *name*
1) "lastname"
2) "firstname"
redis> KEYS a??
1) "age"
redis> KEYS *
1) "lastname"
2) "age"
3) "firstname"
In order to get all the keys available in redis server, you should open redis-cli and type:
KEYS *
In order to get more help please visit this page:
This Link
If your redis is a cluster,you can use this script
#!/usr/bin/env bash
redis_list=("172.23.3.19:7001,172.23.3.19:7002,172.23.3.19:7003,172.23.3.19:7004,172.23.3.19:7005,172.23.3.19:7006")
arr=($(echo "$redis_list" | tr ',' '\n'))
for info in ${arr[#]}; do
echo "start :${info}"
redis_info=($(echo "$info" | tr ':' '\n'))
ip=${redis_info[0]}
port=${redis_info[1]}
echo "ip="${ip}",port="${port}
redis-cli -c -h $ip -p $port set laker$port '湖人总冠军'
redis-cli -c -h $ip -p $port keys \*
done
echo "end"
For the ones that wants a typescript helper (using ioredis)
import Redis from 'ioredis';
import { from, Observable, of } from 'rxjs';
import { first, mergeMap } from 'rxjs/operators';
export function scanKeysFromRedis(redisStore: Redis.Redis, key: string,
target: number = 0, keys: string[] = []): Observable<string[]> {
return from(redisStore.scan(target, 'MATCH', key)).pipe(
first(),
mergeMap((_keys) => {
const _target = Number(_keys[0]);
if (_target !== 0) {
return scanKeysFromRedis(redisStore, key, _target, [...keys, ..._keys[1]]);
}
return of([...keys, ..._keys[1]]);
}),
);
}
and call it with: scanKeysFromRedis(store, 'hello');
If you are using Laravel Framework then you can simply use this:
$allKeyList = Redis::KEYS("*");
print_r($allKeyList);
In Core PHP:
$redis = new Redis();
$redis->connect('hostname', 6379);
$allKeyList = $redis->keys('*');
print_r($allKeyList);
You can simply connect to your redis server using redis-cli, select your database and type KEYS *, please remember it will give you all the keys present in selected redis database.
We should be using --scan --pattern with redis 2.8 and later.
You can try using this wrapper on top of redis-cli.
https://github.com/VijayantSoni/redis-helper