I need to get ignite cluster size(no of server nodes running) preferably using control script or ignite rest api. I am able to get baseline nodes using command below but I don't see any command or rest api to return topology snapshot. Is there a way we could get this information to ignite client rather than looking into logs.
Workaround to get baseline nodes:
baselineNodes=$(kubectl --kubeconfig config.conf exec <ignite-node> -n {client_name} -- /opt/ignite/apache-ignite/bin/./control.sh --baseline | grep "Number of baseline nodes" | cut -d ':' -f2 | sed 's/^ *//g')
It seems that topology REST command could do the trick. Here's the documentation link.
http://host:port/ignite?cmd=top&attr=true&mtr=true&id=c981d2a1-878b-4c67-96f6-70f93a4cd241
Got help from ignite community and below command worked for me. Basically idea is to use metric to extract server nodes.
kubectl --kubeconfig config.conf exec <ignite-node> -n {client_name} -- /opt/ignite/apache-ignite/bin/./control.sh --metric cluster.TotalServerNodes | grep -v "metric" | grep cluster.TotalServerNodes | cut -d " " -f5 | sed 's/^ *//g'
Quoting the reply received:
"You can query any metric value or system view content via control script [1], [2]
control.sh --system-view nodes
or [3]
control.sh —metric cluster.TotalBaselineNodes
control.sh —metric cluster.TotalServerNodes
control.sh —metric cluster.TotalClientNodes
[1] https://ignite.apache.org/docs/latest/tools/control-script#metric-command
[2] https://ignite.apache.org/docs/latest/tools/control-script#system-view-command
[3] https://ignite.apache.org/docs/2.11.1/monitoring-metrics/new-metrics#cluster"
I need to delete all nodes that are not running in a RabbitMQ cluster via the command line.
I have tried rabbitmqctl forget_cluster_node, but I'm not sure how to get the list of non-running nodes.
I see all the nodes and running_nodes in the output of rabbitmqctl cluster_status. Can someone help me parse it and let me know if there is any other solution to delete the nodes from a cluster easily?
Figured out by myself
# Remove nodes that are not running from the cluster
nodes=($(egrep -o '[a-z0-9#-]+' <<< $(sudo rabbitmqctl cluster_status --formatter json | jq .nodes.disc)))
running_nodes=($(egrep -o '[a-z0-9#-]+' <<< $(sudo rabbitmqctl cluster_status --formatter json | jq .running_nodes)))
for node in ${nodes[#]}
do
match_count=0
for rnode in ${running_nodes[#]}
do
if [ "${node}" == "${rnode}" ]
then
match_count=1
break
fi
done
if [ $match_count == 1 ]
then
continue
else
sudo rabbitmqctl forget_cluster_node $node
fi
done
In my case the proposed script didn't worked, possibly due RabbitMQ version output differs on the version I'm using(3.9.13). Anyhow, this is what I ended up using:
#!/bin/bash
offline_nodes=$(rabbitmqctl --quiet --formatter json cluster_status \
| jq -r '.disk_nodes-.running_nodes | .[]')
for node in ${offline_nodes[#]}; do
rabbitmqctl forget_cluster_node "$node"
done
I am attempting to create a new redis cluster on the docker swarm using redis 4.0.11. The closest tutorial I have found was this one: https://get-reddie.com/blog/redis4-cluster-docker-compose/
The problem I am having is this like all other tutorials use the ruby redis-trib.rb script to create the cluster after discovering all of the nodes and this guy seems to be no longer supported:
| WARNING: redis-trib.rb is not longer available!
| You should use redis-cli instead.
|
| All commands and features belonging to redis-trib.rb have been moved
| to redis-cli.
| In order to use them you should call redis-cli with the --cluster
| option followed by the subcommand name, arguments and options.
|
| Use the following syntax:
| redis-cli --cluster SUBCOMMAND [ARGUMENTS] [OPTIONS]
|
| Example:
| redis-cli --cluster create 172.22.0.3:6379 172.22.0.5:6379 172.22.0.7:6379 172.22.0.2:6379 172.22.0.6:6379 172.22.0.4:6379 --cluster-replicas 1
|
| To get help about all subcommands, type:
| redis-cli --cluster help
But yet when I attempt to use the recommended command I get an error:
# redis-cli --cluster create 172.22.0.3:6379 172.22.0.5:6379 172.22.0.7:6379 172.22.0.2:6379 172.22.0.6:6379 172.22.0.4:6379 --cluster-replicas 1
Unrecognized option or bad number of args for: '--cluster'
# redis-cli --cluster help
Unrecognized option or bad number of args for: '--cluster'
Ideas?
redis-cli 4.0.11 hasn't --cluster option.
use following ways:
download https://github.com/antirez/redis/archive/unstable.zip
make
use redis-cli in src/redis
They have changed the docs, with the previous version you had to do it with the redis-trib.rb file as you said.
The easiest way to do it, is to download to your server the previous redis-trib.rb (link) and run this command:
./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 \
127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005
before you do it make sure you have installed ruby.
If you have any questions let me know :)
Good luck!
This is a rather simple question but I haven't been able to find an answer.
I have a large number of jobs running in a cluster (>20) and I'd like to delete them all and start over.
According to this site I should be able to just do:
qdel -u netid
to get rid of them all, but in my case that returns:
qdel: invalid option -- 'u'
usage: qdel [{ -a | -c | -p | -t | -W delay | -m message}] [<JOBID>[<JOBID>]|'all'|'ALL']...
-a -c, -m, -p, -t, and -W are mutually exclusive
which obviously indicates that the command does not work.
Just to check, I did:
qstat -u <username>
and I do get a list of all my jobs, but:
qdel -u <username>
also fails.
Found the answer buried in an old supercluster.org thread:
qselect -u <username> | xargs qdel
Worked flawlessly.
Building on what Gabriel answered:
qselect -u <username> | xargs qdel
qselect -u <username> -s <state> | xargs qdel
<state> would be R for running jobs only.
qselect will allow you to select job based on other criterias, like ressources asked (-l), destination queue (-q) ...
qdel -u <username>
will only work with SGE
sometimes a simple grep/cut can help too:
qstat | grep $USER | cut -d. -f1 | xargs qdel
This way we can also grep on a particular keyword for the jobs and delete them.
HTH
Try
$ qdel {id1..id2}
So for example:
$ qdel {1148613..1148650}
For UGE:
qstat -u | gawk '{print $1}' | xargs qdel
# Delete all jobs owned by the current user.
#
# Command breakdown:
# ------------------
#
# qselect
# -u selects all jobs that belong to the current user
# -s EHQRTW selects all job states except for Complete
#
# xargs
# --no-run-if-empty Do not run qdel if the result set is empty
# to avoid triggering a usage error.
#
# qdel
# -a delete jobs asynchronously
#
# The backslashes are a trick to avoid matching any shell aliases.
\qselect -u $(whoami) -s EHQRTW | \xargs --no-run-if-empty \qdel -a
Another possibility is to do qdel all. It deletes all jobs from everyone. When you don't have access for other people's job, it deletes only your jobs.
It is not the most beautiful solution, but it is surely the shortest!
qstat | cut -d. -f1 | sed "s; \(.*\) 0;qdel \1;" | bash
sed's power.
Just use the following command:
qdel all
It will cancel all jobs running on cluster.
I installed rabbitmqadmin and was able to list all the exchanges and queues. How can I use rabbitmqadmin or rabbitmqctl to delete all the queues.
First, list your queues:
rabbitmqadmin list queues name
Then from the list, you'll need to manually delete them one by one:
rabbitmqadmin delete queue name='queuename'
Because of the output format, doesn't appear you can grep the response from list queues. Alternatively, if you're just looking for a way to clear everything (read: reset all settings, returning the installation to a default state), use:
rabbitmqctl stop_app
rabbitmqctl reset # Be sure you really want to do this!
rabbitmqctl start_app
Actually super easy with management plugin and policies:
Goto Management Console (localhost:15672)
Goto Admin tab
Goto Policies tab(on the right side)
Add Policy
Fill Fields
Virtual Host: Select
Name: Expire All Policies(Delete Later)
Pattern: .*
Apply to: Queues
Definition: expires with value 1 (change type from String to Number)
Save
Checkout Queues tab again
All Queues must be deleted
And don't forget to remove policy!!!!!!.
With rabbitmqadmin you can remove them with this one-liner:
rabbitmqadmin -f tsv -q list queues name | while read queue; do rabbitmqadmin -q delete queue name=${queue}; done
In Rabbit version 3.7.10 you can run below command with root permission:
rabbitmqctl list_queues | awk '{ print $1 }' | xargs -L1 rabbitmqctl delete_queue
Try this:
rabbitmqadmin list queues name | awk '{print $2}' | xargs -I qn rabbitmqadmin delete queue name=qn
If you don't have rabbitmqadmin installed, try to purge queues with rabbitmqctl:
rabbitmqctl list_queues | awk '{ print $1 }' | xargs -L1 rabbitmqctl purge_queue
If you're trying to delete queues because they're unused and you don't want to reset, one option is to set the queue TTL very low via a policy, wait for the queues to be auto-deleted once the TTL is passed and then remove the policy (https://www.rabbitmq.com/ttl.html).
rabbitmqctl.bat set_policy delq ".*" '{"expires": 1}' --apply-to queues
To remove the policy
rabbitmqctl clear_policy delq
Note that this only works for unused queues
Original info here: http://rabbitmq.1065348.n5.nabble.com/Deleting-all-queues-in-rabbitmq-td30933.html
I made a deleteRabbitMqQs.sh, which accepts arguments to search the list of queues for, selecting only ones matching the pattern you want. If you offer no arguments, it will delete them all! It shows you the list of queues its about to delete, letting you quit before doing anything destructive.
for word in "$#"
do
args=true
newQueues=$(rabbitmqctl list_queues name | grep "$word")
queues="$queues
$newQueues"
done
if [ $# -eq 0 ]; then
queues=$(rabbitmqctl list_queues name | grep -v "\.\.\.")
fi
queues=$(echo "$queues" | sed '/^[[:space:]]*$/d')
if [ "x$queues" == "x" ]; then
echo "No queues to delete, giving up."
exit 0
fi
read -p "Deleting the following queues:
${queues}
[CTRL+C quit | ENTER proceed]
"
while read -r line; do
rabbitmqadmin delete queue name="$line"
done <<< "$queues"
If you want different matching against the arguments you pass in, you can alter the grep in line four. When deleting all queues, it won't delete ones with three consecutive spaces in them, because I figured that eventuality would be rarer than people who have rabbitmqctl printing its output out in different languages.
Enjoy!
Here is a way to do it with PowerShell. the URL may need to be updated
$cred = Get-Credential
iwr -ContentType 'application/json' -Method Get -Credential $cred 'http://localhost:15672/api/queues' | % {
ConvertFrom-Json $_.Content } | % { $_ } | ? { $_.messages -gt 0} | % {
iwr -method DELETE -Credential $cred -uri $("http://localhost:15672/api/queues/{0}/{1}" -f [System.Web.HttpUtility]::UrlEncode($_.vhost), $_.name)
}
You can use rabbitmqctl eval as below:
rabbitmqctl eval 'IfUnused = false, IfEmpty = true, MatchRegex =
<<"^prefix-">>, [rabbit_amqqueue:delete(Q, IfUnused, IfEmpty) || Q <-
rabbit_amqqueue:list(), re:run(element(4, element(2, Q)), MatchRegex)
=/= nomatch ].'
The above will delete all empty queues in all vhosts that have a name
beginning with "prefix-".
You can edit the variables IfUnused, IfEmpty,
and MatchRegex as per your requirement.
Removing all queues using rabbitmqctl one liner
rabbitmqctl list_queues | awk '{ print $1 }' | sed 's/Listing//' | xargs -L1 rabbitmqctl purge_queue
You need not reset rabbitmq server to delete non-durable queues. Simply stop the server and start again and it will remove all the non-durable queues available.
In case you only want to purge the queues which are not empty (a lot faster):
rabbitmqctl list_queues | awk '$2!=0 { print $1 }' | sed 's/Listing//' | xargs -L1 rabbitmqctl purge_queue
For me, it takes 2-3 seconds to purge a queue (both empty and non-empty ones), so iterating through 50 queues is such a pain while I just need to purge 10 of them (40/50 are empty).
I tried rabbitmqctl and reset commands but they are very slow.
This is the fastest way I found (replace your username and password):
#!/bin/bash
# Stop on error
set -eo pipefail
USER='guest'
PASSWORD='guest'
curl -sSL -u $USER:$PASSWORD http://localhost:15672/api/queues/%2f/ | jq '.[].name' | sed 's/"//g' | xargs -L 1 -I# curl -XDELETE -sSL -u $USER:$PASSWORD http://localhost:15672/api/queues/%2f/#
# To also delete exchanges uncomment next line
# curl -sSL -u $USER:$PASSWORD http://localhost:15672/api/exchanges/%2f/ | jq '.[].name' | sed 's/"//g' | xargs -L 1 -I# curl -XDELETE -sSL -u $USER:$PASSWORD http://localhost:15672/api/exchanges/%2f/#
Note: This only works with the default vhost /
I tried the above pieces of code but I did not do any streaming.
sudo rabbitmqctl list_queues | awk '{print $1}' > queues.txt; for line in $(cat queues.txt); do sudo rabbitmqctl delete_queue "$line"; done.
I generate a file that contains all the queue names and loops through it line by line to the delete them. For the loops, while read ... did not do it for me. It was always stopping at the first queue name.
Here is a faster version (using parallel install sudo apt-get install parallel) expanding on the excellent answer by #admenva
parallel -j 50 rabbitmqadmin -H YOUR_HOST_OR_LOCALHOST -q delete queue name={} ::: $(rabbitmqadmin -H YOUR_HOST_OR_LOCALHOST -f tsv -q list queues name)
This commands deletes all your queues
python rabbitmqadmin.py \
-H YOURHOST -u guest -p guest -f bash list queues | \
xargs -n1 | \
xargs -I{} \
python rabbitmqadmin.py -H YOURHOST -u guest -p guest delete queue name={}
This script is super simple because it uses -f bash, which outputs the queues as a list.
Then we use xargs -n1 to split that up into multiple variables
Then we use xargs -I{} that will run the command following, and replace {} in the command.
To list queues,
./rabbitmqadmin -f tsv -q list queues
To delete a queue,
./rabbitmqadmin delete queue name=name_of_queue
For whose have a problem with installing rabbitmqadmin, You should firstly install python.
UNIX-like operating system users need to copy rabbitmqadmin to a directory in PATH, e.g. /usr/local/bin.
Windows users will need to ensure Python is on their PATH, and invoke rabbitmqadmin as python.exe rabbitmqadmin.
Then
Browse to http://{hostname}:15672/cli/rabbitmqadmin to download.
Go to the containing folder then run cmd with administrator privilege
To list Queues
python rabbitmqadmin list queues.
To delete Queue
python rabbitmqadmin delete queue name=Name_of_queue
To Delete all Queues
1- Declare Policy
python rabbitmqadmin declare policy name='expire_all_policies' pattern=.* definition={\"expires\":1} apply-to=queues
2- Remove the policy
python rabbitmqadmin delete policy name='expire_all_policies'
Following command worked for me:
sudo rabbitmqctl list_queues | awk '{print $1}' | xargs -I qn sudo rabbitmqctl delete_queue qn
There's a way to remove all queues and exchanges without scripts and full reset. You can just delete and re-create a virtual host from admin interface. This will work even for vhost /.
The only thing you'll need to restore is permissions for the newly created vhost.
Okay, important qualifier for this answer:
The question does ask to use either rabbitmqctl OR rabbitmqadmin to solve this, my answer needed to use both. Also, note that this was tested on MacOS 10.12.6 and the versions of the rabbitmqctl and rabbitmqadmin that are installed when installing rabbitmq with Homebrew and which is identified with brew list --versions as rabbitmq 3.7.0
rabbitmqctl list_queues -p <VIRTUAL_HOSTNAME> name | sed 1,2d | xargs -I qname rabbitmqadmin --vhost <VIRTUAL_HOSTNAME> delete queue name=qname
Another option is to delete the vhost associated with the queues. This will delete everything associated with the vhost, so be warned, but it is easy and fast.
NOTE: the RabbitMQ team monitors the rabbitmq-users mailing list and only sometimes answers questions on StackOverflow.
This is a method I use. It is easy, clear and effective.
This is the document:
Vhost=the_vhost_name
User=user_name
Password=the_passworld
for i in `rabbitmqctl list_queues -p $Vhost | awk '{ print $1 }'`
do
echo "queu_name: $i"
curl -u $User:$Passworld -H "content-type:application/json" -XDELETE http://localhost:15672/api/queues/$Vhost/$i
done
Try this:
rabbitmqctl list_queues -q name > q.txt
IFS=$'\n' read -d '' -r -a queues < q.txt
count=${#queues[#]}
i=1; while (($i < $count)); do echo ${queues[$i]};rabbitmqctl delete_queue ${queues[$i]};i=$((i+1)); done
As per https://stackoverflow.com/a/52002145/3278855
To automate that, it's possible to use this curl:
curl -X PUT --data '{"pattern":".*","apply-to":"all","definition":{"expires":1},"priority":0}' -u guest:guest 'http://localhost:15672/api/policies/%2f/clear' && \
curl -X DELETE -u guest:guest 'http://localhost:15672/api/policies/%2f/clear'
Please note that %2f is default vhost name (/) and guest:guest is login:password
rabbitmqadmin list queues|awk 'NR>3{print $4}'|head -n-1|xargs -I qname rabbitmqadmin delete queue name=qname