I am working on creating a shell script to automate setup of redis cluster. But I am getting stuck at the create cluster command.
When my script is executing the command
redis-cli --cluster create
It asks to type a yes, but I want to make it non interactive & it should proceed with me giving an input.
I have tried:
yes | redis-cli --cluster create
But this is also not working.
Please help. Thanks In Advance.
#!/usr/bin/env python3
from subprocess import Popen, PIPE, STDOUT
cmd = 'redis-cli --cluster create 172.31.104.226:6379 172.31.103.167:6379 172.31.102.56:6379 --cluster-replicas 0'
p = Popen(cmd.split(), stdout=PIPE, stdin=PIPE, stderr=STDOUT)
grep_stdout = p.communicate(input='yes\n'.encode())[0]
print(grep_stdout)
Related
I am trying to execute a lua script against a redis instance running in a container. The code is very simple 2 liner example.
local foo = redis.call("ping")
return foo
As this file is on the host machine, how do i get it to execute inside docker exec ? Or should I install redis-cli locally to test it?
I think I understood what you want. You have a Lua script on your docker host that you want to load into Redis running inside a docker container without needing redis-cli on the host.
So, start the official Redis in a container as a daemon:
docker run --name some-redis -d redis
Now, assuming your script is on the host in a file called script.lua, you want to load that into dockerised Redis:
docker exec -i some-redis redis-cli -x script load < script.lua
That will return a SHA sum - I got 93126620988a563ac9dd347d02a9cdbbbeaee3c1 - and you can then run the script with:
docker exec -i some-redis redis-cli EVALSHA 93126620988a563ac9dd347d02a9cdbbbeaee3c1 0
PONG
If you want to automate the capture of the SHA sum and its subsequent use:
sha=$(docker exec -i some-redis redis-cli -x script load < script.lua)
Then:
docker exec -i some-redis redis-cli EVALSHA $sha 0
PONG
When trying to create a cluster with redis-cli as follows
redis-cli --cluster create
a prompt comes up asking for configuration confirmation?
Is there a way to script this (preferably in ansible) and run it non-interactively?
I am aware of this topic however it addresses data manipulation which is not the scope of this question.
--cluster-yes is the correct option!
EDIT:
The answer of Leonardo Oliveira rightfully points out that the option --cluster-yes will avoid the prompt. For example:
redis-cli --cluster create host1:6379 host2:6379 host3:6379 --cluster-yes
As of the current Redis version (5.0.5) there doesn't seem to be a flag under --cluster that can silence or auto-answer the interactive question:
$ redis-cli --cluster help
Cluster Manager Commands:
create host1:port1 ... hostN:portN
--cluster-replicas <arg>
check host:port
--cluster-search-multiple-owners
info host:port
fix host:port
--cluster-search-multiple-owners
reshard host:port
--cluster-from <arg>
--cluster-to <arg>
--cluster-slots <arg>
--cluster-yes
--cluster-timeout <arg>
--cluster-pipeline <arg>
--cluster-replace
rebalance host:port
--cluster-weight <node1=w1...nodeN=wN>
--cluster-use-empty-masters
--cluster-timeout <arg>
--cluster-simulate
--cluster-pipeline <arg>
--cluster-threshold <arg>
--cluster-replace
add-node new_host:new_port existing_host:existing_port
--cluster-slave
--cluster-master-id <arg>
del-node host:port node_id
call host:port command arg arg .. arg
set-timeout host:port milliseconds
import host:port
--cluster-from <arg>
--cluster-copy
--cluster-replace
help
By using echo you can execute the command and auto-answer the prompt:
echo "yes" | redis-cli --cluster create host1:6379 host2:6379 host3:6379
The default Ansible Redis module only supports a few commands and not --cluster so you would have to create your own logic with command/shell tasks:
- name: Create cluster
shell: echo "yes" | redis-cli --cluster create host1:6379 host2:6379 host3:6379
run_once: true
when: not cluster_setup_done
Well, I don't know about the ansible part. But Redis official site does provide a way to create cluster with script in a little interactive-mode.
Creating a Redis Cluster using the create-cluster script (Please refer docs for more details)
If you don't want to create a Redis Cluster by configuring and executing individual instances manually as explained above, there is a much simpler system (but you'll not learn the same amount of operational details).
Just check utils/create-cluster directory in the Redis distribution. There is a script called create-cluster inside (same name as the directory it is contained into), it's a simple bash script. In order to start a 6 nodes cluster with 3 masters and 3 slaves just type the following commands:
create-cluster start
create-cluster create
Reply to yes in step 2 when the redis-cli utility wants you to accept the cluster layout.
You can now interact with the cluster, the first node will start at port 30001 by default. When you are done, stop the cluster with:
create-cluster stop
Please read the README inside this directory for more information on how to run the script.
You can try implementing this if it helps you in any way!
Cheers :)
Is there a way to get the results of a pig script run on a remote cluster directly without STORE-ing them and retrieving them separately?
So you can use a pig parameters to run your scripts. For example:
example.pig
A = LOAD '$PATH_TO_FOLDER_WITH_DATA' AS (f1:int, f2:int, f3:int);
--# Do Something With Your Data, and get output
C = STORE ouput INTO '$OUTPUT_PATH'
Then you can run the script like:
pig -p "/path/to/local/file" -p "/path/to/the/output" example.pig
So to automate in BASH:
storelocal.sh
#!/bin/bash
pig -p '$PATH_TO_FILES' -p '$PATH_TO_HDFS_OUT' example.pig
hdfs dfs -getmerge '$PATH_TO_HDFS_OUT' '$PATH_TO_LOCAL'
And you can run it ./storelocal.sh /path/to/local/file /path/to/the/local/output
How can i run the following redis-cli command using a cron job?
redis-cli info clients
I wrote a simple bash script with this command, and setup a cron job to run every minute. But it does not seem like redis-cli commands are working with crontab.
Any suggestion how i can achieve this?
As posted by Mark Setchell in the comment
You are probably missing the full path to the command, maybe /usr/local/bin/redis-cli or some such. Presumably you want to send the output somewhere too... use > someFile.txt at the and of the command.
referring to the full path of redis-cli worked for me.
I'm attempting to setup a new compute cluster, and currently experiencing errors when using the qsub command in the SGE. Here's a simple experiment that shows the problem:
test.sh
#!/usr/bin/zsh
test="hello"
echo "${test}"
test.sh.eXX
test=hello: Command not found.
test: Undefined variable.
test.sh.oXX
Warning: no access to tty (Bad file descriptor).
Thus no job control in this shell.
If I ran the script on the head node (sh test.sh), the output is correct. I submit the job to the SGE by typing "qsub test.sh".
If I submit the exact same script job in the same way on an established compute cluster like HPC, it works perfectly as expected. What setting could be causing this problem?
Thanks for any help on this matter.
Most likely the queues on your cluster are set to posix_compliant mode with a default shell of /bin/csh. The posix_compliant setting means your #! line is ignored. You can either change the queues to unix_behavior or specify the required shell using qsub's -S option.
#$ -S /bin/sh