Is it possible to create a script for Redis that flush its memory when it is above a certain value?
In my specific case, I want a flush when the memory is above 90%.
What is the best way, via bash script or Lua script?
I would use a Lua script as it will perform faster, atomically, and it would be easy to use both from redis-cli and any application code.
Here a Lua script to get memory used and maxmemory, the percent, and an action placeholder. It uses both MEMORY STATS and INFO memory to illustrate.
MEMORY STATS brings structured information, but doesn't include maxmemory or total_system_memory, as INFO memory does. CONFIG GET is not allowed from Lua scripts.
local stats = redis.call('MEMORY', 'STATS')
local memused = 0
for i = 1,table.getn(stats),2 do
if stats[i] == 'total.allocated' then
memused = stats[i+1]
break
end
end
local meminfo = redis.call('INFO', 'memory')
local maxmemory = 0
for s in meminfo:gmatch('[^\\r\\n]+') do
if string.sub(s,1,10) == 'maxmemory:' then
maxmemory = tonumber(string.sub(s,11))
end
end
local mempercent = memused/maxmemory
local action = 'No action'
if mempercent > tonumber(ARGV[1]) then
action = 'Flush here'
end
return {memused, maxmemory, tostring(mempercent), action}
Use as:
> EVAL "local stats = redis.call('MEMORY', 'STATS') \n local memused = 0 \n for i = 1,table.getn(stats),2 do \n if stats[i] == 'total.allocated' then \n memused = stats[i+1] \n break \n end \n end \n local meminfo = redis.call('INFO', 'memory') \n local maxmemory = 0 \n for s in meminfo:gmatch('[^\\r\\n]+') do \n if string.sub(s,1,10) == 'maxmemory:' then \n maxmemory = tonumber(string.sub(s,11)) \n end \n end \n local mempercent = memused/maxmemory \n local action = 'No action' \n if mempercent > tonumber(ARGV[1]) then \n action = 'Flush here' \n end \n return {memused, maxmemory, tostring(mempercent), action}" 0 0.9
1) (integer) 860264
2) (integer) 100000000
3) "0.00860264"
4) "No action"
That's the way to obtain the allocated memory redis-cli -h 1.2.3.4 -p 6379 memory stats | sed -n 4p. Now it's easy to create a bash script
Related
I have a file with this structure:
http://paste.ubuntu.com/21136265/
And I have to capture all the data from the line 'ADSTART ACTION(ADD)' to the next line with this same text, to create a single record, or line.
Sorry but I Can't post an example of the output because is all the data between the 'ADSTART' lines in a single line or record, I'm working under z/OS and we have the concept of record length.
I'm trying this in REXX for z/OS and in AWK in UNIX SYSTEM SERVICES for z/OS, but I'm stuck putting all fields in a line, and I can't figure out how to do it.
I'm capturing the data trough nested loops, but I don't know haw to put it in a single line.
If you're using REXX then why don't you just use the parse instruction to scrape the report file? The parse instruction uses a template pattern which is very simple but powerful.
Here's an example:
/* REXX */
queue "ADSTART ACTION(ADD)"
queue " ADID(ABCD0B ) ADVALFROM(111230) CALENDAR(CALSEM7J )"
queue " DESCR('DESCRIPTION ')"
queue " ADTYPE(A)"
queue " GROUP(PBQOPC )"
queue " OWNER('OWNER1')"
queue " PRIORITY( 5) ADSTAT(A)"
queue " ODESCR('ALADIN ')"
queue "ADRUN ACTION(ADD)"
queue " PERIOD(HEB ) RULE(3) VALFROM(091230) VALTO(711231)"
queue " SHIFT( 0) SHSIGN(F)"
queue " DESCR('DESCRIPTION')"
queue " TYPE(N)"
queue " IADAYS( 1, 2, 3, 4, 5, 6, 7)"
queue " IATIME(1700) DLDAY( 1) DLTIME(0600)"
do while queued() > 0
parse pull rec
select
when startswith(rec,"ADSTART") then do
p. = '' /* the output record */
parse var rec with . 'ACTION('p.action')'
do queued()
parse pull rec
if left(rec,1) /= ' ' then do
/* End of parameter group. Re-queue the record and break */
push rec
leave
end
select
when startswith(rec, " ADID") then do
parse var rec with . "ADID("p.adid") ADVALFROM("p.advalfrom")" ,
"CALENDAR("p.calendar")"
end
when startswith(rec, " DESCR") then do
parse var rec with "DESCR('"p.desc"')"
end
when startswith(rec, " PRI") then do
parse var rec with "PRIORITY("p.priority") ASTAT("p.adstat")"
end
otherwise nop
end
end
/* write out the record in 1 line */
say strip(p.action) strip(p.adid) strip(p.advalfrom) strip(p.calendar),
strip(p.desc) strip(p.priority) strip(p.adstat)
end
when startswith(rec,"ADRUN") then do
/* do some stuff to parse this */
end
otherwise nop
end
end
exit 0
startswith:
parse arg input, prefix
input_len = length(input)
if input_len = 0 then return 0
prefix_len = length(prefix)
if prefix_len = 0 then return 0
return input_len >= prefix_len & left(input,prefix_len) = prefix
Seeing as you're comfortable in z/OS UNIX environment, if you want something a little bit more powerful than REXX and/or AWK you should checkout my z/OS port of Lua. It comes with an LPeg package which makes it trivially easy to write lexers and parsers with very few lines of code.
If all you want to do is text flow the TWS control statements onto one line without capturing the fields then that's very simple to do.
/* REXX */
queue "ADSTART ACTION(ADD)"
queue " ADID(ABCD0B ) ADVALFROM(111230) CALENDAR(CALSEM7J )"
queue " DESCR('DESCRIPTION ')"
queue " ADTYPE(A)"
queue " GROUP(PBQOPC )"
queue " OWNER('OWNER1')"
queue " PRIORITY( 5) ADSTAT(A)"
queue " ODESCR('ALADIN ')"
queue "ADRUN ACTION(ADD)"
queue " PERIOD(HEB ) RULE(3) VALFROM(091230) VALTO(711231)"
queue " SHIFT( 0) SHSIGN(F)"
queue " DESCR('DESCRIPTION')"
queue " TYPE(N)"
queue " IADAYS( 1, 2, 3, 4, 5, 6, 7)"
queue " IATIME(1700) DLDAY( 1) DLTIME(0600)"
do while queued() > 0
parse pull rec
if left(rec,1) /= ' ' then do
line = rec
do queued()
parse pull rec
if left(rec,1) /= ' ' then do
push rec;leave
end
line = line rec
end
say space(line,1)
end
end
exit 0
Try this;
sed -n '/ADSTART ACTION(ADD)/,/ADRUN/p' <filename> | sed 's/ADRUN ACTION(ADD)//g'
Maybe this would do it:
awk '/ADSTART ACTION\(ADD\)/{print buf; buf=""} {buf=buf""$0" "} END{print buf}' test.in
Commented version:
/ADSTART ACTION\(ADD\)/ { # for records where ADSTART occurs
print buf # output the buffer variable
buf="" # then empty the buffer
}
{ # for all records
# gsub(/^ +| +$/,"") # here you could trim leading and trailing space
buf=buf""$0" " # build the buffer
}
END { # in the end
print buf # output the remaining buffer
}
Although the solution above could work for not many lines per block, a solution that only prints the text between ADSTART ACTION (ADD) and assumes only one block will be printed
Bash:
gawk 'BEGIN{s=0} /ADSTART.*ACTION(ADD)/ {s=(s+1)%2} (s==1){ print }' | sed ':a;N;$!ba;s/\n//g'
(ADSTART... lines are omitted)
Thank you very much for all the answers.
At last it was pretty easy, because when I do an FTP from z/OS to USS (Unix System Services for z/OS) in binary, all the data is in one line.
At first I was working with a file transfered with FTP (ASCII xlate) to my PC, and then transmitted to USS in binary FTP with WinSCP.
This is the code I used to replace a text pattern with carriage return:
sed 's/ADSTART ACTION(ADD)/\
/g' <input file> ><output file>
with carriage return inserted by pressing enter key, because /r /'$''' /n /x0D didn't worked in USS, I don't know why.
Thank you all again for your time.
Patricio.
Not sure if this one is possible in objective-c without having the script as a file and running it that way.
NSAppleScript *appleScriptGetHH = [[NSAppleScript alloc] initWithSource:#\
"tell application \"System Events\"\n \
tell process \"Grabee\"\n \
with timeout of 0 seconds\n \
get value of attribute \"AXValue\" of text area 1 of scroll area 1 of window \"Demo Window 1" \n \
end timeout \n \
end tell \n \
end tell"];
This works perfectly, but what I would like to do is replace "Demo Window 1" with a string (as it will be changed dynamically in the program)
When I use something like this
NSString *windowName = #"Green Window4";
And then replace this line:
get value of attribute \"AXValue\" of text area 1 of scroll area 1 of window \"Demo Window 1" \n \
With:
get value of attribute \"AXValue\" of text area 1 of scroll area 1 of window \"%#" \n \
And
end tell", windowName];
I receive an error that there are too many arguments, is there a way to do this without having the script separate?
Build the string like this;
NSAppleScript *appleScriptGetHH = [[NSAppleScript alloc] initWithSource:
[[NSString alloc] initWithFormat:#\
"tell application \"System Events\"\n \
tell process \"Grabee\"\n \
with timeout of 0 seconds\n \
get value of attribute \"AXValue\" of text area 1 of scroll area 1 of window \"%#" \n \
end timeout \n \
end tell \n \
end tell", windowName]
];
or build the string as a separate step for a bit more readability.
I have encountered a problem in one of my TCL scripts. I need to run it in an infinite loop with a terminating condition and in every loop I need to write some output. This is the basic code that im using:
proc wr {i} {
puts -nonewline "$i"
}
proc do {roof} {
set end 0
while {$end < $roof} {
after 1000
wr $end
incr end
}
}
do 10
The expected behaviour is that every second there will be a new output until $end == $roof. But instead after running this script, the console window is busy for 10 seconds and after that time, the entire output prints out at once.
Thank you for your advice :)
The problem is that you don't flush stdout.
If you modify your script so it flushes stdout:
proc wr {i} {
puts -nonewline "$i"
flush stdout
}
proc do {roof} {
set end 0
while {$end < $roof} {
after 1000
wr $end
incr end
}
}
do 10
It will work. You can also change the buffering of the stdout channel to none, the default is line:
fconfigure stdout -buffering none
If you write more than one line, the default buffering will flush stdout when it encounters a newline, but you never write a newline.
I have a sample program in 2 formats perl & embperl
The perl version works as a CGI but the embperl version does not work.
Any suggestions or pointers to solutions would be appreciated
OS: Linux version 2.6.35.6-48.fc14.i686.PAE (...) (gcc version 4.5.1 20100924 (Red Hat 4.5.1-4) (GCC) ) #1 SMP Fri Oct 22 15:27:53 UTC 2010
NOTE: I originally posted this question to perlmonks [x] and the embperl mailing list [x] but didn't get a solution.
perl working script
#!/usr/bin/perl
use warnings;
use strict;
use IPC::Open3;
print "Content-type: text/plain\n\n";
my $cmd = 'ls';
my $pid = open3(*HIS_IN, *HIS_OUT, *HIS_ERR, $cmd);
close(HIS_IN); # give end of file to kid, or feed him
my #outlines = <HIS_OUT>; # read till EOF
my #errlines = <HIS_ERR>; # XXX: block potential if massive
print "STDOUT: ", #outlines, "\n";
print "STDERR: ", #errlines, "\n";
waitpid( $pid, 0 );
my $child_exit_status = $? >> 8;
print "child_exit_status: $child_exit_status\n";
embperl non-working script
[-
use warnings;
use strict;
use IPC::Open3;
my $cmd = 'ls';
my $pid = open3(*HIS_IN, *HIS_OUT, *HIS_ERR, $cmd);
close(HIS_IN); # give end of file to kid, or feed him
my #outlines = <HIS_OUT>; # read till EOF
my #errlines = <HIS_ERR>; # XXX: block potential if massive
print OUT "STDOUT: ", #outlines, "\n";
print OUT "STDERR: ", #errlines, "\n";
waitpid( $pid, 0 );
my $child_exit_status = $? >> 8;
print OUT "child_exit_status: $child_exit_status\n";
-]
Here is the output I receive
STDERR: ls: write error: Bad file descriptor
child_exit_status: 2
open3 redirects the file descriptor associated with STDOUT, excepting it to be fd 1 (what the program you exec will consider STDOUT). But it's not 1. It doesn't even have a file descriptor associated with it! I consider this a bug in open3. I think you can work around it as follows:
local *STDOUT;
open(STDOUT, '>&=', 1) or die $!;
...open3...
Thank you sooo much to ikegami!!!!
Here is the embperl code that works. P.S. There is a similar problem with STDIN. I don't know the solution to that yet, but I think it is similar.
[-
use warnings;
use strict;
use IPC::Open3;
use POSIX;
$http_headers_out{'Content-Type'} = "text/plain";
my $cmd = 'ls';
open(my $fh, '>', '/dev/null') or die $!;
dup2(fileno($fh), 1) or die $! if fileno($fh) != 1;
local *STDOUT;
open(STDOUT, '>&=', 1) or die $!;
my $pid = open3(*HIS_IN, *HIS_OUT, *HIS_ERR, $cmd);
close(HIS_IN); # give end of file to kid, or feed him
my #outlines = <HIS_OUT>; # read till EOF
my #errlines = <HIS_ERR>; # XXX: block potential if massive
print OUT "STDOUT: ", #outlines, "\n";
print OUT "STDERR: ", #errlines, "\n";
waitpid( $pid, 0 );
my $child_exit_status = $? >> 8;
print OUT "child_exit_status: $child_exit_status\n";
-]
Is it possible to time out a user input for the c shell? My code so far is :
#!/bin/csh -f
set COUNT = 5
printf "INFO: Start ok (0/1)? "
set INPUT = 0
while ($COUNT > 0 && $INPUT == 0)
printf "\b%d" "$COUNT"
set INPUT = <$
sleep 1
# COUNT --
end
if ($INPUT == 1) then
./execute.sh
end
If no input is given, I want to execute a shell script; if not i want to skip this part.
Unfortunately, the skript does not skip the input part but waits for the input. Any solutions for this problem?
Thanks a lot guys!!!
try this for non-blocking user input in tcsh shell:
set TMPFILE = `mktemp`
set COUNT = 5
printf "INFO: Start ok (0/1)? "
stty -F /dev/tty -icanon
while ($COUNT > 0 && -z $TMPFILE)
printf "\b%d" "$COUNT"
sleep 1
(dd bs=1 count=1 iflag=nonblock > $TMPFILE) >& /dev/null
set INPUT = `cat $TMPFILE`
# COUNT--
end
stty -F /dev/tty icanon
echo ""
if ("$INPUT" == "1") then
./execute.sh
endif