Total Number of connections for each connected Redis Service - redis

How to check which service is consuming more resources on Redis.
Or which service has the highest number of connections on Redis?

You can type the command "CLIENT LIST", You'll see these like :
id=39 addr=127.0.0.1:34706 fd=7 name= age=141156 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=client
id=78 addr=127.0.0.1:58014 fd=5 name= age=63779 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=llen
id=80 addr=127.0.0.1:36826 fd=6 name= age=46776 idle=1685 flags=N db=1 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=del
The most useful keys for your answer is "age" && "idle" , "age" means the total duration of the connection in seconds and "idle" means idle time of this connection. so (age - idle) / age relects this client uses server's cpu more than other client which the value is smaller , but not very precisely
Other command also can give you some suggestions, like "INFO" and "MONITOR" .
INFO gives you a statistics information about redis server , such as the memory usage, the command processed, the cpu usage, the connected clients and so on , you can refer this to get more.
"MONITOR" gives you a real time display which says that what happens now, what command is being executed, who sent this command. Maybe you can compute the every client resource using by the MONITOR output.
e.g.
for every command, you first parse it and using a cost to add the client cost sum. In time consuming computing, SET is O(1) and Lrange is O(N). But this is also difficult to do this very precisely. But you can log this using this command like :
redis-cli monitor > redis-command.log
you can use this log to do some analytics. but notes that MONITOR command will reduce your redis server throughput, check this

If you run the "client list" command against your Redis instance, you should be able to see the entire list of clients connected to your redis instance along with their IP addresses. You can then see which clients (services) have the highest number of connections to your Redis instance.

Do simply
info clients
output
connected_clients:xxx
client_longest_output_list:xxx
client_biggest_input_buf:x
blocked_clients:xx

To get which client is having higher number of connection we can use the below shell script
#!/bin/bash
# Get the list of clients with their connection count
clients=$(redis-cli client list | awk '{print $2}' | sort | uniq -c | sort -rn)
# Print the client with the highest number of connections
highest=$(echo "$clients" | head -n 1)
echo "Client with the highest number of connections: $highest"
This will provide the top first client which has more no of clients I hope it will help!!

Related

Redis pipe mode throw client reached max query buffer length?

I'm trying to import about 3GB neo4j cypher script data into redis, using the redis pipe mode.
docker image:
redislabs/redisgraph:2.8.17
redis version:
Redis server v=6.2.6 sha=00000000:0 malloc=jemalloc-5.1.0 bits=64 build=e15697f1a083b6bc
redisGraph module doc: redis-graph-doc
I refer to the following documents and use python scripts to generate data that conforms to the redis protocol
redis-bulk-loading-doc
data example:
cat -A redis_command.txt | head
*3^M$
$11^M$
GRAPH.QUERY^M$
$9^M$
knowledge^M$
$280^M$
MERGE (n:xxxx) SET n.name="xxx", n.nebula_id="xxx", n.concept_id="xxx", n.create_time="xxx", n.data_source="xxx", n.entity_tag="xxx", n.version="xxx"^M$
^M$
*3^M$
$11^M$
GRAPH.QUERY^M$
$9^M$
knowledge^M$
$270^M$
MERGE (n:xxxx) SET n.name="xxx", n.nebula_id="xxx", n.concept_id="xxx", n.create_time="xxx", n.data_source="xxx", n.entity_tag="xxx", n.version="xxx"^M$
After that I use the following command for data import
cat redis_command.txt | redis-cli --pipe
But the command runs for about 1 minute, and it will automatically exit.
Only this information is printed in the log file of redis
Closing client that reached max query buffer length: id=10084 addr=127.0.0.1:34990 laddr=127.0.0.1:6379 fd=11 name= age=47 idle=0 flags=b db=0 sub=0 psub=0 multi=-1 qbuf=1073756036 qbuf-free=268421234 argv-mem=293 obl=0 oll=0 omem=0 tot-mem=1342198077 events=r cmd=graph.QUERY user=default redir=-1 (qbuf initial bytes: "\r\n*3\r\n$11\r\nGRAPH.QUERY\r\n$9\r\nknowledge\r\n$274\r\nMERGE (n:`xxxxxxxxx")
I don't know where the problem is, how should I solve this problem?

Sending command to GPS device using gpsd python library

I use the gpsd python library in order to read and parse the NMEA string recieved from the gps device. I would like to send some command to gps in order to fine tune the measurement rate, report rate and so on.
It is possible using the gpsd library or I must send the command with in other way?
According to 'gpsd' manual:
To send a binary control string to a specified device, write to the
control socket a '&', followed by the device name, followed by '=',
followed by the control string in paired hex digits.
So if you have gpsd service running as gpsd -F /var/run/gpsd.sock you can use the following code to send commands to the gps device:
import socket
import sys
# Create a socket
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
# Connect the socket to the port where the GPSD is listening
gpsd_address = '/var/run/gpsd.sock'
sock.connect(gpsd_address)
# #BSSL 0x01<CR><LF> - Set NMEA Output Sentence (GGA only)
# cmd = '#BSSL 0x01' + '\r\n'
# #RST<CR><LF> - RST - Reset
cmd = '#RST' + '\r\n'
message = '&/dev/ttyUSB1='
cmd_hex = cmd.encode('utf-8').hex()
print ('cmd_hex {}'.format(cmd_hex))
# cmd_hex 405253540d0a
message += cmd_hex
bin_message = message.encode('utf-8')
print ("bin message {}".format(bin_message))
# bin message b'&/dev/ttyUSB1=405253540d0a'
sock.sendall(bin_message)
data = sock.recv(16)
print ('received {}'.format(data))
# received b'OK\n'
sock.close()
In my case I am sending #RST command followed by CR, LF symbols.

Exclude some text from log output with syslog-ng

i have log message
"2020-11-06T14:28:19.171900+0000 Host-9999 1889 1889 DBG some part of the log"
Is it possible to filter messages like this with syslog-ng functionality to have such output
"DBG some part of the log"
Basically i need to exclude date, host, pid, pid in every log message.
i raised it with rewrite function, and with regex
rewrite test{
subst("^.{0,54}", " ", value("MSG"));
};

New line character within at*smsm2m message string

I'm trying to send a multiline sms from an application I am creating to my phone via telnet to a GSM modem. I would like the message to be output to my phone as shown below but cannot figure out how to add new lines within the message string so that the message is output to my phone as below. I cannot find much documentation on this either and the few character codes (\r\n) I tried either terminated the telnet command were they were displayed in my code or were showing in the SMS received on my phone.
Does anybody know what character code that would give me a new line without terminating the telnet command within the message string?
at*smsm2m= “441234567891
Pinging 192.168.0.31 with 32 bytes of data:
Reply from 192.168.0.31: bytes=32 time=3ms TTL=64<br>
Reply from 192.168.0.31: bytes=32 time=1ms TTL=64<br>
Reply from 192.168.0.31: bytes=32 time=1ms TTL=64<br>
Reply from 192.168.0.31: bytes=32 time=1ms TTL=64
Ping statistics for 192.168.0.31:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
The server side of Telnet is typically just a regular shell so you can do the same things you'd do for embedded newlines as if you were typing at a command line. Try this:
echo -e "this\nhas\nmultiple\nlines" | program
(you may need to look up flags for the echo command on the server if it's not a GNU machine)

How to craft specific packets on the host of Mininet to generate massive Packet-In messages

I am wondering that how to generate massive packet-in messages to the controller to test the response time of SDN controller in the environment of Mininet.
Can you give me some advice on it?
You could use iperf to send packets, like this:
$ iperf -c -F
You could specify the amount of time:
$IPERF_TIME (-t, --time)
The time in seconds to transmit for. Iperf normally works by repeatedly sending an array of len bytes for time seconds. Default is 10 seconds. See also the -l and -n options.
Here is a nice reference for iperf: https://iperf.fr/.
If you would like to use Scapy, try this:
from scapy.all import IP, TCP, send
data = "University of Network blah blah"
a = IP(dst="129.132.2.21")/TCP()/data
send(a)