Due to a huge number of bots accessing my web server too many times a second, I went into my server and issued the following commands in an attempt to limit connections to a maximum of 25 in 5 seconds. I may need to be more strict, but I'm not because I have one page that is rarely accessed that requests 50 images and that might need 51 connections and I dont want users waiting forever for the remaining 26 pictures.
iptables -I INPUT -p tcp --dport 80 -i eth0 -m state --state NEW -m recent --set
iptables -I INPUT -p tcp --dport 80 -i eth0 -m state --state NEW -m recent --update --seconds 5 --hitcount 26 -j DROP
The first command works after verifying with iptables -nL but the second command immediately returns this message: "iptables: Invalid argument. Run `dmesg' for more information."
As soon as I executed it, I receive the following at the end of it:
Call Trace:
<IRQ> [<ffffffff8112f7ba>] ? __alloc_pages_nodemask+0x74a/0x8d0
[<ffffffff81299270>] ? swiotlb_map_page+0x0/0x100
[<ffffffff8116e282>] ? kmem_getpages+0x62/0x170
[<ffffffff8116ee9a>] ? fallback_alloc+0x1ba/0x270
[<ffffffff8116e8ef>] ? cache_grow+0x2cf/0x320
[<ffffffff8116ec19>] ? ____cache_alloc_node+0x99/0x160
[<ffffffff8116fb9b>] ? kmem_cache_alloc+0x11b/0x190
[<ffffffff8144d058>] ? sk_prot_alloc+0x48/0x1c0
[<ffffffff8144e262>] ? sk_clone+0x22/0x2e0
[<ffffffff8149ff16>] ? inet_csk_clone+0x16/0xd0
[<ffffffff814b97d3>] ? tcp_create_openreq_child+0x23/0x470
[<ffffffff814b6f8d>] ? tcp_v4_syn_recv_sock+0x4d/0x310
[<ffffffff814b9576>] ? tcp_check_req+0x226/0x460
[<ffffffff814b69cb>] ? tcp_v4_do_rcv+0x35b/0x490
[<ffffffffa0322557>] ? ipv4_confirm+0x87/0x1d0 [nf_conntrack_ipv4]
[<ffffffff814b825a>] ? tcp_v4_rcv+0x51a/0x900
[<ffffffff81495270>] ? ip_local_deliver_finish+0x0/0x2d0
[<ffffffff8149534d>] ? ip_local_deliver_finish+0xdd/0x2d0
[<ffffffff814955d8>] ? ip_local_deliver+0x98/0xa0
[<ffffffff81494a9d>] ? ip_rcv_finish+0x12d/0x440
[<ffffffff81495025>] ? ip_rcv+0x275/0x350
[<ffffffff8145c3eb>] ? __netif_receive_skb+0x4ab/0x750
[<ffffffff81460058>] ? netif_receive_skb+0x58/0x60
[<ffffffff81460160>] ? napi_skb_finish+0x50/0x70
[<ffffffff814618c9>] ? napi_gro_receive+0x39/0x50
[<ffffffffa024e267>] ? e1000_receive_skb+0x67/0x110 [e1000e]
[<ffffffffa0250e6f>] ? e1000_clean_rx_irq+0x37f/0x580 [e1000e]
[<ffffffffa025397d>] ? e1000e_poll+0x8d/0x2d0 [e1000e]
[<ffffffff814619e3>] ? net_rx_action+0x103/0x2f0
[<ffffffff8107a5f1>] ? __do_softirq+0xc1/0x1e0
[<ffffffff810e6c60>] ? handle_IRQ_event+0x60/0x170
[<ffffffff8107a64f>] ? __do_softirq+0x11f/0x1e0
[<ffffffff8100c30c>] ? call_softirq+0x1c/0x30
[<ffffffff8100fa75>] ? do_softirq+0x65/0xa0
[<ffffffff8107a4a5>] ? irq_exit+0x85/0x90
[<ffffffff81532525>] ? do_IRQ+0x75/0xf0
[<ffffffff8100b9d3>] ? ret_from_intr+0x0/0x11
<EOI>
The contents before refer to usb devices and other system devices. Can someone tell me how to successfully rate-limit incoming connections without experiencing random errors?
Executing dmesg reveals the cause of the error:
[78482.351825] xt_recent: hitcount (26) is larger than packets to be remembered (20)
This means that the recent module of iptables is limited to remembering just 20 packets. In order to solve this issue you can perform any of the following:
Reduce the value of the hitcount parameter in your iptables rule from 26 to 20 or less.
or
Increase the maximum possible value for the hitcount parameter from its default value of 20 by setting the option ip_pkt_list_tot of the xt_recent kernel module. This can be done by creating an ip_pkt_list_tot.conf file in /etc/modprobe.d/ which contains:
options xt_recent ip_pkt_list_tot=26
Once the file is created, reload the xt_recent kernel module via modprobe -r xt_recent && modprobe xt_recent or reboot the system.
Inspecting iptables's man page leads to the same conclusion:
--hitcount hits
This option must be used in conjunction with one of --rcheck or --update. When used, this will narrow the match to only happen when
the address is in the list and packets had been received greater than or equal to the given value. This option may be used along
with --seconds to create an even narrower match requiring a certain number of hits within a specific time frame. The maximum value
for the hitcount parameter is given by the "ip_pkt_list_tot" parameter of the xt_recent kernel module. Exceeding this value on the
command line will cause the rule to be rejected.
Note that xt_recent is called ipt_recent on some systems.
Related
I am working on one sles12 system where IPTables are configured in this way:
num target prot opt source destination
1 ACCEPT tcp -- anywhere anywhere tcp dpt:bctp owner UID match dd-test-user
2 DROP tcp -- anywhere anywhere tcp dpt:bctp
3 DROP all -- anywhere instance-data.us-west-2.compute.internal owner GID match test
4 ACCEPT all -- anywhere ip-100-34-0-0.us-west-2.compute.internal/21 owner GID match test
Can someone please help me understand this?
With IPTable rule 2, All packets will be dropped?
What does dpt:bctp mean here? I could not find anything about it manual.
Does Rule 4 even get chance to be applied for the process running from group id of "test" group?
I tried searching online documentation of iptables, but I could not find answer.
I found out that bctp stands for one of the protocol defined on the system
cat /etc/services | grep bctp
bctp 8999/tcp # Brodos Crypto Trade Protocol [Alexander_Sahler]
bctp 8999/udp # Brodos Crypto Trade Protocol [Alexander_Sahler]
Rule 2 is applied when destination port (dpt) is this protocol (8999/tcp). Rule 3 and 4 are applied for rest of ports from the process belonging to users from the group "test".
#!/bin/bash
#Oracle DB Info for NEXT
HOST="1.2.3.4"
PORT="5678"
SERVICE="MYDB"
DB_USER=$(whoami)
DB_PASS=$(base64 -d ~/.passwd)
DB_SCHEMA="my_db"
#Section for all of our functions.
function SQLConnection(){
sqlplus "$DB_USER"/"$DB_PASS"#"$HOST":"$PORT"/"$SERVICE"
}
function Connected(){
SQLConnection <<EOF
select sys_context('USERENV','SERVER_HOST') from dual;
EOF
}
function GetJMS(){
SQLConnection <<EOF
set echo on timing on lines 200 pages 100
select pd.destination from ${DB_SCHEMA}.pd_notification pd where pd.org_id = '$ORGID';
EOF
}
TODAY=$(date +"%A %B %d, %Y")
read -r -p $'\n\nWhat is the ORG ID? ' ORGID
read -r -p $'\n\nWhat is the REMOTE QUEUE MANAGER NAME? ' RQM
read -r -p $'\n\nWhat is the IP address of the REMOTE QUEUE MANAGER? ' CONN
read -r -p $'\n\nWhat is the PORT of the REMOTE QUEUE MANAGER? ' PORT
echo -en "* $(whoami)\n* $TODAY\n* MQ Setup $ORGID\n\nDEFINE +\n\tCHANNEL('$RQM.LQML') +\n\tCHLTYPE(SDR) +\n\tCONNAME('$CONN($PORT)') +\n\tXMITQ('BUF.2.$ORGID.XMQ')\n\tCHAUTH(TLS_RSA_WITH_AES_256_CBC_SHA256)\n\nDEFINE +\n\tCHANNEL('LQML.$RQM') +\n\tCHLTYPE(RCVR) +\n\tTRPTYPE(TCP)\n\nDEFINE +\n\tQLOCAL('$RQM') +\n\tTRIGDATA('LQML.$RQM') +\n\tINITQ('SYSTEM.CHANNEL.INITQ') +\n\tTRIGGER USAGE(XMITQ)\n\n" > ~/mqsetup.mqsc
CONNECTED=$(Connected | awk 'NR==16')
echo -en "\n\nHello From: $CONNECTED\n\n"
for JMSDESTINATION in $(GetJMS | awk 'NR>=16&&NR<=24{print $1}')
do
read -r -p $'\n\nWhich REMOTE QUEUE NAME matches with this ${JMSDESTINATION}?' RNAME
QDESC=$(echo "$JMSDESTINATION" | tr '.' ' ' | tr '[[:upper:]]' '[[:lower:]]')
echo -en "\n\nDEFINE +\n\tQR($JMSDESTINATION) +\n\t\tREPLACE DESCR('$ORGID $QDESC Queue') +\n\t\tREPLACE MAXDEPTH(5000) +\n\t\tXMITQ('BUF.2.$ORGID.XMQ') +\n\t\tRNAME('$RNAME') +\n\t\tRQMNAME('$RQM')" >> ~/mqsetup.mqsc
done
Here is the script I've built, hoping to automate the setup of IBM MQ Queues and Channels. My problem is that outside this script, I can establish an SQL Session without an issue, directly from the shell, provided I input the variables seen in the script. I can call the functions and everything returns just as I'd hope it would. When I run the exact same things from within the script, I get timeout errors ... the "Hello From" is blank, which tells me there is no DB connection.
I'm totally stumped as to why it all works great from outside the script, but inside it times out.
I appreciate the eyes and the help!
You're overwritng a variable value. You have this at the top of the script:
PORT="5678"
but then later on you do:
read -r -p $'\n\nWhat is the PORT of the REMOTE QUEUE MANAGER? ' PORT
which overwrites your 5678 value with whatever is entered there. That port may not be listening on the DB server at all, or may be doing something else, or if you don't enter a value it'll default to port 1521 when you connect. But either way the connection is going to fail, either quickly or slowly depending on the port state (e.g. slower maybe if a firewall blocks it).
If you test the connection by adding a Connected call before the read calls (as I initially did) then it seems to be working fine; but the connections after the reads don't work because port value it tries to connect to is now wrong.
Use a different name for the two variables, e.g. RQ_PORT for the second one - both in its read command and the subsequent creation of the ~/mqsetup.mqsc file.
You may also find it useful to add the -l flag to your SQL*Plus call so that if the connection fails for some reason it won't re-prompt for credentials, which in some circumstances can make the script appear to hang until you hit enter a few times.
Not directly relevant to the problem, but when automating anything like this I usually also use the -s flag to suppress the banners (which can vary between environments); and if you're only interested in capturing query output I'd usually set headings and/or pagination off, and feedback off, and generally set SQL*Plus up to generate as little noise as possible - it makes parsing out the interesting bits easier.
Can someone let me know how can I use redis-benchmark to do a benchmarking for HMSET, HGETALL with a fixed data size (-d option in redis-benchmark). I am using redis 3.2.5.
I have gone through this answer and tried the below command:-
root#cache-server1:~# redis-benchmark -h a.b.c.d -p XXXX hmset hgetall myhash rand_int rand_string -d 2048
====== hmset hgetall myhash rand_int rand_string -d 2048 ======
10000 requests completed in 0.11 seconds
50 parallel clients
3 bytes payload
keep alive: 1
99.64% <= 1 milliseconds
100.00% <= 1 milliseconds
89285.71 requests per second
But looking at the output it seems it is using only 3 bytes payload.
If it is not possible via redis-benchmark can someone suggest some other alternative?
The payload is only 3 bytes (the default) because the -d is taken as part of the command. The command must be the last argument, and all switches must precede it.
Besides that, you can't use redis-benchmark to run two custom commands. Also, the -d option is only applicable to predefined tests (the ones that run by default or with the -t option) and has no meaning if the user specifies the command used in the benchmark.
If you have a specific benchmarking flow that you want to test, the best thing you can do is mock it with any client that you're comfortable with.
I want to generate some trace file from disk IO, but the problem is I need the actual input data along with timestamp, logical address and access block size, etc.
I've been trying to solve the problem by using the "blktrace | blkparse" with "iozone" on the ubuntu VirtualBox environment, but it seems not working.
There is an option in blkparse for setting the output format to show the packet data, -f "%P", but it dose not print anything.
below is the command that i use:
$> sudo blktrace -a issue -d /dev/sda -o - | blkparse -i - -o ./temp/blktrace.sda.iozone -f "%-12C\t\t%p\t%d\t%S:%n:%N\t\t%P\n"
$> iozone -w -e -s 16M -f ./mnt/iozone.dummy -i 0
In the printing format "%-12C\t\t%p\t%d\t%S:%n:%N\t\t%P\n", all other things are printed well, but the "%P" is not printed at all.
Is there anyone who knows why the packet data is not displayed?
OR anyone who knows other way to get the disk IO packet data with actual input value?
As far as I know blktrace does not capture the actual data. It just capture the metadata. One way to capture real data is to write your own kernel module. Some students at FIU.edu did that in this paper:
"I/O deduplication: Utilizing content similarity to ..."
I would ask this question in linux-btrace mailing list as well:
http://vger.kernel.org/majordomo-info.html
I am wondering that how to generate massive packet-in messages to the controller to test the response time of SDN controller in the environment of Mininet.
Can you give me some advice on it?
You could use iperf to send packets, like this:
$ iperf -c -F
You could specify the amount of time:
$IPERF_TIME (-t, --time)
The time in seconds to transmit for. Iperf normally works by repeatedly sending an array of len bytes for time seconds. Default is 10 seconds. See also the -l and -n options.
Here is a nice reference for iperf: https://iperf.fr/.
If you would like to use Scapy, try this:
from scapy.all import IP, TCP, send
data = "University of Network blah blah"
a = IP(dst="129.132.2.21")/TCP()/data
send(a)