I have a website on Centos 7.4 running Apache 2.4.6
A user reported that they could not access the site on their mobile device (as of the last few days). I was able to recreate this issue on an Android Pixel. When I use a mobile 4G network, I immediately get an ERR_CONNECTION_RESET page if I try to go to any page of the website. When I use my home WIFI network on the phone, with the same browser, I can view all pages as expected.
I tested this, with the same results, on both Chrome and the native browser on this device.
I have enabled SSL logging in apache (ssl_engine, ssl_access and ssl_error) and none of these logs (nor the general httpd error log) show anything relating to this issue. I guess the connection doesn't reach apache?
There is nothing relating to this in the firewalld log either (though I'm not sure it would be logging such things?).
I have greped the fail2ban log too for the mobile network IP and found nothing.
I have also used Wireshark/Tshark to analyse the packets. Comparing the output when I access the site using WIFI (where it connects as expected) and when I use the mobile 4G network (where it fails to connect).
2 0.000065217 [Server IP] -> [Client IP] TCP 74 https > 63878 [SYN, ACK] Seq=0 Ack=1 Win=28960 Len=0 MSS=1460 SACK_PERM=1 TSval=244522926 TSecr=140533229 WS=128
3 0.003907594 [Client IP] -> [Server IP] TCP 74 63879 > https [SYN] Seq=0 Win=65535 Len=0 MSS=1326 SACK_PERM=1 TSval=140533229 TSecr=0 WS=256
4 0.003951717 [Server IP] -> [Client IP] TCP 74 https > 63879 [SYN, ACK] Seq=0 Ack=1 Win=28960 Len=0 MSS=1460 SACK_PERM=1 TSval=244522930 TSecr=140533229 WS=128
5 0.154191732 [Client IP] -> [Server IP] TCP 66 63879 > https [ACK] Seq=1 Ack=1 Win=87808 Len=0 TSval=140533290 TSecr=244522930
6 0.165703034 [Client IP] -> [Server IP] TCP 66 63878 > https [ACK] Seq=1 Ack=1 Win=87808 Len=0 TSval=140533290 TSecr=244522926
7 0.187358660 [Client IP] -> [Server IP] TCP 56 63879 > https [RST, ACK] Seq=1 Ack=1 Win=8222720 Len=0
8 0.204245316 [Client IP] -> [Server IP] TCP 56 63878 > https [RST, ACK] Seq=1 Ack=1 Win=8222720 Len=0
This is as far as the connecting goes on the mobile network. We never get to the next stage which is "SSL 571 Client Hello" on WIFI (the client sending the server a hello).
Any ideas on what could be causing this, or ways I can debug this issue further?
So I've finally got to the bottom of this error, after a few days of scratching my head and spending a lot of time researching and fiddling with the server config.
It turns out the website has been blocked by a couple of ISPs who have decided to now deem the website as containing adult content (it's doesn't and has never had this issue before in its 10 year history).
Rather than providing any kind of useful redirect to let a user know why they can't access the site, they simply don't send a hello packet and up comes the ERR_CONNECTION_RESET.
I must say, this is something that had crossed my mind early on in my investigations. However, rather unhelpfully, when logging into my mobile phone account it said the adult content filter was off. I discovered this wasn't in fact the case when I called them to double check!
If you think this could be happening to you, here's some quick checks:
If all other websites seem to be working fine, try to visit an adult website and see if you also get an ERR_CONNECTION_RESET
Then check with your ISP to see if it's blocking adult content.
Related
My Rails app which has been working great for years suddenly started returning Cloudflare 520 errors. Specifically, api.exampleapp.com backend calls return the 520 whereas hits to the frontend www.exampleapp.com subdomain are working just fine.
The hard part about this is nothing has changed in either my configuration, or code at all. Cloudflare believes this is happening as the Heroku server is returning an empty response.
> GET / HTTP/1.1
> Host: api.exampleapp.com
> Accept: */*
> Accept-Encoding: deflate, gzip
>
{ [5 bytes data]
* TLSv1.2 (IN), TLS alert, close notify (256):
{ [2 bytes data]
* Empty reply from server
* Connection #0 to host ORIGIN_IP left intact
curl: (52) Empty reply from server
error: exit status 52
On the Heroku end, my logs don't even seem to register the request when I hit any of these urls. I also double-checked my SSL setup (Origin certificate created at Cloudflare installed on Heroku), just in case, and it seems to be correct and is not expired.
The app is down for a couple of days now, users are complaining, and no response from either customer care teams despite being a paid customer. My dev ops knowledge is fairly limited.
Welcome to the club: https://community.cloudflare.com/t/sometimes-a-cf-520-error/288733
It seems to be a Cloudflare issue introduced in late July affecting hundreds of sites running very different configurations. It's been almost a month since the issue was first reported, Cloudflare "fixed" it twice, but it's still there. Very frustrating.
Change your webserver logs to a info state and see if your application is not exceeding some HTTP/2 directive while processing the connection.
If this is the case, try to increase the directive size:
#nginx
server {
...
http2_max_field_size 64k;
http2_max_header_size 64k;
}
I am trying to create a SAP RFC connection to a new system.
AFAIK the firewall (in this case to port 3321) is open.
I get this message at the client:
RFC_COMMUNICATION_FAILURE (rc=1): key=RFC_COMMUNICATION_FAILURE, message=
LOCATION SAP-Gateway on host ax-swb-q06.prod.lokal / sapgw21
ERROR timeout during allocate
TIME Thu Jul 26 16:45:48 2018
RELEASE 753
COMPONENT SAP-Gateway
VERSION 2
RC 242
MODULE /bas/753_REL/src/krn/si/gw/gwr3cpic.c
LINE 2210
DETAIL no connect of TP sapdp21 from host 10.190.10.32 after 20 sec
COUNTER 3
[MSG: class=, type=, number=, v1-4:=;;;]
And this message on the SAP server
Any clue what needs to be done, to get RFC working?
With this little info no one can know what the issue is here.
But it is something related to your network and SAP system configuration.
I guess your firewall does some network address translation (NAT) and the new IP behind the firewall does not match anymore with the known one. SAP is doing some own IP / host name security checks.
If not already done, check with opening the ports 3221, 3321 and 4821 in the firewall. Also check the SAP gateway configuration which IP addresses and host names are configured to be valid ones for it (look at what is traced in the beginning of the gateway trace file dev_rd at ABAP side).
Also consider if maybe the usage of a SAProuter would be the better option for your needs.
it works in my case if ashost is the host name, and not an IP address!
Do not ask me why, but this fails:
Connection(user='x', passwd='...', ashost='10.190.10.32', sysnr='21', client='494')
But this works:
Connection(user='x', passwd='...', ashost='ax-swb-q06.prod.lokal', sysnr='21', client='494')
This is strange, since DNS resolution happens before TCP communication.
It seems that the ashost value gets used inside the connection. Strange. For most normal protocols (http, ftp, pop3, ...) this does not matter. Or you get at least a better error message.
I have 2 routers in my network.
A) The one issued by my ISP (limited settings, had even to ask to get portforwarding settings), which is alo my modem.
B) My own router (wher i set my DHCP etc)
Now the generated resolve.txt on raspberrian and archlinux list:
domain local
nameserver <IP of A>
nameserver <IP of B>
As in understand it this is the order it will try to use when resolving names, but her it schould try my internal B before trying to resolve using A.
PS: Both subnetmasks are 255.255.255.0
Router A has 192.168.0.1
Router B has 192.168.1.1
All devices are in the 192.168.1.### range.
PPS: Archlinux is setup to use networkmanager, not a manual configured dhcpcd
NetworkManager may use dnsmasq for dhcp and to handle dns lookups.
I noticed that dnsmasq reverses the order of nameservers. Look at your logs. That would show up better in log if we also set dnsmasq to call dns servers in parallel:
#/etc/dnsmasq.conf
#all-servers
#/etc/dnsmasq.d/laptop.conf
all-servers
log-queries=extra
log-async=100
log-dhcp
#/etc/dnsmasq.d/servers.conf
server=66.187.76.168
server=162.248.241.94
server=165.227.22.116
/var/log/dnsmasq.log--
Mar 14 02:14:20 dnsmasq[3216]: 71700 127.0.0.1/38951 cached firefox.settings.services.mozilla.com is <CNAME>
Mar 14 02:14:20 dnsmasq[3216]: 71700 127.0.0.1/38951 forwarded firefox.settings.services.mozilla.com to 165.227.22.116
Mar 14 02:14:20 dnsmasq[3216]: 71700 127.0.0.1/38951 forwarded firefox.settings.services.mozilla.com to 162.248.241.94
Mar 14 02:14:20 dnsmasq[3216]: 71700 127.0.0.1/38951 forwarded firefox.settings.services.mozilla.com to 66.187.76.168
...order of calls is reversed in log lines!
I got rid of systemd-resolved to rely on dnsmasq.
I am installing the datapusher service for CKAN.
CKAN has been configured to use an NGINX reverse proxy that routes client requests, following instructions here. SSL certificate is installed and configured in NGINX.
When trying to use the datapusher app to upload a file, it fails and Apache log gives this error:
Mon Apr 03 13:49:10.979179 2017] [:error] [pid 15468] 2017-04-03 13:49:10,979 CRITI [ckanext.datapusher.plugin] {'status_code': 403, 'message': 'An Error occurred while sending the job: 403 Client Error: Forbidden', 'details': u'<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">\\n<html><head>\\n<title>403 Forbidden</title>\\n</head><body>\\n<h1>Forbidden</h1>\\n<p>You don\\'t have permission to access /job\\non this server.</p>\\n<hr>\\n<address>Apache/2.4.7 (Ubuntu) Server at 127.0.0.1 Port 8800</address>\\n</body></html>\\n'}
When testing access to the datapusher's 8800 port through openssl this is the output:
[Mon Apr 03 13:49:10.981049 2017] [:error] [pid 15468] [remote 127.0.0.1:6855] Error - <type 'exceptions.TypeError'>: notify() takes exactly 3 arguments (2 given)
open:/etc/ckan> openssl s_client -connect 127.0.0.1:8800
CONNECTED(00000003)
140385459791520:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:794:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 275 bytes
The datapusher docs give a workaround for bypassing SSL here, using the SSL_Verify config. I tried this and there was no change.
I think that I need to either:
1. Force the nginx reverse proxy to allow SSL connections through port 8800 (in addition to 443). Or...
2. Configure ckan/datapusher/apache/nginx to bypass SSL/https on port 880.
Any suggestions?
I believe the 403 error is at the point that CKAN sends a request to DataPusher to ask it to load a particular resource. DataPusher is running on Apache only and thus is on HTTP (not HTTPS) so there should be no issue with SSL. Check your CKAN config is the default:
ckan.datapusher.url = http://127.0.0.1:8800/
DataPusher's SSL_VERIFY setting is for a later request - when datapusher makes a request to CKAN at ckan.site_url, which for you will go via nginx over HTTPS. You may need this setting, depending on whether the SSL in your python is compatible. Reading the code it suggests you need quotes and make sure the key is all caps. i.e. in your datapusher_settings.py:
SSL_VERIFY = 'False'
I am trying to call an old COM library from my XBAP and continue to receive the following exception:
System.AccessViolationException was unhandled
Message: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
I have tried adding the HKLM value for RunUnrestricted to no avail.
I don't get anything else but this error when calling the library. Any ideas? (This library even works from a pure ASP.NET app)
EDIT:
The COM library makes socket calls to a server. It looks like that is happenning but somewhere after the last packet, it bombs with this error.
No. Time Source Destination Protocol Info
10 8.452945 10.10.10.210 10.10.10.250 TCP 50736 > 22700 [SYN] Seq=0 Win=8192 Len=0 MSS=1260 WS=8
14 11.459350 10.10.10.210 10.10.10.250 TCP 50736 > 22700 [SYN] Seq=0 Win=8192 Len=0 MSS=1260 WS=8
21 17.459690 10.10.10.210 10.10.10.250 TCP 50736 > 22700 [SYN] Seq=0 Win=8192 Len=0 MSS=1260
try trusting the site. e.g. trusted sites for IE.
I wish there was a way to cancel a question cause this one is my fault. I was tunneled through a VPN and just had a bad connection no matter how many times I reconnected. After a restart, I was then able to interact with the API.