302 Redirect from CGI script stopped working in Apache 2.4 - cgi

I inherited maintenance of a self-written CGI application without
documentation and have never met the original author. The application
stopped working in Debian 8, but worked in Debian 7 and CentOS 5. The
main changes were the upgrade from Apache 2.2 (used by Debian 7 &
CentOS 5) to Apache 2.4 (used by Debian 8) and the upgrade from perl
5.8 (in CentOS 5) respectively perl 5.14 (in Debian 7) to perl 5.20.
The problematic part boils down to the following script (a
302-redirect):
#!/usr/bin/perl
$|=1; # activate auto-flushing of stdout
use strict;
use warnings;
my $CRLF = "\015\012";
print STDOUT "Status: 302 Moved Temporarily$CRLF" .
"Location: /does_not_matter$CRLF" .
"URI: /does_not_matter$CRLF" .
"Connection: close$CRLF" .
"Content-type: text/html; charset=UTF-8$CRLF$CRLF";
close STDOUT;
while(1) {
sleep 1;
}
The observed behavior is that the redirect never reaches the client
as long as the script is still running when used with Apache 2.4, but
there is no error message in Apache's error.log. I changed the client
(Firefox, Chromium, wget), the Apache module (mod_cgid & mod_cgi),
sent additional headers, removed the close STDOUT, removed the
$|=1, replaced the $CRLF with \n and made the script
fork and exit the parent process (so that Apache is no longer the
parent process), all to no avail. The only things that worked: use
Apache 2.2, turn the script into an NPH-CGI-script (which has to send
complete HTTP headers that Apache will not modify in any way, even if
they contain errors), make the script exit instead of entering an
endless loop. I confirmed via tcpdump that the packages with the
redirect indeed never leave the server before the script is killed.
And from the Date-line in the response and the time of the eventual
arrival I gather that Apache receives my output immediately (&
immediately adds the Date-line to the headers), but does not send the
response to the client.
Don't bother answering, I already figured the solution out by myself
and will write an answer. I just want to make the solution available
to others who might encounter the same problem.

The problem was the missing message body. A 302 redirect may contain
a message body (see RFC 2616, section 4.3 (Message Body): "All other
responses do include a message-body, although it MAY be of zero
length"), but a Content-Length-line is optional (section 4.4 of RFC
2616 says the message body length can be determined by closing the
connection if the Content-Length-line is missing). Since Apache can
not know whether I want to send a message body or not, it has to wait
until I close the connection or actually send the message body
(Apache 2.2 apparently behaved erroneously here by not waiting for
the message body - or maybe close STDOUT; does not do in perl
5.20 what it did in older perl versions). The correct script should
therefore look like this (verified to work both in Apache 2.2 and in
Apache 2.4) - the only difference is an additional $CRLF which
terminates a zero-length message body:
#!/usr/bin/perl
$|=1; # activate auto-flushing of stdout
use strict;
use warnings;
my $CRLF = "\015\012";
print STDOUT "Status: 302 Moved Temporarily$CRLF" .
"Location: /doesNotMatter$CRLF" .
"URI: /doesNotMatter$CRLF" .
"Connection: close$CRLF" .
"Content-type: text/html; charset=UTF-8$CRLF$CRLF$CRLF";
close STDOUT;
while(1) {
sleep 1;
}
It was https://stackoverflow.com/a/8062277/2845840 that pointed me in the right direction.

Related

apache + mod_perl + couchbase = occasional connection problems

We use couchbase as session storage for mod_perl scripts. To avoid delays on clients caused by waiting for a new connection we do preconnect to couchbase on child_init apache stage. So during apache restart / new child creation it connects to couchbase automatically and later use that connection during apche child lifetime.
Generally everything works fine, but sometimes we got the following errors during that preconnection:
Couldn't connect: 0x13 (Operation not supported) at /perl/lib64/perl5/Couchbase/Bucket.pm line 38.
Usually it appears during apache restart and on several (or dozens) of childs, and almost never on one child only. Usually restarting apache again solves the problem.
What can cause such a problems? Is it a problem with code / server configuration / couchbase server itself?
May be it caused somehow with a lot of reconnections at the same time? Some ulimits stuff / or selinux restrictions?
UPD: versions
OS:
Centos 6, 2.6.32-358.2.1.el6.x86_64
libcouchbase:
libcouchbase-devel.x86_64 2.4.7-1.el6
libcouchbase2-core.x86_64 2.4.7-1.el6
libcouchbase2-libevent.x86_64 2.4.7-1.el6
couchbase server:
2.2.0 community edition (build-837)
SDK:
perl (Couchbase::Core v2.0.2)
connection code (isolated & simplified):
# in mod_perl environment
use Couchbase;
use Couchbase::Bucket;
use Couchbase::Document;
use Apache2::ServerUtil ();
my $cb = undef;
# connection handler, initialized once, used during apache child lifetime
sub connect_couchbase_on_child_init {
my ($child_pool, $s) = #_;
my $dsn = 'couchbase://192.168.0.1,192.168.0.2/my_bucket_name?detailed_errcodes=1';
eval { $cb = Couchbase::Bucket->new($dsn); };
# here we get the occasional warnings during apache restarts
if ($#) { warn "COUCHBASE CONNECTION ERROR! $#"; $cb = undef; }
return Apache2::Const::OK;
}
Apache2::ServerUtil->server->push_handlers(PerlChildInitHandler => \&connect_couchbase_on_child_init);
# in request handlers it used with the following calls (only if connected):
# $doc = Couchbase::Document->new($key);
# $cb->get($doc);
# ...
# $cb->replace($doc);
# ...
# $cb->insert($doc);
# ...
# $cb->remove($doc);
Because you are using server 2.2.0 and because this seems to happen when you are connecting many clients at once, my theory is that you are receiving the last error from the server. The current client bootstrap process attempts using bootstrap over memcached (which is only supported from version >= 2.5.0 of the server), that fails and it attempts to use 'terse' bootstrapping (again, only supported on >= 2.5.0 of the server) and finally 'classic' HTTP (which is available on all versions).
Add the following options to your DSN/connection string to cut out some of the steps for your server. Note that should you ever upgrade to >= 2.5 these options should be removed:
bootstrap_on=http Does not try memcached bootstrap
http_urlmode=2 Uses the pre-2.5 style of bootstrapping by default
These two options will not necessarily fix your issue, but they will at least cut out some of the initial connection time, and perhaps show a clearer reason for the error (you can also set LCB_LOGLEVEL=5 in the environment to get actual logging).
In your case, the connection string would be:
couchbase://192.168.0.1,192.168.0.2/my_bucket_name?detailed_errcodes=1&bootstrap_on=http&http_urlmode=2

"End of script output before headers" - CGI on Apache

I have the following CGI script:
#!c:\cygwin\bin\perl.exe
use CGI qw(:standard);
my $query = $CGI->new;
print header (
-type => "text/html",
-status => "404 File not found"
);
print "<b>File not found</b>";
This gives me an error:
Server error!
The server encountered an internal error and was unable to complete your request.
Error message:
End of script output before headers: test.cgi
If you think this is a server error, please contact the webmaster.
Error 500
127.0.0.1
Apache/2.4.10 (Win32) OpenSSL/1.0.1h PHP/5.4.31
I've looked at this (and other similar) question(s), but there the headers were not being printed, as opposed to mine.
I'm using the XAMPP Windows package with Cygwin Perl.
Can anyone help? Thanks.
I don't know why you are using $CGI instead of CGI
I think It should be
my $query = CGI->new;
tested on Linux working perfect.
So, as others have pointed out, your problem was using a variable ($CGI) where you actually needed a class name (CGI). But, in my mind, this raises two more questions.
1/ Why are you trying to create a CGI object in the first place? You are using the function-based interface to CGI (print header(...) for example) so there's no need for a CGI object.
2/ Why are you writing a CGI program in 2014? Perl web programming has moved on a long way this millennium and you seem to be stuck in the 1990s :-/

Magento Soap Error - Premature end of data in tag definitions line 2

My client is using Unleashedsoftware.com to connect to a Magento Store. But it gives this error.
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Body>
<SOAP-ENV:Fault>
<faultcode>WSDL</faultcode>
<faultstring>
SOAP-ERROR: Parsing WSDL: Couldn't load from 'http://www.domain.com/index.php/api/v2_soap/index/wsdl/1/' : Premature end of data in tag definitions line 2
</faultstring>
</SOAP-ENV:Fault>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
When browsing http://www.domain.com/index.php/api/v2_soap/index/ Firebug gives me “500 Internal Service Error”.
When I browse http://www.domain.com/index.php/api/v2_soap/index/wsdl/1/, I am getting valid XML data.
I checked the server log files and it seems like:
[Thu Aug 30 22:22:25 2012] [warn] [client 92.92.92.92] mod_fcgid: stderr: in /home/doaminuser/public_html/lib/Zend/Soap/Server.php on line 762
I been searching for couple of days now and today I tried to duplicate the entire site to another test server, and it seems to be working! So that seems to be a server issue.
Please, anybody got any idea what could be the issue?
Is there any better way of debugging this issue, any sample code or debugging tips.
Magento version is 1.6.2
Thank you.
There's lots of times where Magento's SOAP API fails due to problems your Magento server has communicating with itself.
That is, PHP's SOAP implementation requires that the SOAP server itself fetch the WSDL file via http, and a local network configuration issue gets in the way of Magento fetching it's own WSDL.
You can debug this by SSHing into your Magento server, and running the following command
curl -l 'http://www.example.com/index.php/api/v2_soap/index/wsdl/1/' > /tmp/wsdl.xml
and then examining the wsdl.xml file. Because you're performing this from your web-server, you may get different results than when you're performing it from your local browser.
I had a similar problem when calling the URL
http://www.store.com/index.php/api/v2_soap/?wsdl
After some time I received the message 500 - Internal Server Error and a Premature end of script headers message in the apache error log.
After a whole day of research I figured out, that the Timeout-Directive of the Apache module (configured in httpd.conf on a Linux environment) was set to "20" which caused the server to send the 500 error after 20 seconds. The problem is, that in my case the Magento system needs a longer time to "crawl" through all wsdl.xml files in order to build the WSDL-output (if you are using Magento SOAPv2).
Maybe you should check your Timeout Directive..hope that helps.
"I have memories of this. What worked for me was to put the hostname
in /etc/hosts on the server plus the www alias on 127.0.0.1 However,
in this instance the server was in the building rather than in some
ISP place and the LAN had Windows computers on it. Windows users had
downloaded lots of trojan-virus-porn things that were spending the
whole time spamming the network so the real problem was with the
Windows computers on the network, not with the server or with Magento.
After fdisking the PC's the problem was solved."
Thank You I've been struggling for 2 days with this on magento 1.6 and Windows Server 2008 adding this line to the hosts file (C:\Windows\System32\drivers\etc) solved the issue for me:
127.0.0.1 www.Domain.com
also remember to fix your magento soap (role) because the Roles Resources doesn't save in 1.6 unless you fix this file:
MagentoRoot\app\code\core\Mage\Adminhtml\Block\Api\Tab\Rolesedit.php
replace this:
if (array_key_exists(strtolower($item->getResource_id()), $resources) && $item->getPermission() == 'allow') {
with this:
if (array_key_exists(strtolower($item->getResource_id()), $resources) && $item->getApiPermission() == 'allow') {
In my case the issue was the Mod_Security rule "PHP Easter Egg Access" was enabled.
Rule ID: 380800
Once disabled, the api access worked.
An indicator was in the Apache log file:
Jun 19 09:15:52 httpd[1024961]: [error] [client xyz.xyz.xyz.xyz] ModSecurity: [file "/usr/local/apache/conf/modsec/99_asl_jitp.conf"] [line "116"] [id "380800"] [rev "1"] [msg "Atomicorp.com WAF Rules - Virtual Just In Time Patch: PHP Easter Egg Access"] [data "phpe9568f35-d428-11d2-a769-00aa001acf42"] [severity "CRITICAL"] Access denied with code 403 (phase 2). Pattern match "php(?:e9568f3[56]-d428-11d2-a769-00aa001acf42|b8b5f2a0-3c92-11d3-a3a9-4c7b08c10000)" at REQUEST_URI. [hostname "www.yoursever.com"]...
Magento version: 1.7.0.2
PHP version: 5.3.26
More information about the PHP Easter Egg Access rule:
http://www.atomicorp.com/forums/viewtopic.php?f=3&t=5057
http://www.0php.com/php_easter_egg.php
For those wanting a quick test script to replicate the issue (useful when trying to convince your hosting provider that it's a problem on their end), use:
<?php
$server = new SoapServer("http://<url to your magento shop>/index.php/api/v2_soap/index/wsdl/1/");
?>
This is the line in /lib/Zend/Soap/Server.php that triggers the error.
In my case if you browsed to:
http://< url to your magento shop >/index.php/api/v2_soap/index/wsdl/1/
the xml was fine, but if you ran the above php script on the server, the error was given.
This error most often appeared for me while omitting www for domain given in Magento SOAP url. Url has to match base url specified in the Magento config.

Apache, mod_ssl "request failed: error reading the headers" for a specific user

Currently we have an Apache 2.2.3 server with mod_ssl 2.2.3 running Django, with users authenticating by using a x509 certificate.
So far the system is running perfectly except for a single user, who when trying to upload a file receives 400 Bad Request error, and the contents of the ssl_error_log regarding this operation are:
[<date>] [error] [client <client ip>] request failed: error reading the headers, referer: <referrer url>
The contents of the ssl_access_log are:
<client ip> - - [<date>] "POST <target page> HTTP/1.1" 400 321
Also, the user's browser is Firefox as far as I know.
I am completely unable to reproduce this bug and so far none of the other users have experienced it. Could you point out some reasons for this to happen?
I've experienced connectivity that stops the upstream after an X amount of bytes is sent. X was a pretty low value, as in enough to request some simple pages, but not to deal with ajax requests much less upload files. As far as I recall, this connectivity problem occurred only when tethering (from a specific Android phone, but I didnt even test other phones).
So if the upstream gets interrupted and the upload stalls, it makes sense apache would return this error, according to this post: "Apache waits a time equal to the Timeout directive (defaults to 5 minutes if not defined) for a response from the client. It is likely Apache is waiting for the CRLF that indicates the end of the headers, yet it is never received.."

Unexpected Connection Reset: A PHP or an Apache issue?

I have a PHP script that keeps stopping at the same place every time and my browser reports:
The connection to the server was reset
while the page was loading.
I have tested this on Firefox and IE, same thing happens. So, I am guessing this is an Apache/PHP config problem. Here are few things I have set.
PHP.ini
max_execution_time = 300000
max_input_time = 300000
memory_limit = 256M
Apache (httpd.conf)
Timeout 300000
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 0
Are the above correct? What can be causing this and what can I set?
I am running PHP (5.2.12.12) as a
module on Apache (2.2) on a Windows
Server 2003.
It is very likely this is an Apache or PHP issue as all browsers do the same thing. I think the script runs for exactly 10 mins (600 seconds).
I had a similar issue - turns out apache2 was segfaulting. Cause of the segfault was php5-xdebug for 5.3.2-1ubuntu4.14 on Ubuntu 10.04 LTS. Removing xdebug fixed the problem.
I also had this problem today, it turned out to be a stray break; statement in the PHP code (outside of any switch or any loop), in a function with a try...catch...finally block.
Looks like PHP crashes in this situation:
<?php
function a ()
{
break;
try
{
}
catch (Exception $e)
{
}
finally
{
}
}
This was with PHP version 5.5.5.
Differences between 2 PHP configs were indeed the root cause of the issue on my end. My app is based on the NuSOAP library.
On config 1 with PHP 5.2, it was running fine as PHP's SOAP extension was off.
On config 2 with PHP 5.3, it was giving "Connection Reset" errors as PHP's SOAP extension was on.
Switching the extension off allowed to get my app running on PHP 5.3 without having to rewrite everything.
I had an issue where in certain cases PHP 5.4 + eAccelerator = connection reset. There was no error output in any log files, and it only happened on certain URLs, which made it difficult to diagnose. Turns out it only happened for certain PHP code / certain PHP files, and was due to some incompatibilities with specific PHP code and eAccelerator. Easiest solution was to disable eAccelerator for that specific site, by adding the following to .htaccess file
php_flag eaccelerator.enable 0
php_flag eaccelerator.optimizer 0
(or equivalent lines in php.ini):
eaccelerator.enable="0"
eaccelerator.optimizer="0"
It's an old post, I know, but since I couldn't find the solution to my problem anywhere and I've fixed it, I'll share my experience.
The main cause of my problem was a file_exists() function call.
The file actually existed, but for some reason an extra forward slash on the file location ("//") that normally works on a regular browser, seems not to work in PHP. Maybe your problem is related to something similar. Hope this helps someone!
I'd try setting all of the error reporting options
-b on error batch abort
-V severitylevel
-m error_level
and sending all the output to the client
<?php
echo "<div>starting sql batch</div>\n<pre>"; flush();
passthru('sqlcmd -b -m -1 -V 11 -l 3 -E -S TYHSY-01 -d newtest201 -i "E:\PHP_N\M_Create_Log_SP.sql"');
echo '</pre>done.'; flush();
My PHP was segfaulting without any additional information as to the cause of it as well. It turned out to be two classes calling each other's magic __call() method because both of them didn't have the method being called. PHP just loops until it's out of memory. But it didn't report the usual "Allowed memory size of * bytes exhausted" message, probably because the methods are "magic".
I thought I would add my own experience as well.
I was getting the same error message, which in my case was caused by a PHP error in an exception.
The culprit was a custom exception class that did some logging internally, and a fatal error occurred in that logging mechanism. This caused the exception to not be triggered as expected, and no meaningful message to be displayed either.