fail2ban: apache-auth isn't banning failed login attempts - apache

Trying to implement fail2ban on a Linux Mint 17.1 and cant seem to get it to ban me after multiple login attempts against apache-auth.
I followed this guide to get started.
I think its a problem with the apache-auth regex but cant get it right.
jail.local
[apache]
enabled = true
port = 80,443
filter = apache-auth
logpath = /var/log/apache2/*error.log
maxretry = 3
bantime = 600
##ignoreip = 192.168.3.70
##To block the remote host that is trying to request suspicious URLs, use the below jail.
[apache-overflows]
enabled = true
port = http,https
filter = apache-overflows
logpath = /var/log/apache2/*error.log
maxretry = 3
bantime = 600
ignoreip = 192.168.3.70
##To block the remote host that is trying to search for scripts on the website to execute, use the below jail.
[apache-noscript]
enabled = true
port = http,https
filter = apache-noscript
logpath = /var/log/apache2/*error.log
maxretry = 3
bantime = 600
ignoreip = 192.168.3.70
##To block the remote host that is trying to request malicious bot, use below jail.
[apache-badbots]
enabled = true
port = http,https
filter = apache-badbots
logpath = /var/log/apache2/*error.log
maxretry = 3
bantime = 600
ignoreip = 192.168.3.70
##To stop DOS attack from remote host.
[http-get-dos]
enabled = true
port = http,https
filter = http-get-dos
logpath = /var/log/apache*/access.log
maxretry = 400
findtime = 400
bantime = 200
##ignoreip = 192.168.3.70
action = iptables[name=HTTP, port=http, protocol=tcp]
apache-auth.conf
[INCLUDES]
before = apache-common.conf
[Definition]
failregex = ^%(_apache_error_client)s (AH01797: )?client denied by server configuration: (uri )?\S*(, referer: \S+)?\s*$
^%(_apache_error_client)s (AH01617: )?user .*? authentication failure for "\S*": Password Mismatch(, referer: \S+)?$
^%(_apache_error_client)s (AH01618: )?user .*? not found(: )?\S*(, referer: \S+)?\s*$
^%(_apache_error_client)s (AH01614: )?client used wrong authentication scheme: \S*(, referer: \S+)?\s*$
^%(_apache_error_client)s (AH\d+: )?Authorization of user \S+ to access \S* failed, reason: .*$
^%(_apache_error_client)s (AH0179[24]: )?(Digest: )?user .*?: password mismatch: \S*(, referer: \S+)?\s*$
^%(_apache_error_client)s (AH0179[01]: |Digest: )user `.*?' in realm `.+' (not found|denied by provider): \S*(, referer: \S+)?\s*$
^%(_apache_error_client)s (AH01631: )?user .*?: authorization failure for "\S*":(, referer: \S+)?\s*$
^%(_apache_error_client)s (AH01775: )?(Digest: )?invalid nonce .* received - length is not \S+(, referer: \S+)?\s*$
^%(_apache_error_client)s (AH01788: )?(Digest: )?realm mismatch - got `.*?' but expected `.+'(, referer: \S+)?\s*$
^%(_apache_error_client)s (AH01789: )?(Digest: )?unknown algorithm `.*?' received: \S*(, referer: \S+)?\s*$
^%(_apache_error_client)s (AH01793: )?invalid qop `.*?' received: \S*(, referer: \S+)?\s*$
^%(_apache_error_client)s (AH01777: )?(Digest: )?invalid nonce .*? received - user attempted time travel(, referer: \S+)?\s*$
apache-common.conf
_apache_error_client = \[[^]]*\] \[(error|\S+:\S+)\]( \[pid \d+:\S+\d+\])? \[client <HOST>(:\d{1,5})?\]
/var/log/apache2/error.log
[Thu Aug 02 23:03:42.143209 2018] [auth_basic:error] [pid 8025] [client 192.168.3.60:54788] AH01618: user aaa not found: /SEC/test.php
[Thu Aug 02 23:03:43.216097 2018] [auth_basic:error] [pid 8025] [client 192.168.3.60:54788] AH01618: user aaa not found: /SEC/test.php
[Thu Aug 02 23:03:44.124570 2018] [auth_basic:error] [pid 8025][client 192.168.3.60:54788] AH01618: user aaa not found: /SEC/test.php
[Thu Aug 02 23:03:45.104747 2018] [auth_basic:error] [pid 8025][client 192.168.3.60:54788] AH01618: user aaa not found: /SEC/test.php
[Thu Aug 02 23:03:46.001161 2018] [auth_basic:error] [pid 8025][client 192.168.3.60:54788] AH01618: user aaa not found: /SEC/test.php
[Thu Aug 02 23:03:46.871802 2018] [auth_basic:error] [pid 8025][client 192.168.3.60:54788] AH01618: user aaa not found: /SEC/test.php
[Thu Aug 02 23:03:47.843740 2018] [auth_basic:error] [pid 8025][client 192.168.3.60:54788] AH01618: user aaa not found: /SEC/test.php
As I said I think the problem is in the regex, since the http-get-dos jail worked fine. The regex for apache-common.conf doesn't look right since my log files seem to follow a different standard, BUT I'm not sure if that's true or how to properly format the regex.
ANy help is appreciated.

Found my problem
I had to change the line in apache-common.conf to the following;
_apache_error_client = \[[^]]*\] \[(error|\S+:\S+)\]( \[pid \d+\])? \[client <HOST>(:\d{1,5})?\]
The difference being in the PID section. I used Regex101 to work the expression until it matched.

Related

setup varnish to listen on different port while apache listens on port 80

i have setup varnish 4 to run on port 8081 while apache is configured to run on port 80.
The problem with my setup is that when i browse my domain
http//:mydomain.com:8180/.
i get a (301) permanent redirect to http//:mydomain.com/.
Due to this redirect am unable to get the difference between calling the cached domain http//:mydomain.com:8180/ visa v the uncached domain http//:mydomain.com/.
my varnish config
DAEMON_OPTS="-a :8180\
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-S /etc/varnish/secret \
-s malloc,1G"
.......
also the vcl
backend mydomain {
.host = "x.x.x.x";
.port = "80";
.connect_timeout = 60s;
.first_byte_timeout = 60s;
.between_bytes_timeout = 60s;
.max_connections = 800;
}
.......
the responce header shows that apache is the one redirecting.
HTTP/1.1 301 Moved Permanently
Date: Fri, 04 Sep 2015 11:58:04 GMT
Server: Apache
X-Powered-By: PHP/5.3.3
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
X-Pingback: http//:mydomain.com/xmlrpc.php
Location: http//:mydomain.com/
Vary: Accept-Encoding
Content-Length: 0
Content-Type: text/html; charset=UTF-8
X-Varnish: 32795
Age: 0
Via: 1.1 varnish-v4
Connection: keep-alive
my question is How do i stop the redirect?
fixed this by adding
set req.http.host = "http//:mydomain.com";
in vlc as shown below.
if (req.http.host ~ "mydomain.com:8180") {
set req.http.host = "mydomain.com";
set req.backend_hint = mydomain;
}
by doing this we ensure that the request host is recognized by apache hence apache will not redirect

Https Proxy with Restlet Client

I am trying to get an https page over a proxy in restlet 2.1.7 with HTTPClient 4.1 and it is not authorizing with the proxy. If I connect to a page through http the authorization works. Connecting with httpurlconnect through the proxy to the same https site also works.
public class TestProxy {
public static void main(String[] args) {
Series<Parameter> parameters = null;
ClientResource clientResource = null;
Representation representation = null;
try {
Client client = new Client(new Context(), Protocol.HTTPS);
parameters = client.getContext().getParameters();
// proxy with credentials Works as excepted with http sites
parameters.add("proxyHost", PROXYIP);
parameters.add("proxyPort", PROXYPORT);
// create trust manager that trusts everything to eliminate
// certificates as an issue
TrustManager tm = new X509TrustManager() {
public void checkClientTrusted(X509Certificate[] chain,
String authType) throws CertificateException {
}
public X509Certificate[] getAcceptedIssuers() {
return new X509Certificate[0];
}
public void checkServerTrusted(X509Certificate[] chain,
String authType) throws CertificateException {
// This will never throw an exception.
// This doesn't check anything at all: it's insecure.
}
};
final SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(null, new TrustManager[] { tm }, null);
Context context = client.getContext();
context.getAttributes().put("sslContextFactory",
new SslContextFactory() {
public void init(Series<Parameter> parameters) {
}
public SSLContext createSslContext() {
return sslContext;
}
});
clientResource = new ClientResource("https://www.google.com");
clientResource.setProxyChallengeResponse(
ChallengeScheme.HTTP_BASIC, USER, PASS);
clientResource.setNext(client);
representation = clientResource.get();
System.out.println(representation.getText());
clientResource.get();
} catch (Exception e) {
e.printStackTrace();
}
}
}
It then throws a 407 proxy not authorized exception
Starting the Apache HTTP client
ThreadSafeClientConnManager.java:221) Get connection: HttpRoute[{tls}->http://PROXYIP:3128->https://www.google.com], timeout = 0
13 Jul 2015 10:18:11,747 DEBUG (ConnPoolByRoute.java:350) [HttpRoute[{tls}->http://PROXYIP:3128->https://www.google.com]] total kept alive: 0, total issued: 0, total allocated: 0 out of 20
13 Jul 2015 10:18:11,748 DEBUG (ConnPoolByRoute.java:523) No free connections [HttpRoute[{tls}->http://PROXYIP:3128->https://www.google.com]][null]
13 Jul 2015 10:18:11,749 DEBUG (ConnPoolByRoute.java:369) Available capacity: 10 out of 10 [HttpRoute[{tls}->http://PROXYIP:3128->https://www.google.com]][null]
13 Jul 2015 10:18:11,749 DEBUG (ConnPoolByRoute.java:549) Creating new connection [HttpRoute[{tls}->http://PROXYIP:3128->https://www.google.com]]
13 Jul 2015 10:18:11,769 DEBUG (DefaultClientConnectionOperator.java:145) Connecting to /PROXYIP:3128
13 Jul 2015 10:18:16,426 DEBUG (RequestAuthCache.java:75) Auth cache not set in the context
13 Jul 2015 10:18:16,426 DEBUG (DefaultClientConnection.java:264) Sending request: CONNECT www.google.com:443 HTTP/1.1
13 Jul 2015 10:18:16,428 DEBUG (Wire.java:63) >> "CONNECT www.google.com:443 HTTP/1.1[\r][\n]"
13 Jul 2015 10:18:16,430 DEBUG (Wire.java:63) >> "Host: www.google.com[\r][\n]"
13 Jul 2015 10:18:16,430 DEBUG (Wire.java:63) >> "Proxy-Connection: Keep-Alive[\r][\n]"
13 Jul 2015 10:18:16,431 DEBUG (Wire.java:63) >> "[\r][\n]"
13 Jul 2015 10:18:16,431 DEBUG (DefaultClientConnection.java:268) >> CONNECT www.google.com:443 HTTP/1.1
13 Jul 2015 10:18:16,432 DEBUG (DefaultClientConnection.java:271) >> Host: www.google.com
13 Jul 2015 10:18:16,432 DEBUG (DefaultClientConnection.java:271) >> Proxy-Connection: Keep-Alive
13 Jul 2015 10:18:16,455 DEBUG (Wire.java:63) << "HTTP/1.0 407 Proxy Authentication Required[\r][\n]"
13 Jul 2015 10:18:16,458 DEBUG (Wire.java:63) << "Server: squid/2.6.STABLE21[\r][\n]"
13 Jul 2015 10:18:16,459 DEBUG (Wire.java:63) << "Date: Mon, 13 Jul 2015 09:29:46 GMT[\r][\n]"
13 Jul 2015 10:18:16,459 DEBUG (Wire.java:63) << "Content-Type: text/html[\r][\n]"
13 Jul 2015 10:18:16,459 DEBUG (Wire.java:63) << "Content-Length: 1298[\r][\n]"
13 Jul 2015 10:18:16,460 DEBUG (Wire.java:63) << "Expires: Mon, 13 Jul 2015 09:29:46 GMT[\r][\n]"
13 Jul 2015 10:18:16,460 DEBUG (Wire.java:63) << "X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0[\r][\n]"
13 Jul 2015 10:18:16,461 DEBUG (Wire.java:63) << "Proxy-Authenticate: Basic realm="Squid proxy-caching web server"[\r][\n]"
13 Jul 2015 10:18:16,461 DEBUG (Wire.java:63) << "X-Cache: MISS from hostname.domain.tld[\r][\n]"
13 Jul 2015 10:18:16,461 DEBUG (Wire.java:63) << "X-Cache-Lookup: NONE from hostname.domain.tld:3128[\r][\n]"
13 Jul 2015 10:18:16,462 DEBUG (Wire.java:63) << "Via: 1.0 hostname.domain.tld:3128 (squid/2.6.STABLE21)[\r][\n]"
13 Jul 2015 10:18:16,462 DEBUG (Wire.java:63) << "Proxy-Connection: close[\r][\n]"
13 Jul 2015 10:18:16,462 DEBUG (Wire.java:63) << "[\r][\n]"
13 Jul 2015 10:18:16,464 DEBUG (DefaultClientConnection.java:249) Receiving response: HTTP/1.0 407 Proxy Authentication Required
13 Jul 2015 10:18:16,464 DEBUG (DefaultClientConnection.java:252) << HTTP/1.0 407 Proxy Authentication Required
13 Jul 2015 10:18:16,464 DEBUG (DefaultClientConnection.java:255) << Server: squid/2.6.STABLE21
13 Jul 2015 10:18:16,465 DEBUG (DefaultClientConnection.java:255) << Date: Mon, 13 Jul 2015 09:29:46 GMT
13 Jul 2015 10:18:16,465 DEBUG (DefaultClientConnection.java:255) << Content-Type: text/html
13 Jul 2015 10:18:16,465 DEBUG (DefaultClientConnection.java:255) << Content-Length: 1298
13 Jul 2015 10:18:16,466 DEBUG (DefaultClientConnection.java:255) << Expires: Mon, 13 Jul 2015 09:29:46 GMT
13 Jul 2015 10:18:16,466 DEBUG (DefaultClientConnection.java:255) << X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
13 Jul 2015 10:18:16,466 DEBUG (DefaultClientConnection.java:255) << Proxy-Authenticate: Basic realm="Squid proxy-caching web server"
13 Jul 2015 10:18:16,467 DEBUG (DefaultClientConnection.java:255) << X-Cache: MISS from hostname.domain.tld
13 Jul 2015 10:18:16,467 DEBUG (DefaultClientConnection.java:255) << X-Cache-Lookup: NONE from hostname.domain.tld:3128
13 Jul 2015 10:18:16,467 DEBUG (DefaultClientConnection.java:255) << Via: 1.0 hostname.domain.tld:3128 (squid/2.6.STABLE21)
13 Jul 2015 10:18:16,468 DEBUG (DefaultClientConnection.java:255) << Proxy-Connection: close
13 Jul 2015 10:18:16,470 DEBUG (ResponseProcessCookies.java:78) Cookie spec not specified in HTTP context
13 Jul 2015 10:18:16,473 DEBUG (Wire.java:63) << "<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">[\n]"
13 Jul 2015 10:18:16,474 DEBUG (Wire.java:63) << "<HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">[\n]"
13 Jul 2015 10:18:16,474 DEBUG (Wire.java:63) << "<TITLE>ERROR: Cache Access Denied</TITLE>[\n]"
13 Jul 2015 10:18:16,475 DEBUG (Wire.java:63) << "<STYLE type="text/css"><!--BODY{background-color:#ffffff;font-family:verdana,sans-serif}PRE{font-family:sans-serif}--></STYLE>[\n]"
13 Jul 2015 10:18:16,475 DEBUG (Wire.java:63) << "</HEAD>[\n]"
13 Jul 2015 10:18:16,476 DEBUG (Wire.java:63) << "<BODY>[\n]"
13 Jul 2015 10:18:16,476 DEBUG (Wire.java:63) << "<H1>ERROR</H1>[\n]"
13 Jul 2015 10:18:16,477 DEBUG (Wire.java:63) << "<H2>Cache Access Denied</H2>[\n]"
13 Jul 2015 10:18:16,477 DEBUG (Wire.java:63) << "<HR noshade size="1px">[\n]"
13 Jul 2015 10:18:16,478 DEBUG (Wire.java:63) << "<P>[\n]"
13 Jul 2015 10:18:16,478 DEBUG (Wire.java:63) << "While trying to retrieve the URL:[\n]"
13 Jul 2015 10:18:16,479 DEBUG (Wire.java:63) << "www.google.com:443[\n]"
13 Jul 2015 10:18:16,479 DEBUG (Wire.java:63) << "<P>[\n]"
13 Jul 2015 10:18:16,480 DEBUG (Wire.java:63) << "The following error was encountered:[\n]"
13 Jul 2015 10:18:16,480 DEBUG (Wire.java:63) << "<UL>[\n]"
13 Jul 2015 10:18:16,480 DEBUG (Wire.java:63) << "<LI>[\n]"
13 Jul 2015 10:18:16,481 DEBUG (Wire.java:63) << "<STRONG>[\n]"
13 Jul 2015 10:18:16,481 DEBUG (Wire.java:63) << "Cache Access Denied.[\n]"
13 Jul 2015 10:18:16,482 DEBUG (Wire.java:63) << "</STRONG>[\n]"
13 Jul 2015 10:18:16,482 DEBUG (Wire.java:63) << "</UL>[\n]"
13 Jul 2015 10:18:16,483 DEBUG (Wire.java:63) << "</P>[\n]"
13 Jul 2015 10:18:16,483 DEBUG (Wire.java:63) << "[\n]"
13 Jul 2015 10:18:16,483 DEBUG (Wire.java:63) << "<P>Sorry, you are not currently allowed to request:[\n]"
13 Jul 2015 10:18:16,484 DEBUG (Wire.java:63) << "<PRE> www.google.com:443</PRE>[\n]"
13 Jul 2015 10:18:16,484 DEBUG (Wire.java:63) << "from this cache until you have authenticated yourself.[\n]"
13 Jul 2015 10:18:16,485 DEBUG (Wire.java:63) << "</P>[\n]"
13 Jul 2015 10:18:16,485 DEBUG (Wire.java:63) << "[\n]"
13 Jul 2015 10:18:16,486 DEBUG (Wire.java:63) << "<P>[\n]"
13 Jul 2015 10:18:16,486 DEBUG (Wire.java:63) << "You need to use Netscape version 2.0 or greater, or Microsoft Internet[\n]"
13 Jul 2015 10:18:16,487 DEBUG (Wire.java:63) << "Explorer 3.0, or an HTTP/1.1 compliant browser for this to work. Please[\n]"
13 Jul 2015 10:18:16,487 DEBUG (Wire.java:63) << "contact the cache administrator if you have[\n]"
13 Jul 2015 10:18:16,487 DEBUG (Wire.java:63) << "difficulties authenticating yourself or [\n]"
13 Jul 2015 10:18:16,488 DEBUG (Wire.java:63) << "change your default password.[\n]"
13 Jul 2015 10:18:16,488 DEBUG (Wire.java:63) << "</P>[\n]"
13 Jul 2015 10:18:16,488 DEBUG (Wire.java:63) << "[\n]"
13 Jul 2015 10:18:16,489 DEBUG (Wire.java:63) << "<BR clear="all">[\n]"
13 Jul 2015 10:18:16,489 DEBUG (Wire.java:63) << "<HR noshade size="1px">[\n]"
13 Jul 2015 10:18:16,489 DEBUG (Wire.java:63) << "<ADDRESS>[\n]"
13 Jul 2015 10:18:16,490 DEBUG (Wire.java:63) << "Generated Mon, 13 Jul 2015 09:29:46 GMT by hostname.domain.tld (squid/2.6.STABLE21)[\n]"
13 Jul 2015 10:18:16,490 DEBUG (Wire.java:63) << "</ADDRESS>[\n]"
13 Jul 2015 10:18:16,490 DEBUG (Wire.java:63) << "</BODY></HTML>[\n]"
13 Jul 2015 10:18:16,491 DEBUG (DefaultClientConnection.java:165) Connection closed
13 Jul 2015 10:18:16,491 DEBUG (DefaultRequestDirector.java:418) CONNECT refused by proxy: HTTP/1.0 407 Proxy Authentication Required
13 Jul 2015 10:18:16,492 DEBUG (ThreadSafeClientConnManager.java:272) Released connection is not reusable.
13 Jul 2015 10:18:16,492 DEBUG (ConnPoolByRoute.java:434) Releasing connection [HttpRoute[{tls}->http://PROXYIP:3128->https://www.google.com]][null]
13 Jul 2015 10:18:16,493 DEBUG (ConnPoolByRoute.java:679) Notifying no-one, there are no waiting threads
Proxy Authentication Required (407) - Proxy Authentication Required
at org.restlet.resource.ClientResource.doError(ClientResource.java:612)
at org.restlet.resource.ClientResource.handleInbound(ClientResource.java:1202)
at org.restlet.resource.ClientResource.handle(ClientResource.java:1069)
at org.restlet.resource.ClientResource.handle(ClientResource.java:1044)
at org.restlet.resource.ClientResource.handle(ClientResource.java:950)
at org.restlet.resource.ClientResource.get(ClientResource.java:658)
at TestProxy.main(TestProxy.java:68)

Error wsgi in server installed in MAC Yosemite

I have problems wsgi module and do not understand why.
I installed Apache, remove MAMP because it gave me too many problems.
I have configured the port and my page loads fine.
Install MYSQL load my script and all is well.
Install the Python-MySQL connector and make the connection and actually connects.
but when I access the site and I want to register strip mistake, nose if it reaches the database or not.
Someone help me understand what happens.
Attached codes.
httpd.conf
ServerRoot "/usr/local/apache2"
Listen 8080
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule authn_core_module modules/mod_authn_core.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule access_compat_module modules/mod_access_compat.so
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule socache_shmcb_module modules/mod_socache_shmcb.so
LoadModule reqtimeout_module modules/mod_reqtimeout.so
LoadModule filter_module modules/mod_filter.so
LoadModule mime_module modules/mod_mime.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule env_module modules/mod_env.so
LoadModule headers_module modules/mod_headers.so
#LoadModule unique_id_module modules/mod_unique_id.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule version_module modules/mod_version.so
LoadModule ssl_module modules/mod_ssl.so
LoadModule unixd_module modules/mod_unixd.so
#LoadModule dav_module modules/mod_dav.so
LoadModule status_module modules/mod_status.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule alias_module modules/mod_alias.so
LoadModule wsgi_module modules/mod_wsgi.so
<IfModule unixd_module>
User daemon
Group daemon
</IfModule>
ServerAdmin usuario#usuario.com
ServerName localhost:8080
<Directory />
AllowOverride none
Require all denied
</Directory>
DocumentRoot "/Users/usuario/Sites/usuariocloud/client"
<Directory "/Users/usuario/Sites/usuariocloud/client">
Options Indexes FollowSymLinks
AllowOverride None
Require all granted
</Directory>
<IfModule dir_module>
DirectoryIndex index.html
</IfModule>
<Files ".ht*">
Require all denied
</Files>
ErrorLog "logs/error_log"
LogLevel warn
<IfModule log_config_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
<IfModule logio_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
</IfModule>
CustomLog "logs/access_log" common
</IfModule>
<IfModule alias_module>
ScriptAlias /cgi-bin/ "/Users/usuario/Sites/usuariocloud/server/"
</IfModule>
<IfModule cgid_module>
</IfModule>
Alias /cgi-bin /Users/usuario/Sites/usuariocloud/server/
<Location /cgi-bin>
SetHandler wsgi-script
Options +ExecCGI
</Location>
#WSGIScriptAlias /cgi-bin /Users/usuario/Sites/usuariocloud/server/
<Directory "/Users/usuario/Sites/usuariocloud/server/">
AllowOverride None
Options None
Require all granted
</Directory>
<IfModule mime_module>
TypesConfig conf/mime.types
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
</IfModule>
<IfModule proxy_html_module>
Include conf/extra/proxy-html.conf
</IfModule>
Include conf/extra/httpd-ssl.conf
<IfModule ssl_module>
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
</IfModule>
controller.wsgi
# Library Imports
import json, cgi, sys, os
path = os.path.dirname(__file__)
if path not in sys.path:
sys.path.append(path)
# Own Libraries
from petition_solver.solver import Solver
def application(env, resp):
response = { "response": "fail",
"error" : """Expecting ?Json=
{
"function":"functionName",
"entity":"entityName",
"params":
{
"param1":"value1",
"param2":"value2",
"paramn":"value n"
}
}""".replace('\r', ' ').replace('\n', ' ').replace('\t', '')
}
# Read Params
form = cgi.FieldStorage(fp=env['wsgi.input'], environ=env)
if form:
userAgent = env["HTTP_USER_AGENT"] if "HTTP_USER_AGENT" in env else ""
param_json = form['json'].value
petition = json.loads(param_json)
if('file' in form and "params" in petition):
param_file = form['file']
if(isinstance(param_file, list)):
petition['params']['files'] = []
for f in param_file:
filename = unicode(f.filename, 'utf-8')
petition['params']['files'].append({"originalName" : filename, "file" : f.file.read()})
else:
petition['params']['file'] = param_file.file.read()
filename = unicode(param_file.filename, 'utf-8')
petition['params']['originalName'] = filename
solver = Solver()
response = solver.solvePetition(petition, userAgent=userAgent)
if (response and "download" in response and response["download"]):
detail = response["file"]["storage"]
mime = detail["mime"].decode('utf-8')
name = detail["originalName"].encode("ascii", 'replace')
data = detail["file"]
resp('200 OK',[('Content-type', '{0}'.format(mime)),('Content-disposition', 'attachment; filename*=UTF-8''{0}; filename={0}'.format(name))])
yield data
else:
resp('200 OK',[('Content-type', 'text/html;charset=utf-8')])
yield json.dumps(response)
solver.py
#solver module
#class Solver
from log import Log
from error import NotValidPetitionError
class Solver:
userId = -1
def solvePetition(self, petition, petitionId=0, userAgent=None):
lg = Log.Instance()
if("function" not in petition or "entity" not in petition):
response = {"response":"fail", "error" : "{0}".format(NotValidPetitionError())}
lg.logError("Not a valid petition found", lg.SOLVER_LEVEL)
return response
innerPetition = self._getCopy(petition)
function = innerPetition["function"]
entityName = innerPetition["entity"]
params = innerPetition["params"] if "params" in innerPetition else {}
key = innerPetition["key"] if "key" in innerPetition else ""
#petitionId = petitionId if petitionId else self._getPetitionId()
#lg.logDebug("Received petition {0} ".format(self._getJsonRepr(petition)), lg.SOLVER_LEVEL, petitionId)
entity = None
entityType = None
if(entityName == "user"):
from entities.user import User
entityType = User
params["userAgent"] = userAgent
elif(entityName == "group"):
from entities.group import Group
entityType = Group
elif(entityName == "tag"):
from entities.tag import Tag
entityType = Tag
elif(entityName == "attribute"):
from entities.attribute import Attribute
entityType = Attribute
elif(entityName == "template"):
from entities.template import Template
entityType = Template
elif(entityName == "directory"):
from entities.directory import Directory
entityType = Directory
elif(entityName == "staticCatalog"):
from entities.staticCatalog import StaticCatalog
entityType = StaticCatalog
elif(entityName == "dynamicCatalog"):
from entities.dynamicCatalog import DynamicCatalog
entityType = DynamicCatalog
elif(entityName == "document"):
from entities.document import Document
entityType = Document
elif(entityName == "file"):
from entities.file import File
entityType = File
elif(entityName == "inbox"):
from entities.inbox import Inbox
entityType = Inbox
elif(entityName == "storageFile"):
from entities.storageFile import StorageFile
entityType = StorageFile
elif(entityName == "structure"):
from entities.structure import Structure
entityType = Structure
elif(entityName == "dictionaryCatalog"):
from entities.dictionaryCatalog import DictionaryCatalog
entityType = DictionaryCatalog
elif(entityName == "patternTemplate"):
from entities.patternTemplate import PatternTemplate
entityType = PatternTemplate
if petitionId:
petitionId = petitionId
valueReturned = self._operationsManager(params, petitionId, key, entityType, entityName, function, False, petition)
else:
petitionId = self._getPetitionId()
valueReturned = self._operationsManager(params, petitionId, key, entityType, entityName, function, True, petition)
lg.logDebug("Received petition {0} ".format(self._getJsonRepr(petition)), lg.SOLVER_LEVEL, petitionId)
try:
entity = entityType(params, key, petitionId)
response = entity.process(function)
pet_id = self._getPetition(entity, petitionId)
queryUpdate = self._getDBTemplate("UPDATE_OPERATION_STATUS").format(pet_id)
newId = entity._resolveUpdate(queryUpdate, audit=False)
if newId > 0:
lg.logDebug("UPDATE_PETITION_ID: {0} ".format(pet_id), lg.SOLVER_LEVEL, petitionId)
except Exception as e:
response = self._manageError(e, petition, petitionId)
finally:
del (entity)
del (innerPetition)
lg.logDebug("Response to petition is {0}".format(self._getJsonRepr(response)), lg.SOLVER_LEVEL, petitionId)
response["petition"] = self._getJsonWOFile(petition)
return response
def _getJsonRepr(self, json):
j = self._getJsonWOFile(json["file"]) if "file" in json else self._getJsonWOFile(json)
return "{0}".format(j)
def _getJsonWOFile(self, json):
needsCopy = json and "file" in json or (
"params" in json and json["params"] and (
"file" in json["params"] or "files" in json["params"]
)
) or "storage" in json
if needsCopy:
copy = self._getCopy(json)
if ("file" in copy):
copy["file"] = "FILE DATA OMITTED" if copy["file"] else "EMPTY FILE"
if ("storage" in copy):
if ("file" in copy["storage"]):
copy["storage"]["file"] = "FILE DATA OMITTED" if copy["storage"]["file"] else "EMPTY FILE"
if("params" in copy):
if("files" in copy["params"]):
for f in copy['params']["files"]:
f["file"] = "FILE DATA OMITTED" if f["file"] else "EMPTY FILE"
if("file" in copy["params"]):
copy["params"]["file"] = "FILE DATA OMITTED" if copy["params"]["file"] else "EMPTY FILE"
return copy
else:
return json
def _getCopy(self, json):
import copy
copy = copy.deepcopy(json)
return copy
def _manageError(self, err, petition, petitionId):
from error import usuarioError
innerError = err if isinstance(err, usuarioError) else usuarioError()
lg = Log.Instance()
lgMethod = lg.logWarning if innerError.code < 400 else lg.logError
lgMethod("{0} found while resolving petition {1}".format( str(innerError) , petitionId), lg.SOLVER_LEVEL, petitionId)
response = {
"response":"fail",
"error" : {
"code" : "{0}".format(innerError.code),
"message" : str(innerError)
}
}
return response
def _getPetitionId(self):
import uuid
uuidObj = uuid.uuid4()
return uuidObj.hex
def _getDBTemplate(self, templateName):
dbTemplateProvider = None
if not dbTemplateProvider:
from db_template_provider import DBTemplateProvider
dbTemplateProvider = DBTemplateProvider.Instance()
return dbTemplateProvider.getDBTemplate(templateName)
def _findFunction(self, functionName, entityType):
queryFunction = self._getDBTemplate("FIND_FUNCTION_ID").format(functionName)
rows = entityType._resolveQuery(queryFunction, function=functionName, audit=False)
if rows:
functionId = rows[0]
fcId = functionId[0]
return fcId
return 0
def _findEntity(self, entityName, entityType):
queryEntity = self._getDBTemplate("FIND_ENTITY_ID").format(entityName)
rows = entityType._resolveQuery(queryEntity, audit=False)
if rows:
entityId = rows[0]
entId = entityId[0]
return entId
return 0
def _addOperation(self, function, entityName, entity, newId, typeOper, petitionId):
lg = Log.Instance()
functionId = self._findFunction(function, entity)
entityId = self._findEntity(entityName, entity)
queryOperation = ""
if typeOper:
queryOperation = self._getDBTemplate("CREATE_OPERATIONS").format(newId, functionId, entityId, 0, 2)
else:
queryOperation = self._getDBTemplate("CREATE_OPERATIONS").format(newId, functionId, entityId, 0, 1)
entity._resolveUpdate(queryOperation, False)
lg.logDebug("Operation Added: {0}".format(newId), lg.SOLVER_LEVEL, petitionId)
def _getPetition(self, entityType, petitionId):
queryPetition = self._getDBTemplate("FIND_PETITION_ID").format(petitionId)
required = []
rows = entityType._resolveQuery(queryPetition, audit=False)
if rows:
petId = rows[0]
petId_ = petId[0]
return petId_
return 0
def _operationsManager(self, params, petitionId, key, entityType, entityName, function, typeOper, petition):
entity = None
newId = 0
lg = Log.Instance()
try:
entity = entityType(params, key, petitionId)
if typeOper:
jsonStr = self._getJsonRepr(petition).replace("\'", "\\\'")
userName = self._findUserName(entity, key)
if self.userId != -1:
queryRegistry = self._getDBTemplate("CREATE_REGISTRY_PETITIONS").format(petitionId, jsonStr, "final", 0, self.userId, userName, 5)
lg.logDebug("REGISTRY QUERY: {0}".format(jsonStr), lg.SOLVER_LEVEL, petitionId)
newId = entity._resolveUpdate(queryRegistry, audit=False)
if newId > 0:
lg.logDebug("Petition Added: {0}".format(jsonStr), lg.SOLVER_LEVEL, petitionId)
self._addOperation(function, entityName, entity, newId, True, petitionId)
else:
return False
return True
else:
return False
else:
newId = self._getPetition(entity, petitionId)
self._addOperation(function, entityName, entity, newId, False, petitionId)
except Exception as e:
lg.logError(self._manageError(e, petition, petitionId), lg.SOLVER_LEVEL, petitionId)
return False
def _findUserName(self, entity, key):
userTemplate = self._getDBTemplate("QUERY_USER_BY_KEY").format(key)
rowsId = entity._resolveQuery(userTemplate)
if rowsId:
self.userId = rowsId[0][0]
nameTemplate = self._getDBTemplate("QUERY_USERNAME").format(self.userId)
rowsUsr = entity._resolveQuery(nameTemplate)
if rowsUsr:
userName = rowsUsr[0][0]
return userName
return None
and the error is
and the logs are the nexts
[Thu Jun 18 12:04:37.413641 2015] [wsgi:error] [pid 2048:tid 4367495168] [client ::1:49302] mod_wsgi (pid=2048): Exception occurred processing WSGI script '/Users/usuario/Sites/usuariocloud/server/controller.wsgi'., referer: http://localhost:8080/\
[Thu Jun 18 12:04:37.413692 2015] [wsgi:error] [pid 2048:tid 4367495168] [client ::1:49302] Traceback (most recent call last):, referer: http://localhost:8080/\
[Thu Jun 18 12:04:37.413719 2015] [wsgi:error] [pid 2048:tid 4367495168] [client ::1:49302] File "/Users/usuario/Sites/usuariocloud/server/controller.wsgi", line 53, in application, referer: http://localhost:8080/\
[Thu Jun 18 12:04:37.413759 2015] [wsgi:error] [pid 2048:tid 4367495168] [client ::1:49302] response = solver.solvePetition(petition, userAgent=userAgent), referer: http://localhost:8080/\
[Thu Jun 18 12:04:37.413775 2015] [wsgi:error] [pid 2048:tid 4367495168] [client ::1:49302] File "/Users/usuario/Sites/usuariocloud/server/petition_solver/solver.py", line 13, in solvePetition, referer: http://localhost:8080/\
[Thu Jun 18 12:04:37.413795 2015] [wsgi:error] [pid 2048:tid 4367495168] [client ::1:49302] lg = Log.Instance(), referer: http://localhost:8080/\
[Thu Jun 18 12:04:37.413805 2015] [wsgi:error] [pid 2048:tid 4367495168] [client ::1:49302] File "/Users/usuario/Sites/usuariocloud/server/petition_solver/singleton.py", line 34, in Instance, referer: http://localhost:8080/\
[Thu Jun 18 12:04:37.413823 2015] [wsgi:error] [pid 2048:tid 4367495168] [client ::1:49302] self._instance = self._decorated(), referer: http://localhost:8080/\
[Thu Jun 18 12:04:37.413833 2015] [wsgi:error] [pid 2048:tid 4367495168] [client ::1:49302] File "/Users/usuario/Sites/usuariocloud/server/petition_solver/log.py", line 24, in __init__, referer: http://localhost:8080/\
[Thu Jun 18 12:04:37.413849 2015] [wsgi:error] [pid 2048:tid 4367495168] [client ::1:49302] os.makedirs(directory), referer: http://localhost:8080/\
[Thu Jun 18 12:04:37.413859 2015] [wsgi:error] [pid 2048:tid 4367495168] [client ::1:49302] File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 157, in makedirs, referer: http://localhost:8080/\
[Thu Jun 18 12:04:37.413878 2015] [wsgi:error] [pid 2048:tid 4367495168] [client ::1:49302] mkdir(name, mode), referer: http://localhost:8080/\
[Thu Jun 18 12:04:37.413897 2015] [wsgi:error] [pid 2048:tid 4367495168] [client ::1:49302] OSError: [Errno 13] Permission denied: '//logs', referer: http://localhost:8080/\
[Thu Jun 18 12:54:13.192665 2015] [mpm_worker:notice] [pid 2046:tid 140735125234432] AH00295: caught SIGTERM, shutting down\
Your code is trying to write to a log file using a relative path. You cannot do that as the current working directory of the process will be '/' and not where your code is. See:
http://code.google.com/p/modwsgi/wiki/ApplicationIssues#Application_Working_Directory
Use an absolute path explicitly, or calculate it relative to the code location by using os.path.dirname(__file__) as a base.

Using custom functions in Volt force apache process to die

I worked with Phalcon and Volt under WAMP. Recently we moved to another dev environment (CentOS) and there I have PHP 5.5.17 with latest Phalcon build (I compiled and tested 2 versions lower as well).
Now, when Volt tries to compile template with custom function, it crashes (PHP process). The same is
about custom filters.
Error log of Apache
[Tue Sep 30 06:06:24.809476 2014] [proxy_fcgi:error] [pid 31199:tid 140596014397184] (104)Connection reset by peer: [client 10.0.2.2:53931] AH01075: Error dispatching request to :3080:
[Tue Sep 30 06:06:27.216226 2014] [proxy_fcgi:error] [pid 31200:tid 140596161255168] [client 10.0.2.2:53941] AH01067: Failed to read FastCGI header
[Tue Sep 30 06:06:27.216249 2014] [proxy_fcgi:error] [pid 31200:tid 140596161255168] (104)Connection reset by peer: [client 10.0.2.2:53941] AH01075: Error dispatching request to :3080:
PHP error log
[30-Sep-2014 06:06:27] WARNING: [pool www] child 32519 exited on signal 11 (SIGSEGV - core dumped) after 204.725812 seconds from start
[30-Sep-2014 06:06:27] NOTICE: [pool www] child 32529 started
PHP code looks like
$di->set('view', function () use ($config) {
$view = new View();
$view->setViewsDir($config->application->viewsDir);
$view->registerEngines(array(
'.volt' => function ($view, $di) use ($config) {
$volt = new VoltEngine($view, $di);
$volt->setOptions(array(
'compiledPath' => $config->application->cacheDir,
'compiledSeparator' => '_',
'compileAlways' => $config->application->debug
));
$compiler = $volt->getCompiler();
$compiler->addFunction(
'last',
function ($resolvedArgs) use ($compiler) {
return 'array_pop('. $resolvedArgs .')';
}
);
return $volt;
}
));
return $view;
}, true);
And in Volt for example
{{ last(['1', '2', '3']) }}
And I really stuck on this problem, because I have pretty a lot of custom functions and I do need them. Tried to debug it, but, as soon as volt tried to parse line with custom function, process die.
Phalcon bug submitted. Solution: totally disable xdebug for current build. More here: https://github.com/xdebug/xdebug/pull/120

Several error_log() messages on same line in error log

I have this PHP code :
error_log('my message 1');
....
error_log('my message 2');
...
error_log('my message 3');
This produces in apache error_log one line with all messages :
[Wed Nov 13 17:24:55.880399 2013] [proxy_fcgi:error] [pid xx] [client xxx] AH01071: Got error 'PHP message: my message 1\n'PHP message: my message 2\n'PHP message: my message 3
My config :
Apache 2.4
PHP : 5.4
PHP-FPM with proxypassmatch directive.
My question : Why messages are on the same line, and how to do to have one line per message ?
Thanks for yours answers.
EDIT
One line per message should look like :
[Wed Nov 13 17:24:55.880399 2013] [proxy_fcgi:error] [pid xx] [client xxx] AH01071: Got error 'PHP message: my message 1'
[Wed Nov 13 17:24:55.880399 2013] [proxy_fcgi:error] [pid xx] [client xxx] AH01071: Got error 'PHP message: my message 2'
[Wed Nov 13 17:24:55.880399 2013] [proxy_fcgi:error] [pid xx] [client xxx] AH01071: Got error 'PHP message: my message 3'
error_log("error message \r\n");
PHP disregards special ASCII characters within single quotes(it renders it as separate chars), you need to use double quotes.
In addition:
You should open your php.ini file, the one in the /etc/php5/apache2/ folder, and chnage the error_log directive to point to a file.
It is important that Apache will have sufficient privileges to write into this file.
so
chown www-data:www-data /var/www/somefile.log
should do it
If it's currently undefined, the logs will go through syslog, and there new lines are not allowed.
Additional edit:
To penetrate output buffering you need to raise an exception.
example:
try{
ob_start();
doSomething($userInput);
ob_end_flush();
}
catch(Exception $e){
error_log($e->getMessage());
}
function doSomething($data = null){
if($data === null){
throw new Exception("Data is required");
}
else{
//do something
}
}
User \r\n
error_log("my message 1\r\n");
....
error_log("my message 2\r\n");
...
error_log("my message 3\r\n");