Laravel Echo + Pusher Authorization Server Error 500 - laravel-echo

I'm trying to test and learn laravel's broadcasting with echo. But after trying and trying I can't achieve what I want.
It works with public channel channel.
But when it comes to presence channels it doesnt.
For solving error I did:
Increasing php memory limit to 1 GB
I'm using Jetstream and Fortify. Also InertiaJS.
Browser debug authorization headers
authorization headers
Got error
error: "Unable to retrieve auth string from auth endpoint - received status: 500 from /broadcasting/auth. Clients must be authenticated to join private or presence channels. See: https://pusher.com/docs/authenticating_users"
status: 500
App.js
window.Pusher = require('pusher-js');
window.Echo = new Echo({
broadcaster: 'pusher',
key: process.env.MIX_PUSHER_APP_KEY,
cluster: process.env.MIX_PUSHER_APP_CLUSTER,
forceTLS: true,
});
window.Echo.join(`chat.1`)
.here((users) => {
console.log(users);
})
.joining((user) => {
console.log(user.name);
})
.leaving((user) => {
console.log(user.name);
})
.error((error) => {
console.error(error);
});
BroadcastServiceProvider.php
public function boot()
{
Broadcast::routes();
require base_path('routes/channels.php');
}
channels.php
Broadcast::channel('chat.{ida}', function ($user, $ida) {
if (auth()->check()) {
return $user->toArray();
}
});
site.net.error.log
[Sun May 16 10:24:49.754522 2021] [fcgid:warn] [pid 14900] [client ip:38966] mod_fcgid: stderr: PHP Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 262144 bytes) in Unknown on line 0, referer: https://abcabc.net/dashboard
[Sun May 16 10:26:31.944725 2021] [fcgid:warn] [pid 4715] [client ip:39000] mod_fcgid: stderr: PHP Fatal error: Out of memory (allocated 861929472) (tried to allocate 262144 bytes) in *sitepath*/private/app_data/vendor/pusher/pusher-php-server/src/Pusher.php on line 840, referer: https://abcabc.net/dashboard
[Sun May 16 10:26:31.954042 2021] [fcgid:warn] [pid 4715] [client ip:39000] mod_fcgid: stderr: PHP Fatal error: Out of memory (allocated 861929472) (tried to allocate 262144 bytes) in Unknown on line 0, referer: https://abcabc.net/dashboard
[Sun May 16 10:27:12.115963 2021] [fcgid:warn] [pid 19725] [client ip:39026] mod_fcgid: stderr: PHP Fatal error: Out of memory (allocated 394264576) (tried to allocate 262144 bytes) in *sitepath*/private/app_data/vendor/pusher/pusher-php-server/src/Pusher.php on line 840, referer: https://abcabc.net/dashboard
[Sun May 16 10:27:12.120752 2021] [fcgid:warn] [pid 19725] [client ip:39026] mod_fcgid: stderr: PHP Fatal error: Out of memory (allocated 394264576) (tried to allocate 262144 bytes) in Unknown on line 0, referer: https://abcabc.net/dashboard
[Sun May 16 10:30:19.624249 2021] [fcgid:warn] [pid 14898] [client ip:39066] mod_fcgid: stderr: PHP Fatal error: Out of memory (allocated 710934528) (tried to allocate 262144 bytes) in *sitepath*/private/app_data/vendor/pusher/pusher-php-server/src/Pusher.php on line 840, referer: https://abcabc.net/dashboard
[Sun May 16 10:30:19.636461 2021] [fcgid:warn] [pid 14898] [client ip:39066] mod_fcgid: stderr: PHP Fatal error: Out of memory (allocated 710934528) (tried to allocate 262144 bytes) in Unknown on line 0, referer: https://abcabc.net/dashboard
[Sun May 16 10:30:38.961347 2021] [fcgid:warn] [pid 14898] [client ip:39086] mod_fcgid: stderr: PHP Fatal error: Out of memory (allocated 551550976) (tried to allocate 262144 bytes) in *sitepath*/private/app_data/vendor/pusher/pusher-php-server/src/Pusher.php on line 840, referer: https://abcabc.net/dashboard
[Sun May 16 10:30:38.963699 2021] [fcgid:warn] [pid 14898] [client ip:39086] mod_fcgid: stderr: PHP Fatal error: Out of memory (allocated 551550976) (tried to allocate 262144 bytes) in Unknown on line 0, referer: https://abcabc.net/dashboard
[Sun May 16 10:31:03.040755 2021] [fcgid:warn] [pid 14897] [client ip:39104] mod_fcgid: stderr: PHP Fatal error: Out of memory (allocated 276824064) (tried to allocate 262144 bytes) in *sitepath*/private/app_data/vendor/pusher/pusher-php-server/src/Pusher.php on line 840, referer: https://abcabc.net/dashboard
[Sun May 16 10:31:03.049553 2021] [fcgid:warn] [pid 14897] [client ip:39104] mod_fcgid: stderr: PHP Fatal error: Out of memory (allocated 276824064) (tried to allocate 262144 bytes) in Unknown on line 0, referer: https://abcabc.net/dashboard

For those who have this problem look to laravel event classes. Misconfiguration.

I was having a similar problem and I resolved it by creating an authentication route and controller. Have a look at my solution in this post: What can I do to resolve this pusher error-JSON returned from auth endpoint was invalid, yet status code was 200?.

Related

Can't consume message that is enqeued by test

I am having an issue writing a basic test that publishes a message to a point-point queue.
When using an #JmsListener bean, the message is consumed.
When not using an #JmsListener and using a consumer obtained from the connectionFactory via the #Autowired JmsTemplate in the test class the message is not consumed.
I have added some logging and debug output and can not see why I can not consume the message inside the test class but an #JmsListener bean does.
#SpringBootTest
#ActiveProfiles("tc")
#Log4j2
public class SessionActiveMQIT {
#Autowired
public JmsTemplate jmsTemplate;
#Test
void canEnqueueAndPersistClientAck() throws JMSException, InterruptedException {
final ActiveMQQueue activeMQQueue = new ActiveMQQueue("TEST_QUEUE");
jmsTemplate.setDeliveryPersistent(true);
jmsTemplate.setSessionAcknowledgeMode(JmsProperties.AcknowledgeMode.CLIENT.getMode());
jmsTemplate.setSessionTransacted(true);
jmsTemplate.setDefaultDestination(activeMQQueue);
jmsTemplate.setPubSubDomain(false);
jmsTemplate.setPubSubNoLocal(false);
final ActiveMQTextMessage activeMQTextMessage = new ActiveMQTextMessage();
activeMQTextMessage.setText("MESSAGE");
activeMQTextMessage.setPersistent(true);
jmsTemplate.execute("TEST_QUEUE", ((session, messageProducer) -> {
try {
log.info("Sending to Queue.");
messageProducer.send(activeMQTextMessage, DeliveryMode.PERSISTENT, 4, 30000);
session.commit();
session.close();
log.info("Committed and Closed.");
} catch (Exception e) {
e.printStackTrace();
log.error(e.getMessage());
session.rollback();
session.close();
}
return session;
}));
log.info("Create session from conn factory.");
final Session session = jmsTemplate.getConnectionFactory().createConnection().createSession();
log.info("Consumer creation.");
final ActiveMQMessageConsumer consumer = (ActiveMQMessageConsumer) session.createConsumer(activeMQQueue);
log.info("Consume Message");
log.info(consumer.receive(100L));
}
}
The log output:
02 Mar 2021 16:48:34,298 [ INFO] --- o.a.a.b.BrokerService : Using Persistence Adapter: MemoryPersistenceAdapter
02 Mar 2021 16:48:34,438 [ INFO] --- o.a.a.b.BrokerService : Apache ActiveMQ 5.16.1 (localhost, ID:devbox-44103-1614703714311-0:1) is starting
02 Mar 2021 16:48:34,442 [DEBUG] --- o.a.a.b.j.Log4JConfigView : Could not locate log4j classes on classpath.
02 Mar 2021 16:48:34,442 [ INFO] --- o.a.a.b.BrokerService : Apache ActiveMQ 5.16.1 (localhost, ID:devbox-44103-1614703714311-0:1) started
02 Mar 2021 16:48:34,442 [ INFO] --- o.a.a.b.BrokerService : For help or more information please see: http://activemq.apache.org
02 Mar 2021 16:48:34,445 [DEBUG] --- o.a.a.b.r.AbstractRegion : localhost adding destination: topic://ActiveMQ.Advisory.MasterBroker
02 Mar 2021 16:48:34,452 [DEBUG] --- o.a.a.t.TaskRunnerFactory : Initialized TaskRunnerFactory[ActiveMQ BrokerService[localhost] Task] using ExecutorService: java.util.concurrent.ThreadPoolExecutor#55bf08a5[Running, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
02 Mar 2021 16:48:34,456 [DEBUG] --- o.a.a.t.v.VMTransportFactory : binding to broker: localhost
02 Mar 2021 16:48:34,459 [ INFO] --- o.a.a.b.TransportConnector : Connector vm://localhost started
02 Mar 2021 16:48:34,463 [DEBUG] --- o.a.a.t.TaskRunnerFactory : Initialized TaskRunnerFactory[ActiveMQ VMTransport: vm://localhost#0] using ExecutorService: java.util.concurrent.ThreadPoolExecutor#297db6ad[Running, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
02 Mar 2021 16:48:34,472 [DEBUG] --- o.a.a.t.TaskRunnerFactory : Initialized TaskRunnerFactory[ActiveMQ VMTransport: vm://localhost#1] using ExecutorService: java.util.concurrent.ThreadPoolExecutor#170437d4[Running, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
02 Mar 2021 16:48:34,474 [DEBUG] --- o.a.a.b.TransportConnection : Setting up new connection id: ID:devbox-44103-1614703714311-4:1, address: vm://localhost#0, info: ConnectionInfo {commandId = 1, responseRequired = true, connectionId = ID:devbox-44103-1614703714311-4:1, clientId = ID:devbox-44103-1614703714311-3:1, clientIp = null, userName = admin, password = *****, brokerPath = null, brokerMasterConnector = false, manageable = true, clientMaster = true, faultTolerant = false, failoverReconnect = false}
02 Mar 2021 16:48:34,475 [DEBUG] --- o.a.a.b.TransportConnector : Publishing: vm://localhost for broker transport URI: vm://localhost
02 Mar 2021 16:48:34,475 [DEBUG] --- o.a.a.b.TransportConnector : Publishing: vm://localhost for broker transport URI: vm://localhost
02 Mar 2021 16:48:34,475 [DEBUG] --- o.a.a.b.r.AbstractRegion : localhost adding destination: topic://ActiveMQ.Advisory.Connection
02 Mar 2021 16:48:34,480 [DEBUG] --- o.a.a.b.r.AbstractRegion : localhost adding consumer: ID:devbox-44103-1614703714311-4:1:-1:1 for destination: ActiveMQ.Advisory.TempQueue,ActiveMQ.Advisory.TempTopic
02 Mar 2021 16:48:34,514 [DEBUG] --- o.a.a.b.r.AbstractRegion : localhost adding destination: queue://TEST_QUEUE
02 Mar 2021 16:48:34,529 [DEBUG] --- o.a.a.b.r.Queue : queue://TEST_QUEUE, subscriptions=0, memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0, pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0, dequeueCount: 0, memUsage:0, maxPageSize:200
02 Mar 2021 16:48:34,530 [DEBUG] --- o.a.a.b.TransportConnector : Publishing: vm://localhost for broker transport URI: vm://localhost
02 Mar 2021 16:48:34,530 [DEBUG] --- o.a.a.b.TransportConnector : Publishing: vm://localhost for broker transport URI: vm://localhost
02 Mar 2021 16:48:34,530 [DEBUG] --- o.a.a.b.r.AbstractRegion : localhost adding destination: topic://ActiveMQ.Advisory.Queue
02 Mar 2021 16:48:34,532 [DEBUG] --- o.a.a.b.TransportConnector : Publishing: vm://localhost for broker transport URI: vm://localhost
02 Mar 2021 16:48:34,532 [DEBUG] --- o.a.a.b.TransportConnector : Publishing: vm://localhost for broker transport URI: vm://localhost
02 Mar 2021 16:48:34,532 [DEBUG] --- o.a.a.b.r.AbstractRegion : localhost adding destination: topic://ActiveMQ.Advisory.Producer.Queue.TEST_QUEUE
02 Mar 2021 16:48:34,534 [ INFO] --- m.a.w.c.SessionActiveMQIT : Sending to Queue.
02 Mar 2021 16:48:34,535 [DEBUG] --- o.a.a.TransactionContext : Begin:TX:ID:devbox-44103-1614703714311-4:1:1
02 Mar 2021 16:48:34,536 [DEBUG] --- o.a.a.ActiveMQSession : ID:devbox-44103-1614703714311-4:1:1 Transaction Commit :TX:ID:devbox-44103-1614703714311-4:1:1
02 Mar 2021 16:48:34,536 [DEBUG] --- o.a.a.TransactionContext : Commit: TX:ID:devbox-44103-1614703714311-4:1:1 syncCount: 0
02 Mar 2021 16:48:34,539 [DEBUG] --- o.a.a.t.LocalTransaction : commit: TX:ID:devbox-44103-1614703714311-4:1:1 syncCount: 1
02 Mar 2021 16:48:34,540 [DEBUG] --- o.a.a.b.r.Queue : localhost Message ID:devbox-44103-1614703714311-4:1:1:1:1 sent to queue://TEST_QUEUE
02 Mar 2021 16:48:34,541 [ INFO] --- m.a.w.c.SessionActiveMQIT : Committed and Closed.
02 Mar 2021 16:48:34,541 [DEBUG] --- o.a.a.b.r.Queue : queue://TEST_QUEUE, subscriptions=0, memory=0%, size=1, pending=0 toPageIn: 1, force:false, Inflight: 0, pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 1, dequeueCount: 0, memUsage:1038, maxPageSize:200
02 Mar 2021 16:48:34,545 [ INFO] --- m.a.w.c.SessionActiveMQIT : Create session from conn factory.
02 Mar 2021 16:48:34,545 [DEBUG] --- o.a.a.b.j.ManagementContext : Unregistering MBean org.apache.activemq:type=Broker,brokerName=localhost,destinationType=Queue,destinationName=TEST_QUEUE,endpoint=Producer,clientId=ID_devbox-44103-1614703714311-3_1,producerId=ID_devbox-44103-1614703714311-4_1_1_1
02 Mar 2021 16:48:34,546 [ INFO] --- m.a.w.c.SessionActiveMQIT : Consumer creation.
02 Mar 2021 16:48:34,546 [DEBUG] --- o.a.a.b.TransportConnector : Publishing: vm://localhost for broker transport URI: vm://localhost
02 Mar 2021 16:48:34,546 [DEBUG] --- o.a.a.b.TransportConnector : Publishing: vm://localhost for broker transport URI: vm://localhost
02 Mar 2021 16:48:34,552 [DEBUG] --- o.a.a.b.r.AbstractRegion : localhost adding consumer: ID:devbox-44103-1614703714311-4:1:2:1 for destination: queue://TEST_QUEUE
02 Mar 2021 16:48:34,558 [DEBUG] --- o.a.a.b.r.Queue : queue://TEST_QUEUE add sub: QueueSubscription: consumer=ID:devbox-44103-1614703714311-4:1:2:1, destinations=0, dispatched=0, delivered=0, pending=0, prefetch=1000, prefetchExtension=0, dequeues: 0, dispatched: 0, inflight: 0
02 Mar 2021 16:48:34,560 [DEBUG] --- o.a.a.b.TransportConnector : Publishing: vm://localhost for broker transport URI: vm://localhost
02 Mar 2021 16:48:34,560 [DEBUG] --- o.a.a.b.r.Queue : queue://TEST_QUEUE, subscriptions=1, memory=0%, size=1, pending=0 toPageIn: 1, force:false, Inflight: 0, pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 1, dequeueCount: 0, memUsage:1038, maxPageSize:200
02 Mar 2021 16:48:34,560 [DEBUG] --- o.a.a.b.TransportConnector : Publishing: vm://localhost for broker transport URI: vm://localhost
02 Mar 2021 16:48:34,560 [DEBUG] --- o.a.a.b.r.AbstractRegion : localhost adding destination: topic://ActiveMQ.Advisory.Consumer.Queue.TEST_QUEUE
02 Mar 2021 16:48:34,562 [ INFO] --- m.a.w.c.SessionActiveMQIT : Consume Message
02 Mar 2021 16:48:34,662 [ INFO] --- m.a.w.c.SessionActiveMQIT : null
I believe you need to call start() on your instance of javax.jms.Connection in order to get messages to flow to the consumer, e.g.:
final Connection connection = jmsTemplate.getConnectionFactory().createConnection();
final Session session = connection.createSession();
connection.start()
Also, be sure to close your resources (i.e. connection, session, consumer) when you're done with them. Currently they just fall out of scope which means they are being leaked. I understand this is just a test, but even still it's good practice.

MFP 7.1 - FWLSE0342E: Grant code validation failed: Grant code was already used

Facing Grand code validation failed issue when client request is made for one of the adapter.
This happens only in cluster environment. We have configured it as Round Robin.
When running server independently one or the other we don't see this issue. This issue arrises when both are up and running.
Logs from Application server messages.log
[INFO ] SRVE0242I: [DHSProject] [/DHSProject] [PushWorksApplication]: Initialization successful.
[INFO ] The following JAX-RS application has been processed: com.ibm.ws.jaxrs.webcontainer.JAXRSDefaultApplicationSubclassProxy
[INFO ] The server has registered the JAX-RS resource class com.worklight.oauth.TokenValidationEndpoint with #Path(/validation).
[INFO ] The server has registered the JAX-RS resource class com.worklight.oauth.TokenEndpoint with #Path(/token).
[INFO ] There are no custom JAX-RS providers defined in the application.
[INFO ] SRVE0242I: [DHSProject] [/DHSProject] [AuthorizationServer]: Initialization successful.
[AUDIT ] CWWKZ0001I: Application DHSProject started in 64.058 seconds.
[INFO ] FWLSE0277I: Creating an ILMT record in the file '/var/opt/IBM/WebSphere/Liberty/usr/servers/mobilefirst/logs/ae6695ccf7cfe74ee108bf753b1a76d5.slmtag'.
[INFO ] Resource conf/jndi/default.properties not found. This is not an error. Context path is /worklightconsole
[INFO ] The endpoint used to invoke the MFP administration services is https://MyHostname:443/worklightadmin
[INFO ] Detected Liberty farm runtime
[INFO ] Detected Liberty farm runtime
[INFO ] Detected Liberty farm runtime
[INFO ] Detected Liberty farm runtime
[INFO ] Detected Liberty farm runtime
[INFO ] Detected Liberty farm runtime
[INFO ] Detected Liberty farm runtime
[INFO ] Detected Liberty farm runtime
[INFO ] Detected Liberty farm runtime
[INFO ] Detected Liberty farm runtime
[INFO ] Detected Liberty farm runtime
[INFO ] Detected Liberty farm runtime
[INFO ] Detected Liberty farm runtime
[INFO ] Detected Liberty farm runtime
[INFO ] Detected Liberty farm runtime
[INFO ] Detected Liberty farm runtime
[INFO ] SRVE0242I: [DHSProject] [/DHSProject] [RESTAdaptersServiceServlet]: Initialization successful.
[INFO ] SRVE0242I: [DHSProject] [/DHSProject] [GadgetAPIServlet]: Initialization successful.
[INFO ] SRVE0242I: [DHSProject] [/DHSProject] [ClientLogUploaderServlet]: Initialization successful.
[ERROR ] FWLSE0342E: Grant code validation failed: Grant code was already used. ClientId:0459405be995d13e209ac40d56c1d73a6eee8ec6, sessionId:187AE2B5-1673-41CA-87EB-8BFA0FDC7772, grant code:CkvKJl4tDyIaTtcKb1KiBwYMhaWdvHq60z6PsdbbyPGAv-TnU4RP1Vqldw-qVWLyqmychO0FST_myOJKzaZDnx9zG5ZbtkR1uV34QvW4F-g7rXJdMsG1XdnIVkq5RAJaSVogUNfEGGeMM4W3UbwbISTCKhPbuu3esBzKh86fCRg [project DHSProject]
Grant code was already used
[ERROR ] FWLSE0342E: Grant code validation failed: Grant code was already used. ClientId:0459405be995d13e209ac40d56c1d73a6eee8ec6, sessionId:17E31752-B5FB-46D0-843C-407893F2417F, grant code:arVpm_yAPDg3Q28uFtBzSP-6PVh6Os-LXSkTcszKeoBANO3TJhW1ydNfej4KvsfhkTEQI7alzdlFsop9QvdzQP_NFh5q7LsHPOSIV5YE6lnBfwzkeKDRMCZnjvOECT7P_hiLdGPzEYFB8vhSHnHusB4jrdkV4a96-LIMOi9cXMY [project DHSProject]
Grant code was already used
[ERROR ] FWLSE0342E: Grant code validation failed: Grant code was already used. ClientId:0459405be995d13e209ac40d56c1d73a6eee8ec6, sessionId:17E31752-B5FB-46D0-843C-407893F2417F, grant code:JVckFNnonbcw0UpoZ7-Iq7CmXWBgYXofD0cavY_9ctIUN1dIqK3K_WsqSJoUPEAgONSSvyyjBdVNkU5tova_XiYQ7bO_mlcaB2ynQJNQX-X3RU-4yXhaS9zdPkV5o0aBUQGUqN2L7aeArxVi-yTRnQNiDf4UyOzJ0R310efebts [project DHSProject]
Grant code was already used
This is the log from my Mobile device Iphone
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [DEBUG] [WORKLIGHT] +[WLClient sharedInstance] in WLClient.m:165 :: IBMMobilieFirstFoundation.framework version = 7.1-2016/01/29 19:42:47
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: SessionTimeoutService elapsedTime : NaN
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] Constructing
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] open method GET url https://myhostname:443/MYProject/adapters/FaqAdapter/faqs
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] setRequestHeader name Authorization value Bearer eyJhbGciOiJSUzI1NiIsImpwayI6eyJhbGciOiJSU0EiLCJleHAiOiJBUUFCIiwibW9kIjoiQU1VTDg0WXZiN0RXVVpaUFRlaEIydUVwT2Mwd0ZNLXdPQ05aODFLQTkyYkc4Sm00UlNCNUQyU2xiU2U3UGlQVVZfcmJsd1BrSWQ2NWdvLTExaDg5Y0ltZDQxc0NxcnY4Q3hzVThGYUNVV1Q5TXl6Sy1tR2RBeWJGOTNUWTM2WEVUaC1tMzVGcE9qeG1abXpFVFcxalZZU1N2eDVzNFFjOXZfdDZaTG9ZN0VIOEMySjZucXdwcENqMlhVR1MwWWhLYkNEYWlXRmcwTURTbHNIWWdVaGJqZXloTVdZWHM1WnVKeUc3SmNUQ0V2OENScVdVNEIwMEV3a3hOZUxnRHVTd3RXdVI1MnYtY19vNm41Wi1VODhwRGU3MG51RWVhYmIxX21EQXE0T3cyUTIzakdmR2hCa1Z4OHlhR2NtZjFfMkozbmtvRGJQWF9vR25MbXA0dDltRFpZOCJ9fQ.eyJleHAiOjE0NjE1MTMzMDIsImltZi5zY29wZSI6eyJ3bF9kaXJlY3RVcGRhdGVSZWFsbSI6eyJleHAiOjE0NjE1MTMyODcsIm1hbmRhdG9yeSI6dHJ1ZX0sIndsX3JlbW90ZURpc2FibGVSZWFsbSI6eyJleHAiOjE0NjE1MDk5ODcsIm1hbmRhdG9yeSI6dHJ1ZX0sIndsX2FudGlYU1JGUmVhbG0iOnsiZXhwIjoxNDYxNTEzMjg4LCJtYW5kYXRvcnkiOnRydWV9LCJ3bF9kZXZpY2VOb1Byb3Zpc2lvbmluZ1JlYWxtIjp7ImV4cCI6MTQ2MTUxMzI4NywibWFuZGF0b3J5Ijp0cnVlfSwid2xfYW5vbnltb3VzVXNlclJlYWxtIjp7ImV4cCI6MTQ2MTUxMzI4NywibWFuZGF0b3J5Ijp0cnVlfX0sImlzcyI6Imh0dHBzOlwvXC91YXRjc3Btb2JpbGUuZGhzLmdhLmdvdjo0NDNcL0RIU1Byb2plY3RcL2F1dGhvcml6YXRpb25cLyIsInBybiI6IjA0NTk0MDViZTk5NWQxM2UyMDlhYzQwZDU2YzFkNzNhNmVlZThlYzYifQ.uyvCAj7zER-golAnI5bziibDHcS4ug1j1u20LSICNUHYTj3Dsmp4vxr5A8zZoylzVIDqEyu3PrtVGhIO58Ywjfr-W13n07IFtAGtWBp2a_ELy6z0lJsfy4-6sNW0YHoJxeUj6UeoZJJkUg6lXBa6sMaNebhY7Sfv_CwF_sh_4KsZprpRWhJmI2XPNqNcbpFrhzcFfIPjh0ANhIWDpeJDLcU6Bs4YNCw0yUcHxd2izlwGdBrh4ErlUsrPkSTvSSYOKwicO7s-XtG-4o4SzwfItjtvzjuS6M0SHgtRWUe8pgabf9G9bvK_AalU2IKzYtrOL1GDIM8djcI3MULNM2q25A eyJhbGciOiJSUzI1NiIsImpwayI6eyJhbGciOiJSU0EiLCJleHAiOiJBUUFCIiwibW9kIjoiQU1VTDg0WXZiN0RXVVpaUFRlaEIydUVwT2Mwd0ZNLXdPQ05aODFLQTkyYkc4Sm00UlNCNUQyU2xiU2U3UGlQVVZfcmJsd1BrSWQ2NWdvLTExaDg5Y0ltZDQxc0NxcnY4Q3hzVThGYUNVV1Q5TXl6Sy1tR2RBeWJGOTNUWTM2WEVUaC1tMzVGcE9qeG1abXpFVFcxalZZU1N2eDVzNFFjOXZfdDZaTG9ZN0VIOEMySjZucXdwcENqMlhVR1MwWWhLYkNEYWlXRmcwTURTbHNIWWdVaGJqZXloTVdZWHM1WnVKeUc3SmNUQ0V2OENScVdVNEIwMEV3a3hOZUxnRHVTd3RXdVI1MnYtY19vNm41Wi1VODhwRGU3MG51RWVhYmIxX21EQXE0T3cyUTIzakdmR2hCa1Z4OHlhR2NtZjFfMkozbmtvRGJQWF9vR25MbXA0dDltRFpZOCJ9fQ.eyJleHAiOjE0NjE1MTMzMDIsInN1YiI6Ijo6MDQ1OTQwNWJlOTk1ZDEzZTIwOWFjNDBkNTZjMWQ3M2E2ZWVlOGVjNiIsImltZi5hbmFseXRpY3MiOnsiaWQiOiJEQ1NTTW9iaWxlQXBwIiwiZW52aXJvbm1lbnQiOiJpcGhvbmUiLCJ2ZXJzaW9uIjoiMS4wIn0sImltZi5hcHBsaWNhdGlvbiI6eyJpZCI6IkRDU1NNb2JpbGVBcHAiLCJlbnZpcm9ubWVudCI6ImlwaG9uZSIsInZlcnNpb24iOiIxLjAifSwiaXNzIjoiaHR0cHM6XC9cL3VhdGNzcG1vYmlsZS5kaHMuZ2EuZ292OjQ0M1wvREhTUHJvamVjdFwvYXV0aG9yaXphdGlvblwvIiwiaWF0IjoxNDYxNTA5NzAyLCJpbWYuZGV2aWNlIjp7ImlkIjoiMTYyMTEzOTAtQjYxMS00NUE1LUFCRjItRTUxM0M2NTQzREYwIiwicGxhdGZvcm0iOiJpT1MiLCJtb2RlbCI6ImlQaG9uZSIsIm9zVmVyc2lvbiI6IjkuMy4xIn19.Cy2PdsqyngYDyNHtQsqXMFavXJW8ZHKYbYgEkyqbI12LZzGQEqAtMXn_WaAEfMiDvL1FO-rzB7pU1rvyFTfwpW2WmIBJZXVfxg5YGQPJDAhPCGo3lOU23BBRnQgr62zG57IVUmN23HBNgfmFuOv4NawJYoKYNDgs9ChRNC_ZVFkek27ECU1xOLMW59u_yJ9ZmYwQSVG34rrWgUwgAJ0HOhFzH04Pl3tKxPuva5W1I1elAWhLLuLtmCGytumsZJOaJHeXpAkU2vGEnrE0hk2bScAAIiqRmvGV4MYd9qyR06-z2MZSXo3r7sfWpl-gvCH99rrGzCSP7C8BL45dv-mh9w
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] send
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [DEBUG] [WL_AFHTTPRequestOperationManagerWrapper_PACKAGE] +[WLAFHTTPRequestOperationManagerWrapper requestWithURL:] in WLAFHTTPRequestOperationManagerWrapper.m:52 :: Request url is https://myhostname:443/MYProject/adapters/FaqAdapter/faqs
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [DEBUG] [WL_AFHTTPRequestOperationManagerWrapper_PACKAGE] -[WLAFHTTPRequestOperationManagerWrapper start] in WLAFHTTPRequestOperationManagerWrapper.m:332 :: Starting the request with URL https://myhostname:443/MYProject/adapters/FaqAdapter/faqs
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [DEBUG] [WL_AFHTTPRequestOperationManagerWrapper_PACKAGE] -[WLAFHTTPRequestOperationManagerWrapper requestFailed:error:] in WLAFHTTPRequestOperationManagerWrapper.m:364 :: Request Failed
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [DEBUG] [WL_AFHTTPRequestOperationManagerWrapper_PACKAGE] -[WLAFHTTPRequestOperationManagerWrapper requestFailed:error:] in WLAFHTTPRequestOperationManagerWrapper.m:365 :: Response Status Code : 403
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [DEBUG] [WL_AFHTTPRequestOperationManagerWrapper_PACKAGE] -[WLAFHTTPRequestOperationManagerWrapper requestFailed:error:] in WLAFHTTPRequestOperationManagerWrapper.m:366 :: Response Error : Request failed: forbidden (403)
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [DEBUG] [WORKLIGHT] +[WLClient sharedInstance] in WLClient.m:165 :: IBMMobilieFirstFoundation.framework version = 7.1-2016/01/29 19:42:47
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [DEBUG] [NONE] Request [https://myhostname:443/MYProject/authorization/v1/authorization]
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [NONE] Application details header: {"applicationDetails":{"platformVersion":"7.1.0.0","nativeVersion":"193323332","skinName":"default","skinChecksum":3762916697,"skinLoaderChecksum":"(null)"}}
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: THREAD WARNING: ['WLAuthorizationManagerPlugin'] took '17.318848' ms. Plugin should use a background thread.
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] Constructing
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] open method GET url https://myhostname:443/MYProject/authorization/v1/authorization?response_type=code&client_id=0459405be995d13e209ac40d56c1d73a6eee8ec6&redirect_uri=http%3A%2F%2Fmfpredirecturi&isAjaxRequest=true&x=0.8985385324340314
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] setRequestHeader name X-Requested-With value XMLHttpRequest
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] setRequestHeader name Accept value text/javascript, text/html, application/xml, text/xml, */*
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] setRequestHeader name Accept-Language value en-US
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] setRequestHeader name x-wl-app-version value 1.0
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] setRequestHeader name x-wl-app-details value {"applicationDetails":{"platformVersion":"7.1.0.0","nativeVersion":"193323332","skinName":"default","skinChecksum":3762916697,"skinLoaderChecksum":"(null)"}}
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] setRequestHeader name X-WL-Session value 17E31752-B5FB-46D0-843C-407893F2417F
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] setRequestHeader name X-WL-ClientId value 0459405be995d13e209ac40d56c1d73a6eee8ec6
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] setRequestHeader name X-WL-S-ClientId value eyJhbGciOiJSUzI1NiIsImpwayI6eyJhbGciOiJSU0EiLCJtb2QiOiJBSl9QbmI5aVpvQVNRcFFYZEc0NzNHQnctV3FCcFNITGw1N1ZIdG8yZHQyR2dULTUzM0o0TFpwaEpkX2gyYnJMa0h5MmFqZUZYYlRWVXhJQTVja290MkU9IiwiZXhwIjoiQVFBQiJ9fQ==.eyJjbGllbnRJZCI6IjA0NTk0MDViZTk5NWQxM2UyMDlhYzQwZDU2YzFkNzNhNmVlZThlYzYifQ==.U4ls1NvDvDA5aDiL3XN4vaBITv2pI0WQLi6PwYEjjFvJpqgMeyGiFuOjB4idjWgZGQOBSXIAVo3ykLhAPHcUyw==
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] send
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [DEBUG] [WL_AFHTTPRequestOperationManagerWrapper_PACKAGE] +[WLAFHTTPRequestOperationManagerWrapper requestWithURL:] in WLAFHTTPRequestOperationManagerWrapper.m:52 :: Request url is https://myhostname:443/MYProject/authorization/v1/authorization?response_type=code&client_id=0459405be995d13e209ac40d56c1d73a6eee8ec6&redirect_uri=http%3A%2F%2Fmfpredirecturi&isAjaxRequest=true&x=0.8985385324340314
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [DEBUG] [WL_AFHTTPRequestOperationManagerWrapper_PACKAGE] -[WLAFHTTPRequestOperationManagerWrapper start] in WLAFHTTPRequestOperationManagerWrapper.m:332 :: Starting the request with URL https://myhostname:443/MYProject/authorization/v1/authorization?response_type=code&client_id=0459405be995d13e209ac40d56c1d73a6eee8ec6&redirect_uri=http%3A%2F%2Fmfpredirecturi&isAjaxRequest=true&x=0.8985385324340314
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [DEBUG] [WL_AFHTTPRequestOperationManagerWrapper_PACKAGE] -[WLAFHTTPRequestOperationManagerWrapper requestFailed:error:] in WLAFHTTPRequestOperationManagerWrapper.m:364 :: Request Failed
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [DEBUG] [WL_AFHTTPRequestOperationManagerWrapper_PACKAGE] -[WLAFHTTPRequestOperationManagerWrapper requestFailed:error:] in WLAFHTTPRequestOperationManagerWrapper.m:365 :: Response Status Code : 401
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [DEBUG] [WL_AFHTTPRequestOperationManagerWrapper_PACKAGE] -[WLAFHTTPRequestOperationManagerWrapper requestFailed:error:] in WLAFHTTPRequestOperationManagerWrapper.m:366 :: Response Error : Request failed: unauthorized (401)
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [DEBUG] [WORKLIGHT] +[WLClient sharedInstance] in WLClient.m:165 :: IBMMobilieFirstFoundation.framework version = 7.1-2016/01/29 19:42:47
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WL_AUTH] -[WLDeviceAuthManager getWLUniqueDeviceId] in WLDeviceAuthManager.m:71 :: returning UUID from the keychain
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [NONE] Application details header: {"applicationDetails":{"platformVersion":"7.1.0.0","nativeVersion":"193323332","skinName":"default","skinChecksum":3762916697,"skinLoaderChecksum":"(null)"}}
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [DEBUG] [NONE] Request [https://myhostname:443/MYProject/authorization/v1/authorization]
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: THREAD WARNING: ['WLAuthorizationManagerPlugin'] took '12.122803' ms. Plugin should use a background thread.
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] Constructing
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] open method GET url https://myhostname:443/MYProject/authorization/v1/authorization?response_type=code&client_id=0459405be995d13e209ac40d56c1d73a6eee8ec6&redirect_uri=http%3A%2F%2Fmfpredirecturi&isAjaxRequest=true&x=0.5182551073376089
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] setRequestHeader name X-Requested-With value XMLHttpRequest
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] setRequestHeader name Accept value text/javascript, text/html, application/xml, text/xml, */*
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] setRequestHeader name Accept-Language value en-US
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] setRequestHeader name x-wl-app-version value 1.0
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] setRequestHeader name x-wl-app-details value {"applicationDetails":{"platformVersion":"7.1.0.0","nativeVersion":"193323332","skinName":"default","skinChecksum":3762916697,"skinLoaderChecksum":"(null)"}}
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] setRequestHeader name Authorization value {"wl_deviceNoProvisioningRealm":{"ID":{"token":"jc8scbrogj2cstljm4e6f6osov","app":{"id":"MyMobileApp","version":"1.0"},"device":{"id":"16211390-B611-45A5-ABF2-E513C6543DF0","os":"9.3.1","model":"iPhone7,2","environment":"iphone"},"custom":{}}}}
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] setRequestHeader name X-WL-Session value 17E31752-B5FB-46D0-843C-407893F2417F
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] setRequestHeader name X-WL-ClientId value 0459405be995d13e209ac40d56c1d73a6eee8ec6
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] setRequestHeader name X-WL-S-ClientId value eyJhbGciOiJSUzI1NiIsImpwayI6eyJhbGciOiJSU0EiLCJtb2QiOiJBSl9QbmI5aVpvQVNRcFFYZEc0NzNHQnctV3FCcFNITGw1N1ZIdG8yZHQyR2dULTUzM0o0TFpwaEpkX2gyYnJMa0h5MmFqZUZYYlRWVXhJQTVja290MkU9IiwiZXhwIjoiQVFBQiJ9fQ==.eyJjbGllbnRJZCI6IjA0NTk0MDViZTk5NWQxM2UyMDlhYzQwZDU2YzFkNzNhNmVlZThlYzYifQ==.U4ls1NvDvDA5aDiL3XN4vaBITv2pI0WQLi6PwYEjjFvJpqgMeyGiFuOjB4idjWgZGQOBSXIAVo3ykLhAPHcUyw==
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WLNativeXHR] send
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [DEBUG] [WL_AFHTTPRequestOperationManagerWrapper_PACKAGE] +[WLAFHTTPRequestOperationManagerWrapper requestWithURL:] in WLAFHTTPRequestOperationManagerWrapper.m:52 :: Request url is https://myhostname:443/MYProject/authorization/v1/authorization?response_type=code&client_id=0459405be995d13e209ac40d56c1d73a6eee8ec6&redirect_uri=http%3A%2F%2Fmfpredirecturi&isAjaxRequest=true&x=0.5182551073376089
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [DEBUG] [WL_AFHTTPRequestOperationManagerWrapper_PACKAGE] -[WLAFHTTPRequestOperationManagerWrapper start] in WLAFHTTPRequestOperationManagerWrapper.m:332 :: Starting the request with URL https://myhostname:443/MYProject/authorization/v1/authorization?response_type=code&client_id=0459405be995d13e209ac40d56c1d73a6eee8ec6&redirect_uri=http%3A%2F%2Fmfpredirecturi&isAjaxRequest=true&x=0.5182551073376089
Apr 24 11:23:45 My-iPhone MyMobileApp[794] <Warning>: [DEBUG] [WL_AFHTTPRequestOperationManagerWrapper_PACKAGE] -[WLAFHTTPRequestOperationManagerWrapper requestFailed:error:] in WLAFHTTPRequestOperationManagerWrapper.m:364 :: Request Failed
Apr 24 11:23:46 My-iPhone MyMobileApp[794] <Warning>: [DEBUG] [WL_AFHTTPRequestOperationManagerWrapper_PACKAGE] -[WLAFHTTPRequestOperationManagerWrapper requestFailed:error:] in WLAFHTTPRequestOperationManagerWrapper.m:365 :: Response Status Code : 401
Apr 24 11:23:46 My-iPhone MyMobileApp[794] <Warning>: [DEBUG] [WL_AFHTTPRequestOperationManagerWrapper_PACKAGE] -[WLAFHTTPRequestOperationManagerWrapper requestFailed:error:] in WLAFHTTPRequestOperationManagerWrapper.m:366 :: Response Error : Request failed: unauthorized (401)
Apr 24 11:23:46 My-iPhone MyMobileApp[794] <Warning>: [DEBUG] [WORKLIGHT] +[WLClient sharedInstance] in WLClient.m:165 :: IBMMobilieFirstFoundation.framework version = 7.1-2016/01/29 19:42:47
Apr 24 11:23:46 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [WL_AUTH] -[WLDeviceAuthManager getWLUniqueDeviceId] in WLDeviceAuthManager.m:71 :: returning UUID from the keychain
Apr 24 11:23:46 My-iPhone MyMobileApp[794] <Warning>: [TRACE] [NONE] Application details header: {"applicationDetails":{"platformVersion":"7.1.0.0","nativeVersion":"193323332","skinName":"default","skinChecksum":3762916697,"skinLoaderChecksum":"(null)"}}
If you observe in client log when mobile app invokes one of the Adapter
GET url https://myhostname:443/MYProject/adapters/FaqAdapter/faqs
second call it makes for Authorization which return as 401. This authorization call is made in loop for many times and finally fails
https://myhostname:443/MYProject/authorization/v1/authorization?response_type=code&client_id=0459405be995d13e209ac40d56c1d73a6eee8ec6&redirect_uri=http%3A%2F%2Fmfpredirecturi&isAjaxRequest=true&x=0.8985385324340314
Probably the Grand code is issued by one server from cluster but the same code failed on another server. Perhaps the Grand code is not in snyc between 2 cluster servers.
My worklight Properties
mfp.session.independent=false
serverSessionTimeout=10
mfp.attrStore.type=httpSession
Taking in the following parameters that you explained:
You run in session-mode.
It fails only when you run multiple servers.
This strongly leads me to believe that you do not correctly follow the requirement of using sticky sessions. Meaning, each client session must always use the same server. If the client switches between server, then the session is lost and the context of the request is lost as well.
Or, alternatively, you can switch to session independent mode if you do not want to use sticky sessions.

Apache 2.4.10 + mod_proxy_fcgi + PHP-FPM with CHROOT => 404 Error

First of all I've tried to set up a basic configuration for Apache 2.4, mod_proxy_fcgi and PHP-FPM on a Debian Jessie (Testing) machine.
Everything works fine when opening .php-files.
However, If I activate chroot for PHP-FPM I only get a "File not found." message inside my browser.
.
Configuration
partial content of apache2.conf
<FilesMatch \.php$>
SetHandler "proxy:unix:/var/run/php5-fpm.sock|fcgi://localhost"
</FilesMatch>
content of /var/wwww/html
x1#vm1:~$ ls -l /var/www/html/
-rw-r--r-- 1 www-data www-data 19 Jan 15 23:37 index.php
partial content of /etc/php5/fpm/pool.d/www*
prefix = /var/www/html
chroot = $prefix
chdir = /
catch_workers_output = yes
.Steps / logfiles for searching the error
Apache error.log
[proxy_fcgi:error] [pid 12615:tid 140653535131392] [client 1.2.3.4:123] AH01071: Got error 'Primary script unknown\n'
Apache Access.log
1.2.3.4- - [16/Jan/2015:01:22:58 +0100] "GET /index.php HTTP/1.1" 404 365 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko)
php5-fpm.log
[16-Jan-2015 01:22:55] NOTICE: configuration file /etc/php5/fpm/php-fpm.conf test is successful
[16-Jan-2015 01:22:56] NOTICE: fpm is running, pid 12781
[16-Jan-2015 01:22:56] NOTICE: ready to handle connections
[16-Jan-2015 01:22:56] NOTICE: systemd monitor interval set to 10000ms
Apache error.log with loglevel trace8 and PHP5-FPM chroot on
[core:trace6] [pid 9794:tid 140072171042560] core_filters.c(527): [client 1.2.3.4:61149] core_output_filter: flushing because of FLUSH bucket
[core:trace5] [pid 9794:tid 140072332166912] protocol.c(618): [client 1.2.3.4:61152] Request received from client: GET /index.php HTTP/1.1
[http:trace4] [pid 9794:tid 140072332166912] http_request.c(301): [client 1.2.3.4:61152] Headers received from client:
[http:trace4] [pid 9794:tid 140072332166912] http_request.c(305): [client 1.2.3.4:61152] Host: example.com
[http:trace4] [pid 9794:tid 140072332166912] http_request.c(305): [client 1.2.3.4:61152] Connection: keep-alive
[http:trace4] [pid 9794:tid 140072332166912] http_request.c(305): [client 1.2.3.4:61152] Cache-Control: max-age=0
[http:trace4] [pid 9794:tid 140072332166912] http_request.c(305): [client 1.2.3.4:61152] Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
[http:trace4] [pid 9794:tid 140072332166912] http_request.c(305): [client 1.2.3.4:61152] User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36
[http:trace4] [pid 9794:tid 140072332166912] http_request.c(305): [client 1.2.3.4:61152] Accept-Encoding: gzip, deflate, sdch
[http:trace4] [pid 9794:tid 140072332166912] http_request.c(305): [client 1.2.3.4:61152] Accept-Language: de-DE,de;q=0.8,en-US;q=0.6,en;q=0.4
[authz_core:debug] [pid 9794:tid 140072332166912] mod_authz_core.c(809): [client 1.2.3.4:61152] AH01626: authorization result of Require all granted: granted
[authz_core:debug] [pid 9794:tid 140072332166912] mod_authz_core.c(809): [client 1.2.3.4:61152] AH01626: authorization result of <RequireAny>: granted
[core:trace3] [pid 9794:tid 140072332166912] request.c(238): [client 1.2.3.4:61152] request authorized without authentication by access_checker_ex hook: /index.php
[proxy:trace2] [pid 9794:tid 140072332166912] proxy_util.c(1938): [client 1.2.3.4:61152] *: found reverse proxy worker for unix:/var/run/php5-fpm.sock|fcgi://localhost/var/www/html/index.php
[proxy:trace2] [pid 9794:tid 140072332166912] proxy_util.c(1972): [client 1.2.3.4:61152] *: rewrite of url due to UDS(/var/run/php5-fpm.sock): fcgi://localhost/var/www/html/index.php (proxy:fcgi://localhost/var/www/html/index.php)
[proxy:debug] [pid 9794:tid 140072332166912] mod_proxy.c(1155): [client 1.2.3.4:61152] AH01143: Running scheme unix handler (attempt 0)
[proxy_fcgi:debug] [pid 9794:tid 140072332166912] mod_proxy_fcgi.c(786): [client 1.2.3.4:61152] AH01076: url: fcgi://localhost/var/www/html/index.php proxyname: (null) proxyport: 0
[proxy_fcgi:debug] [pid 9794:tid 140072332166912] mod_proxy_fcgi.c(793): [client 1.2.3.4:61152] AH01078: serving URL fcgi://localhost/var/www/html/index.php
[proxy:debug] [pid 9794:tid 140072332166912] proxy_util.c(2131): AH00942: FCGI: has acquired connection for (*)
[proxy:debug] [pid 9794:tid 140072332166912] proxy_util.c(2184): [client 1.2.3.4:61152] AH00944: connecting fcgi://localhost/var/www/html/index.php to localhost:8000
[proxy:debug] [pid 9794:tid 140072332166912] proxy_util.c(2217): [client 1.2.3.4:61152] AH02545: fcgi: has determined UDS as /var/run/php5-fpm.sock
[proxy:debug] [pid 9794:tid 140072332166912] proxy_util.c(2385): [client 1.2.3.4:61152] AH00947: connected /var/www/html/index.php to httpd-UDS:0
[proxy_fcgi:error] [pid 9794:tid 140072332166912] [client 1.2.3.4:61152] AH01071: Got error 'Primary script unknown\n'
[proxy_fcgi:trace4] [pid 9794:tid 140072332166912] util_script.c(522): [client 1.2.3.4:61152] Headers from script 'index.php':
[proxy_fcgi:trace4] [pid 9794:tid 140072332166912] util_script.c(523): [client 1.2.3.4:61152] Status: 404 Not Found
[proxy_fcgi:trace1] [pid 9794:tid 140072332166912] util_script.c(602): [client 1.2.3.4:61152] Status line from script 'index.php': 404 Not Found
[proxy_fcgi:trace4] [pid 9794:tid 140072332166912] util_script.c(523): [client 1.2.3.4:61152] X-Powered-By: PHP/5.6.4-4
[proxy_fcgi:trace4] [pid 9794:tid 140072332166912] util_script.c(523): [client 1.2.3.4:61152] Content-type: text/html; charset=UTF-8
[proxy:debug] [pid 9794:tid 140072332166912] proxy_util.c(2146): AH00943: FCGI: has released connection for (*)
[headers:trace2] [pid 9794:tid 140072332166912] mod_headers.c(874): AH01502: headers: ap_headers_output_filter()
[http:trace3] [pid 9794:tid 140072332166912] http_filters.c(1045): [client 1.2.3.4:61152] Response sent with status 404, headers:
[http:trace5] [pid 9794:tid 140072332166912] http_filters.c(1052): [client 1.2.3.4:61152] Date: Thu, 15 Jan 2015 23:54:44 GMT
[http:trace5] [pid 9794:tid 140072332166912] http_filters.c(1055): [client 1.2.3.4:61152] Server: Apache
[http:trace4] [pid 9794:tid 140072332166912] http_filters.c(874): [client 1.2.3.4:61152] X-Powered-By: PHP/5.6.4-4
[http:trace4] [pid 9794:tid 140072332166912] http_filters.c(874): [client 1.2.3.4:61152] X-Frame-Options: SAMEORIGIN
[http:trace4] [pid 9794:tid 140072332166912] http_filters.c(874): [client 1.2.3.4:61152] X-XSS-Protection: 1; mode=block
[http:trace4] [pid 9794:tid 140072332166912] http_filters.c(874): [client 1.2.3.4:61152] X-Content-Security-Policy: allow 'self';
[http:trace4] [pid 9794:tid 140072332166912] http_filters.c(874): [client 1.2.3.4:61152] X-Frame-Options: DENY
[http:trace4] [pid 9794:tid 140072332166912] http_filters.c(874): [client 1.2.3.4:61152] Content-Length: 16
[http:trace4] [pid 9794:tid 140072332166912] http_filters.c(874): [client 1.2.3.4:61152] Keep-Alive: timeout=5, max=100
[http:trace4] [pid 9794:tid 140072332166912] http_filters.c(874): [client 1.2.3.4:61152] Connection: Keep-Alive
[http:trace4] [pid 9794:tid 140072332166912] http_filters.c(874): [client 1.2.3.4:61152] Content-Type: text/html; charset=UTF-8
Apache error.log with loglevel trace8 and PHP5-FPM chroot off
[core:trace5] [pid 9794:tid 140072323774208] protocol.c(618): [client 1.2.3.4:61135] Request received from client: GET /index.php HTTP/1.1
[http:trace4] [pid 9794:tid 140072323774208] http_request.c(301): [client 1.2.3.4:61135] Headers received from client:
[http:trace4] [pid 9794:tid 140072323774208] http_request.c(305): [client 1.2.3.4:61135] Host: example.com
[http:trace4] [pid 9794:tid 140072323774208] http_request.c(305): [client 1.2.3.4:61135] Connection: keep-alive
[http:trace4] [pid 9794:tid 140072323774208] http_request.c(305): [client 1.2.3.4:61135] Cache-Control: max-age=0
[http:trace4] [pid 9794:tid 140072323774208] http_request.c(305): [client 1.2.3.4:61135] Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
[http:trace4] [pid 9794:tid 140072323774208] http_request.c(305): [client 1.2.3.4:61135] User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36
[http:trace4] [pid 9794:tid 140072323774208] http_request.c(305): [client 1.2.3.4:61135] Accept-Encoding: gzip, deflate, sdch
[http:trace4] [pid 9794:tid 140072323774208] http_request.c(305): [client 1.2.3.4:61135] Accept-Language: de-DE,de;q=0.8,en-US;q=0.6,en;q=0.4
[authz_core:debug] [pid 9794:tid 140072323774208] mod_authz_core.c(809): [client 1.2.3.4:61135] AH01626: authorization result of Require all granted: granted
[authz_core:debug] [pid 9794:tid 140072323774208] mod_authz_core.c(809): [client 1.2.3.4:61135] AH01626: authorization result of <RequireAny>: granted
[core:trace3] [pid 9794:tid 140072323774208] request.c(238): [client 1.2.3.4:61135] request authorized without authentication by access_checker_ex hook: /index.php
[proxy:trace2] [pid 9794:tid 140072323774208] proxy_util.c(1938): [client 1.2.3.4:61135] *: found reverse proxy worker for unix:/var/run/php5-fpm.sock|fcgi://localhost/var/www/html/index.php
[proxy:trace2] [pid 9794:tid 140072323774208] proxy_util.c(1972): [client 1.2.3.4:61135] *: rewrite of url due to UDS(/var/run/php5-fpm.sock): fcgi://localhost/var/www/html/index.php (proxy:fcgi://localhost/var/www/html/index.php)
[proxy:debug] [pid 9794:tid 140072323774208] mod_proxy.c(1155): [client 1.2.3.4:61135] AH01143: Running scheme unix handler (attempt 0)
[proxy_fcgi:debug] [pid 9794:tid 140072323774208] mod_proxy_fcgi.c(786): [client 1.2.3.4:61135] AH01076: url: fcgi://localhost/var/www/html/index.php proxyname: (null) proxyport: 0
[proxy_fcgi:debug] [pid 9794:tid 140072323774208] mod_proxy_fcgi.c(793): [client 1.2.3.4:61135] AH01078: serving URL fcgi://localhost/var/www/html/index.php
[proxy:debug] [pid 9794:tid 140072323774208] proxy_util.c(2131): AH00942: FCGI: has acquired connection for (*)
[proxy:debug] [pid 9794:tid 140072323774208] proxy_util.c(2184): [client 1.2.3.4:61135] AH00944: connecting fcgi://localhost/var/www/html/index.php to localhost:8000
[proxy:debug] [pid 9794:tid 140072323774208] proxy_util.c(2217): [client 1.2.3.4:61135] AH02545: fcgi: has determined UDS as /var/run/php5-fpm.sock
[proxy:debug] [pid 9794:tid 140072323774208] proxy_util.c(2385): [client 1.2.3.4:61135] AH00947: connected /var/www/html/index.php to httpd-UDS:0
[proxy_fcgi:trace4] [pid 9794:tid 140072323774208] util_script.c(522): [client 1.2.3.4:61135] Headers from script 'index.php':
[proxy_fcgi:trace4] [pid 9794:tid 140072323774208] util_script.c(523): [client 1.2.3.4:61135] X-Powered-By: PHP/5.6.4-4
[proxy_fcgi:trace4] [pid 9794:tid 140072323774208] util_script.c(523): [client 1.2.3.4:61135] Content-type: text/html; charset=UTF-8
[filter:trace4] [pid 9794:tid 140072323774208] mod_filter.c(169): [client 1.2.3.4:61135] Content-Type 'text/html; charset=UTF-8' ...
[filter:trace4] [pid 9794:tid 140072323774208] mod_filter.c(181): [client 1.2.3.4:61135] ... did not match 'application/xml'
[filter:trace2] [pid 9794:tid 140072323774208] mod_filter.c(188): [client 1.2.3.4:61135] Content-Type condition for 'deflate' did not match
[filter:trace4] [pid 9794:tid 140072323774208] mod_filter.c(169): [client 1.2.3.4:61135] Content-Type 'text/html; charset=UTF-8' ...
[filter:trace4] [pid 9794:tid 140072323774208] mod_filter.c(181): [client 1.2.3.4:61135] ... did not match 'application/rss+xml'
[filter:trace2] [pid 9794:tid 140072323774208] mod_filter.c(188): [client 1.2.3.4:61135] Content-Type condition for 'deflate' did not match
[filter:trace4] [pid 9794:tid 140072323774208] mod_filter.c(169): [client 1.2.3.4:61135] Content-Type 'text/html; charset=UTF-8' ...
[filter:trace4] [pid 9794:tid 140072323774208] mod_filter.c(181): [client 1.2.3.4:61135] ... did not match 'application/x-javascript'
[filter:trace4] [pid 9794:tid 140072323774208] mod_filter.c(181): [client 1.2.3.4:61135] ... did not match 'application/javascript'
[filter:trace4] [pid 9794:tid 140072323774208] mod_filter.c(181): [client 1.2.3.4:61135] ... did not match 'application/ecmascript'
[filter:trace2] [pid 9794:tid 140072323774208] mod_filter.c(188): [client 1.2.3.4:61135] Content-Type condition for 'deflate' did not match
[filter:trace4] [pid 9794:tid 140072323774208] mod_filter.c(169): [client 1.2.3.4:61135] Content-Type 'text/html; charset=UTF-8' ...
[filter:trace4] [pid 9794:tid 140072323774208] mod_filter.c(181): [client 1.2.3.4:61135] ... did not match 'text/css'
[filter:trace2] [pid 9794:tid 140072323774208] mod_filter.c(188): [client 1.2.3.4:61135] Content-Type condition for 'deflate' did not match
[filter:trace4] [pid 9794:tid 140072323774208] mod_filter.c(169): [client 1.2.3.4:61135] Content-Type 'text/html; charset=UTF-8' ...
[filter:trace4] [pid 9794:tid 140072323774208] mod_filter.c(175): [client 1.2.3.4:61135] ... matched 'text/html'
[filter:trace2] [pid 9794:tid 140072323774208] mod_filter.c(188): [client 1.2.3.4:61135] Content-Type condition for 'deflate' matched
[proxy:debug] [pid 9794:tid 140072323774208] proxy_util.c(2146): AH00943: FCGI: has released connection for (*)
[headers:trace2] [pid 9794:tid 140072323774208] mod_headers.c(874): AH01502: headers: ap_headers_output_filter()
[http:trace3] [pid 9794:tid 140072323774208] http_filters.c(1045): [client 1.2.3.4:61135] Response sent with status 200, headers:
[...]
[http:trace5] [pid 9794:tid 140072323774208] http_filters.c(1052): [client 1.2.3.4:61135] Date: Thu, 15 Jan 2015 23:53:47 GMT
[http:trace5] [pid 9794:tid 140072323774208] http_filters.c(1055): [client 1.2.3.4:61135] Server: Apache
[http:trace4] [pid 9794:tid 140072323774208] http_filters.c(874): [client 1.2.3.4:61135] X-Powered-By: PHP/5.6.4-4
[http:trace4] [pid 9794:tid 140072323774208] http_filters.c(874): [client 1.2.3.4:61135] X-Frame-Options: SAMEORIGIN
[http:trace4] [pid 9794:tid 140072323774208] http_filters.c(874): [client 1.2.3.4:61135] Vary: Accept-Encoding
[http:trace4] [pid 9794:tid 140072323774208] http_filters.c(874): [client 1.2.3.4:61135] X-XSS-Protection: 1; mode=block
[http:trace4] [pid 9794:tid 140072323774208] http_filters.c(874): [client 1.2.3.4:61135] X-Content-Security-Policy: allow 'self';
[http:trace4] [pid 9794:tid 140072323774208] http_filters.c(874): [client 1.2.3.4:61135] X-Frame-Options: DENY
[http:trace4] [pid 9794:tid 140072323774208] http_filters.c(874): [client 1.2.3.4:61135] Content-Length: 2
[http:trace4] [pid 9794:tid 140072323774208] http_filters.c(874): [client 1.2.3.4:61135] Keep-Alive: timeout=5, max=100
[http:trace4] [pid 9794:tid 140072323774208] http_filters.c(874): [client 1.2.3.4:61135] Connection: Keep-Alive
[http:trace4] [pid 9794:tid 140072323774208] http_filters.c(874): [client 1.2.3.4:61135] Content-Type: text/html; charset=UTF-8
As you can see, there is no difference between those two up to the line
AH00947: connected /var/www/html/index.php to httpd-UDS:0
.
Does anyone encountered a similar problem a knows a solution?
Futhermore I've data from tcpdump (while PHP5-FPM was listening on a TCP port instead of using a socket and I've data from sniffing the socket.
If those are needed please let me know - I didn't wanted to make the question too large.
.
Edit:
I've done some further research in hope to localize the problem.
Maybee this helps us further...
PHP-FPM Status page
strace on the worker process (chroot = off)
accept(0, {sa_family=AF_INET, sin_port=htons(50759), sin_addr=inet_addr("127.0.0.1")}, [16]) = 5
clock_gettime(CLOCK_MONOTONIC, {1397, 223489054}) = 0
times({tms_utime=0, tms_stime=0, tms_cutime=0, tms_cstime=0}) = 1718096640
poll([{fd=5, events=POLLIN}], 1, 5000) = 1 ([{fd=5, revents=POLLIN}])
read(5, "\1\1\0\1\0\10\0\0", 8) = 8
read(5, "\0\1\1\0\0\0\0\0", 8) = 8
read(5, "\1\4\0\1\3\341\0\0", 8) = 8
read(5, "\t\30UNIQUE_IDVLmGr38AAQEAAAVjkB4AAAAE\21\1proxy-nokeepalive1\t&HTTP_HOSTexample.com\17\nHTTP_CONNECTIO"..., 993) = 993
read(5, "\1\4\0\1\0\0\0\0", 8) = 8
lstat("/var/www/html/index.php", {st_mode=S_IFREG|0644, st_size=538, ...}) = 0
lstat("/var/www/html", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/var/www", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/var", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
.
strace on the worker process (chroot = on)
accept(0, {sa_family=AF_INET, sin_port=htons(50751), sin_addr=inet_addr("127.0.0.1")}, [16]) = 5
clock_gettime(CLOCK_MONOTONIC, {1208, 313176419}) = 0
times({tms_utime=0, tms_stime=0, tms_cutime=0, tms_cstime=0}) = 1718077750
poll([{fd=5, events=POLLIN}], 1, 5000) = 1 ([{fd=5, revents=POLLIN}])
read(5, "\1\1\0\1\0\10\0\0", 8) = 8
read(5, "\0\1\1\0\0\0\0\0", 8) = 8
read(5, "\1\4\0\1\3\341\0\0", 8) = 8
read(5, "\t\30UNIQUE_IDVLmF8n8AAQEAAAVjkB0AAAAS\21\1proxy-nokeepalive1\t&HTTP_HOSTexample.com\17\nHTTP_CONNECTIO"..., 993) = 993
read(5, "\1\4\0\1\0\0\0\0", 8) = 8
lstat("/var/www/html/index.php", 0x7fff98aa5d20) = -1 ENOENT (No such file or directory)
stat("/var/www/html", 0x7fff98aa8160) = -1 ENOENT (No such file or directory)
stat("/var/www", 0x7fff98aa8160) = -1 ENOENT (No such file or directory)
stat("/var", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat("", 0x7fff98aa8160) = -1 ENOENT (No such file or directory)
When you are using directive
<FilesMatch \.php$>
SetHandler "proxy:unix:/var/run/php5-fpm.sock|fcgi://localhost"
</FilesMatch>
script path sended to proxy is full path of file matched by FilesMatch, but this path does not exists in chroot.
Setting doc_root in php.ini solves this problem

Specific HTTP Redirect help - "Request exceeded the limit of 10 internal redirects"

There are many threads about this problem. But none has my specific issue.
Intro to our setup: we run Nginx on port 80, which sits in front of Apache on port 8889.
We haven't changed anything in Nginx or Apache, and this setup has been working for a while. Which is why we're scratching our head for what happened to suddenly cause this starting this morning. We are now getting 500 errors in our website.
From looking at the log, I suppose they're not caused by Nginx but by Apache:
[Fri Aug 01 23:08:33 2014] [error] [client 100.99.98.97] Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace., referer: http://somedomain.com/acordes/2011/01/entre-palos-y-alegrias/
[Fri Aug 01 23:08:33 2014] [debug] core.c(3112): [client 100.99.98.97] r->uri = /beta/actionize, referer: http://somedomain.com/acordes/2011/01/entre-palos-y-alegrias/
[Fri Aug 01 23:08:33 2014] [debug] core.c(3118): [client 100.99.98.97] redirected from r->uri = /beta/actionize, referer: http://somedomain.com/acordes/2011/01/entre-palos-y-alegrias/
[Fri Aug 01 23:08:33 2014] [debug] core.c(3118): [client 100.99.98.97] redirected from r->uri = /beta/actionize, referer: http://somedomain.com/acordes/2011/01/entre-palos-y-alegrias/
[Fri Aug 01 23:08:33 2014] [debug] core.c(3118): [client 100.99.98.97] redirected from r->uri = /beta/actionize, referer: http://somedomain.com/acordes/2011/01/entre-palos-y-alegrias/
[Fri Aug 01 23:08:33 2014] [debug] core.c(3118): [client 100.99.98.97] redirected from r->uri = /beta/actionize, referer: http://somedomain.com/acordes/2011/01/entre-palos-y-alegrias/
[Fri Aug 01 23:08:33 2014] [debug] core.c(3118): [client 100.99.98.97] redirected from r->uri = /beta/actionize, referer: http://somedomain.com/acordes/2011/01/entre-palos-y-alegrias/
[Fri Aug 01 23:08:33 2014] [debug] core.c(3118): [client 100.99.98.97] redirected from r->uri = /beta/actionize, referer: http://somedomain.com/acordes/2011/01/entre-palos-y-alegrias/
[Fri Aug 01 23:08:33 2014] [debug] core.c(3118): [client 100.99.98.97] redirected from r->uri = /beta/actionize, referer: http://somedomain.com/acordes/2011/01/entre-palos-y-alegrias/
[Fri Aug 01 23:08:33 2014] [debug] core.c(3118): [client 100.99.98.97] redirected from r->uri = /beta/actionize, referer: http://somedomain.com/acordes/2011/01/entre-palos-y-alegrias/
[Fri Aug 01 23:08:33 2014] [debug] core.c(3118): [client 100.99.98.97] redirected from r->uri = /beta/actionize, referer: http://somedomain.com/acordes/2011/01/entre-palos-y-alegrias/
[Fri Aug 01 23:08:33 2014] [debug] core.c(3118): [client 100.99.98.97] redirected from r->uri = /beta/actionize, referer: http://somedomain.com/acordes/2011/01/entre-palos-y-alegrias/
In the .htaccess file of this domain, we have the following code:
Options -Indexes +FollowSymLinks -MultiViews
DirectoryIndex index index.php index.htm index.html
DefaultType application/x-httpd-php
RewriteEngine On
# If someone types just the folder name
RewriteRule ^beta$ http://%{HTTP_HOST}/beta/index [L,R=301]
# If someone types the correct file, just show it
RewriteRule ^beta/(.*)$ /beta/$1 [L]
# All other URLs..
RewriteCond %{REQUEST_URI} !^(mailman|pipermail|w3c)/
RewriteRule ^(.*)$ /beta/get?u=$1 [L,QSA]
The logic is simple. For all files that are .htm etc in the root, we just show them. For any URL that starts with /beta/[xyz] we also show the file as it exists in the hard "beta" folder in the root folder.
But for all other URLs starting with our domain, we want to forward the code to /beta/get? with the parameter being that part of URI.
Example, if someone types: http://EXAMPLE.com/xyz123, we want to actually redirect internally to: http://EXAMPLE.com/beta/get?u=xyz123
(Except when the words there mailman or pipermail etc, which are used for our specific mailing lists)
Just FYI, the NGINX.CONF file is as below:
#-------------------- START ------------------#
http {
server_name_in_redirect off;
server_names_hash_max_size 10240;
server_names_hash_bucket_size 1024;
gzip on;
gzip_static on;
gzip_disable "MSIE [1-6]\.";
gzip_vary on;
gzip_proxied any;
gzip_comp_level 5; # Not a huge compression beyond this..so save CPU cycles
gzip_buffers 16 8k;
gzip_http_version 1.1;
gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript image/png image/gif image/jpeg;
connection_pool_size 256;
client_header_buffer_size 256k;
large_client_header_buffers 4 256k;
client_max_body_size 200M;
client_body_buffer_size 128k;
request_pool_size 64k;
output_buffers 16 256k;
open_file_cache max=5000 inactive=20s;
open_file_cache_valid 60s;
include mime.types;
default_type application/octet-stream;
error_log logs/error.log warn;
access_log off;
server_tokens off;
autoindex off;
sendfile off;
ignore_invalid_headers on; # Malware protection
reset_timedout_connection on; # DDoS protection
# Timeouts
client_header_timeout 5;
client_body_timeout 50;
send_timeout 50;
keepalive_timeout 65; # No need for default 65, but having some keepalive speeds things up
resolver_timeout 1s;
proxy_cache_path /dev/shm/proxy_cache levels=1:2 keys_zone=proxyone:200m inactive=1h max_size=2g;
proxy_cache_key "$scheme$host$request_uri$cookie___snippa$cookie___sniplang";
proxy_cache proxyone;
proxy_cache_min_uses 5;
proxy_cache_valid any 60s;
proxy_cache_valid 200 1m;
proxy_cache_valid 404 0s;
proxy_cache_valid 410 90d;
proxy_cache_valid 500 502 503 504 1s;
proxy_cache_valid 301 60m;
proxy_ignore_headers X-Accel-Expires Expires Cache-Control;
set_real_ip_from 192.168.1.0/24;
set_real_ip_from 192.168.2.1;
set_real_ip_from 127.0.0.1;
real_ip_header X-Real-IP;
server
{
listen 127.0.0.1:80;
listen 74.86.191.114:80;
listen 100.99.98.97:80;
listen 75.126.161.225:80;
listen 75.126.161.226:80;
listen 10.17.32.66:80;
server_name EXAMPLE.com www.EXAMPLE.com;
root /home/EXAMPLE/public_html;
index index.htm index.php index index.htm;
error_page 403 = #backend;
error_page 404 = #backend;
error_page 405 = #backend;
error_page 406 = #backend;
error_page 500 = #backend;
error_page 501 = #backend;
error_page 502 = #backend;
error_page 503 = #backend;
error_page 504 = #backend;
error_page 505 = #backend;
# IMPORTANT FILES
location = /\.htaccess { deny all; access_log off; log_not_found off; }
location ~ /\.ht { deny all; access_log off; log_not_found off; }
location = /favicon.ico { log_not_found off; access_log off; expires max; }
location = /robots.txt { allow all; log_not_found off; access_log off; expires max; }
location = /sitemap.gz { allow all; log_not_found off; access_log off; expires max; }
location = /crossdomain.xml { allow all; log_not_found off; access_log off; expires max; }
location / {
location ~.*\.(gif|jpg|png|ico|swf|rss|xml|htm|txt|js|css|gz|doc|xls|pdf|html|woff|eot|svg)$ {
expires max;
try_files $uri #backend;
log_not_found off;
}
proxy_pass http://100.99.98.97:8889;
include proxy.inc;
}
location #backend {
internal;
proxy_pass http://100.99.98.97:8889;
include proxy.inc;
}
} # End of EXAMPLE.COM server block
} # End of http block
What might be causing this? The loglevel debug in apache doesn't seem to be helping much. Much appreciate any advice or pointers!
You're rules are causing a loop. The rewrite engine re-applies all the rules over and over again until the URI stops changing. Try adding some more restirctions:
Options -Indexes +FollowSymLinks -MultiViews
DirectoryIndex index index.php index.htm index.html
DefaultType application/x-httpd-php
RewriteEngine On
# If someone types just the folder name
RewriteRule ^beta/?$ http://%{HTTP_HOST}/beta/index [L,R=301]
# If someone types the correct file, just show it
RewriteRule ^beta/(.+)$ /beta/$1 [L]
# All other URLs..
RewriteCond %{REQUEST_URI} !^(mailman|pipermail|w3c|beta)/
RewriteRule ^(.*)$ /beta/get?u=$1 [L,QSA]
Namely, an optional / at the end of just /beta, a + instead of a * to ensure there is at least one character after the /, and add beta to the conditions for the last rule.

gss_display_name() failed: A required input parameter could not be read: An invalid name was supplied (, Unknown error)

I'm trying to setup Kerberos authentication on Apache 2.2.15-30 (CentOs 6.5), and am facing an issue that I'm not able to debug or solve. I can see the TGS request in the KDC log and Firefox sends the correct Authorization: Negotiate header, but something in Apache goes wrong and I'm getting an HTTP 500.
krb5kdc.log
Jul 02 20:59:03 infa.domain.local krb5kdc[1847](info): TGS_REQ (6 etypes {18 17 16 23 25 26}) 192.168.218.201: ISSUE: authtime 1404320175, etypes {rep=23 tkt=23 ses=23}, Administrator#DOMAIN.LOCAL for HTTP/infa.domain.local#DOMAIN.LOCAL
Apache error_log
[Wed Jul 02 20:59:01 2014] [debug] src/mod_auth_kerb.c(1940): [client 192.168.218.1] kerb_authenticate_user entered with user (NULL) and auth_type Kerberos
[Wed Jul 02 20:59:03 2014] [debug] src/mod_auth_kerb.c(1940): [client 192.168.218.1] kerb_authenticate_user entered with user (NULL) and auth_type Kerberos
[Wed Jul 02 20:59:03 2014] [debug] src/mod_auth_kerb.c(1279): [client 192.168.218.1] Acquiring creds for HTTP/infa.domain.local
[Wed Jul 02 20:59:03 2014] [debug] src/mod_auth_kerb.c(1692): [client 192.168.218.1] Verifying client data using KRB5 GSS-API
[Wed Jul 02 20:59:03 2014] [debug] src/mod_auth_kerb.c(1708): [client 192.168.218.1] Client didn't delegate us their credential
[Wed Jul 02 20:59:03 2014] [debug] src/mod_auth_kerb.c(1727): [client 192.168.218.1] GSS-API token of length 941 bytes will be sent back
[Wed Jul 02 20:59:03 2014] [debug] src/mod_auth_kerb.c(1139): [client 192.168.218.1] GSS-API major_status:01020000, minor_status:00000000
[Wed Jul 02 20:59:03 2014] [error] [client 192.168.218.1] gss_display_name() failed: A required input parameter could not be read: An invalid name was supplied (, Unknown error)
HTTP dump
GET http://infa.domain.local/server-status HTTP/1.1
Host: infa.domain.local
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:30.0) Gecko/20100101 Firefox/30.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Cache-Control: max-age=0
HTTP/1.1 401 Authorization Required
Date: Wed, 02 Jul 2014 19:32:39 GMT
Server: Apache/2.2.15 (CentOS)
WWW-Authenticate: Negotiate
Content-Length: 484
Connection: close
Content-Type: text/html; charset=iso-8859-1
Proxy-Support: Session-Based-Authentication
GET http://infa.domain.local/server-status HTTP/1.1
Host: infa.domain.local
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:30.0) Gecko/20100101 Firefox/30.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Cache-Control: max-age=0
Authorization: Negotiate YIID5QYGKwYBBQUCoIID2TCCA9WgCjAIBgYrBgEFAgWiggPFBIIDwWCCA70GBisGAQUCBQUBMBChDgQMRE9NQUlOLkxPQ0FMbIIDnTCCA5mhAwIBBaIDAgEMo4IDFzCCAxMwggIvoQMCAQGiggImBIICIm6CAh4wggIaoAMCAQ WhAwIBDqIHAwUAAAAAAKOCAU9hggFLMIIBR6ADAgEFoQ4bDERPTUFJTi5MT0NBTKIhMB+gAwIBAqEYMBYbBmtyYnRndBsMRE9NQUlOLkxPQ0FMo4IBCzCCAQegAwIBEqEDAgEBooH6BIH3jDiOe80e8vCv7Tmsd+t0spncJWnD v99vLDpi5PYc1Gj8vGH7xJxnz4dsr6WavFLmgYCRnvrF+Y+lU/QVF/AUNiqIG7ifGAJGD4IKHzcyYfNo9BLlNBGBckLdIhC3o2G8VfHWxv+Zo6DNfZUJsIVfoN2bls2C8K9K2pv/qd/FHR96+3JpCkRSb2tKqh2VQBA2mplvJML38nvHQkp5Y0rHQ ecbc0bHns1ddh/RLIlPcwdy8r7xDx7m5QUWH3gI6nSEhrcd/sIKoRJ88ezcMfqumXq2UxvBdBJAH86q9r9r/t74jXpyDFlRgF/Z6OLMwMdus2AkBNrbiaSBsTCBrqADAgEXooGmBIGj4DUpIRQjvddUEpp7sft5UjlnOPOCia9BSyxYBszOihLHr2D 2B6mL6fmqx7IcAVfVzV66B/gqQ4roAh0z4YKensKtqIAG7au2RsXtYNAjEgUFgh7dEE7kACUFoVB2VUK2mtjuHabbwMZ4gprrRIgDeFqROIhxWasVgxhak6dXQAKGEyvVlGoeLTJTPER5s2tcDRkoVTLFO0hBJxarNI/GTk1e1jCB3aEEAgIAi KKB1ASB0aCBzjCBy6EcMBqgBAIC/3ahEgQQSjwHqwdg2yuvh3nbGzDVuqKBqjCBp6ADAgEXooGfBIGccNThLwiDzyz8cJYPfI6hU505ydEQdRt6N036ZZ98Y49YfV+WWpCgXxhmL/8zhilAC2mQi5cvE5XOJOzGrWHnzl6AO1KfJQKjvogV zrFhdoPMVssGnBkrD40fsIA2uPJ2e0OeKRC/tOizUg8tVIdhkoivnh69Q1BDAx3JFjx3txRtDoSZHz6x4mlBSs72xFIlIkA7yhXH+nmml4yfpHIwcKAHAwUAUIEAAKIOGwxET01BSU4uTE9DQUyjJDAioAMCAQOhGzAZGwRIVFRQGxFpbmZh LmRvbWFpbi5sb2NhbKURGA8yMDE0MDcwMzAyNTYxNlqnBgIEU7ReW6gUMBICARICARECARACARcCARkCARo=
HTTP/1.1 500 Internal Server Error
Date: Wed, 02 Jul 2014 19:32:42 GMT
Server: Apache/2.2.15 (CentOS)
WWW-Authenticate: Negotiate oYIDqTCCA6WgAwoBAaEIBgYrBgEFAgWiggOSBIIDjgUBMBChDgQMRE9NQUlOLkxPQ0FMbYIDdjCCA3KgAwIBBaEDAgENooHVMIHSMIHPoQQCAgCIooHGBIHDoIHAMIG9oIG6MIG3oAMCAReiga8Egawhq77nnFYKOC2elIoQEMv 3HoPncmPLVp6/yr+HtLIuoyAsAUdbvyXars5ixGdPlg1IaceQQ3ThVvvsRthV86O4M2l55LfhlfIINZr7xQks3EKTAEA1OfsggBXdmShHV/29W2iLaQP60BvBlYCOGePMyMKp8jcgdNUQ6jLqq6No0Qk7Kro8IIjESMmVR3BAndbUfpDNYqO+IxY am/pl96xCQgu4iNznoglrYBf7ow4bDERPTUFJTi5MT0NBTKQaMBigAwIBAaERMA8bDUFkbWluaXN0cmF0b3KlggFjYYIBXzCCAVugAwIBBaEOGwxET01BSU4uTE9DQUyiJDAioAMCAQOhGzAZGwRIVFRQGxFpbmZhLmRvbWFpbi5sb 2NhbKOCARwwggEYoAMCARehAwIBAaKCAQoEggEGyeo+gzn7hHLgwIGfZiT3kfiua+yD1d0EDhyoAmctFzukkw7xqdyMZn+gfDna6O0WI7TC6Yv2pQqg1Ph76SZ11ZQu4xXn4FBPu3G9LwbPUxN9+cohhCTPmAX6SLyNu7n9UAKLsccjb kLq8HJjUgzfLus6AqUeerqjc3eSyr+r1onfQSL9JCNtpOUWtuxGIThTQfOXEYVlVyjMi37bnAFPMrxPERL/7m3vYm3x60HBu5KHy7xfbab8jftIsr33Z/2nnMxNi5LjqVBail4BpZiuRCMmko566KSLKWRSpvr6x/YUR5TPmhXjO3YGdi2VucDn6QW t81q2dQSYvAQnbuHDL84IQUY126aB+jCB96ADAgEXooHvBIHsmFwxE55S5Gi5VkPG0cS11MHsQvllqJAIxGMkzakyyYCfMKCpHFfyIf/2bIGPvSyCCWOqFxnMOA1a/c2d3eUk6Yr+H5c8PDFePxVbKijvZRVRVJ1pAifpm9kUoKcGMo0SH 9m0H4yu94/ESE7QbEcx7pQac1Udq894rgF7OmnQXZZ6mX2VUrIb0xHxaaj9oR8+zC8vGWyyqVSZhtURxQ8Anr+MifqWKPP2QpWFohptl/zl8bYmMqs1nEH3TIe1wvtOgeqGh6KumbC4rc9IVCN8rx+3XCVr/2BM27nURT21MUzwU1tbpQM LSqT0gFE=
Content-Length: 617
Connection: close
Content-Type: text/html; charset=iso-8859-1
kdc.conf
[kdcdefaults]
kdc_ports = 88
kdc_tcp_ports = 88
[realms]
DOMAIN.LOCAL = {
#master_key_type = aes256-cts
acl_file = /var/kerberos/krb5kdc/kadm5.acl
dict_file = /usr/share/dict/words
admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
forwardable = true
proxiable = true
supported_enctypes = rc4-hmac:normal
}
/etc/krb5.conf
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
[libdefaults]
default_realm = DOMAIN.LOCAL
dns_lookup_realm = false
dns_lookup_kdc = false
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true
proxiable = true
default_tgs_enctypes = rc4-hmac
default_tkt_enctypes = rc4-hmac
[realms]
DOMAIN.LOCAL = {
kdc = infa.domain.local:88
admin_server = infa.domain.local:749
}
[domain_realm]
.domain.local = DOMAIN.LOCAL
domain.local = DOMAIN.LOCAL
auth_kerb.conf
LoadModule auth_kerb_module modules/mod_auth_kerb.so
<Location /server-status>
#SSLRequireSSL
AuthType Kerberos
AuthName "Kerberos Login"
KrbMethodNegotiate On
KrbMethodK5Passwd Off
KrbAuthRealms DOMAIN.LOCAL
Krb5KeyTab /etc/httpd/conf/http.keytab
KrbServiceName HTTP/infa.domain.local
require valid-user
</Location>
klist -e -k /etc/httpd/conf/http.keytabb
Keytab name: FILE:/etc/httpd/conf/http.keytab
KVNO Principal
---- --------------------------------------------------------------------------
0 HTTP/infa.domain.local#DOMAIN.LOCAL (arcfour-hmac)
Does anyone have an idea of what the problem might be? I'd be very thankful for any comments.
Thank you,
Martin
I got this exactmessage when the client's clock skew was too great. Setting the clock (and enabling ntp :-) got it working.