Solr 'service unaveliable' - apache

So, i've been following 'djnaog by example' and one of the assignments is to integrate django and solr with haystack. I tried to follow the readme included in solr package, but idk why, it doestn seem to work when i visit localhost 8983. At http://127.0.0.1:9999/solr/ i get message 'Service Unavailable', in terminal i get :
"* [WARN] Your open file limit is currently 1024. It should
be set to 65000 to avoid operational disruption. If you no longer
wish to see this warning, set SOLR_ULIMIT_CHECKS to false in your
profile or solr.in.sh
[WARN] * Your Max Processes Limit is currently 62864. It should be set to 65000 to avoid operational disruption. If you no
longer wish to see this warning, set SOLR_ULIMIT_CHECKS to false in
your profile or solr.in.sh Waiting up to 180 seconds to see Solr
running on port 9999 [|] Started Solr server on port 9999
(pid=11255). Happy searching!"
I have no idea what to do so please help
Let me add that i used
./bin/solr start
in bash
Update:
Tried:
./solr status commnad
Output:
"Found 4 Solr nodes: Solr process 11255 from
/home/pawe/Pulpit/solr-8.1.1/solr/bin/solr-9999.pid not found.
Solr process 10702 from
/home/pawe/Pulpit/solr-8.1.1/solr/bin/solr-8888.pid not found.
Solr process 3174 running on port 8983
Error: Could not find or load main class org.apache.solr.util.SolrCLI
Caused by: java.lang.ClassNotFoundException:
org.apache.solr.util.SolrCLI Solr process 11124 from
/home/pawe/Pulpit/solr-8.1.1/solr/bin/solr-8866.pid not found. "

Related

Apache/PHP7.3 running in Docker randomly drops connection with empty response

I have found several similar questions:
APACHE, PHP Server return randomly empty response
https://serverfault.com/questions/66662/apache-gives-empty-reply
and others
However these does not seem to help to find the cause. I can replicate the behaviour when reloading a specific page ~20 times.
Running current apache2 (= 2.4.38-3+deb10u4). I tried to disable opcache, remove MaxRequestsPerChild with no effect.
Apache log does not show any error. The request is not even logged.
The USE_ZEND_ALLOC=0 seem to have no effect and the problem persists.
I tried to install mod_forensic which shows that the request came in. No error or finished request is then logged.
The container is running in Kubernetes and I cannot replicate the issue locally running directly with Docker machine, that is why I think this might be caused by some memory setting. However I couldn't find what might be causing this as there is no single error message.
Can you think of any reason why this might be happening?
Edit1:
I tried to set log level to trace:
https://gist.github.com/knyttl/861e8a0fe5651408df37cd5c3874946b
The request is handled and then you can see:
[Tue Oct 20 08:37:55.825454 2020] [core:trace4] [pid 1] mpm_common.c(536): mpm child 388 (gen 2/slot 4) exited
With no error and no response.
Edit2:
I updated to php7.4 and the issue persists.
I finally found it:
The process is silently killed by OOM killer of the host machine:
[4019392.626796] Memory cgroup out of memory: Kill process 4178127 (apache2) score 1137 or sacrifice child
[4019392.636520] Killed process 4178127 (apache2) total-vm:143960kB, anon-rss:22856kB, file-rss:10472kB, shmem-rss:28228kB
This is never logged within the container so it was hard to find.
Why don't you use Jorge's answers ?
Finally solved by adding to /etc/apache2/envvars:
export USE_ZEND_ALLOC=0
https://serverfault.com/a/66759

Apache crashes with Parent: child process exited with status 3221226356

Apache crashes and I get the following error in the apache log:
AH00428: Parent: child process exited with status 3221226356 -- Restarting.
Backtrace:
Count: 2
Exception #: 0XC0000008
Stack:
ntdll!KiRaiseUserExceptionDispatcher+0x3a
KERNELBASE!CloseHandle+0x1b
libapr_1!apr_shm_size_get+0x27d
libapr_1!apr_shm_destroy+0x12
mod_socache_shmcb+0x161c
mod_ssl!ssl_run_proxy_post_handshake+0x8d12
mod_ssl!ssl_run_pre_handshake+0x3d97
libapr_1!apr_pool_clear+0x6e
httpd!OPENSSL_Applink+0xcef
httpd!OPENSSL_Applink+0x1f98
KERNEL32!BaseThreadInitThunk+0x22
ntdll!RtlUserThreadStart+0x34
Seems to have to do with mod_ssl. I do have a http proxy set up in a https virtual host. However, the same error appears in the Apache log even if I comment out the proxy portion of the config file. This error also tends to be followed by a couple of MySQL errors in the Windows application log complaining about a lost connection. I think that might just be because Apache crashed while the connections were open. I am using the latest versions of Apache 2.4.20, PHP 7.0.8, and MySQL 5.7. Any ideas? Thanks!

Liferay 6.2 clustering issue with multicast

I am trying to cluster ehcache and lucene with Liferay 6.2 EE sp2 bundle on 2 servers with mutlicast enabled. WE have Apache HTTPD servers fronting tomcat servers using reverse proxy. A valid 6.2 license is deployed on both the nodes.
We user the following properties in the portal-ext.properties:
cluster.link.enabled=true
lucene.replicate.write=true
ehcache.cluster.link.replication.enabled=true
# Since we are using SSL on the frontend
web.server.protocol=https
# set this to any server that is visible to both the nodes
cluster.link.autodetect.address=dbserverip:dbport
#ports and ips we know work in our environment for multicast
multicast.group.address["cluster-link-control"]=ip
multicast.group.port["cluster-link-control"]=port1
multicast.group.address["cluster-link-udp"]=ip
multicast.group.port["cluster-link-udp"]=port2
multicast.group.address["cluster-link-mping"]=ip
multicast.group.port["cluster-link-mping"]=port3
multicast.group.address["hibernate"]=ip
multicast.group.port["hibernate"]=port4
multicast.group.address["multi-vm"]=ip
multicast.group.port["multi-vm"]=port5
We are running into issues with the ehcache and lucene clustering not working. The following tests fail :
Moving a portlet on node 1, does not show up on node 2
There are no errors except for a startup error with lucene.
14:19:35,771 ERROR
[CLUSTER_EXECUTOR_CALLBACK_THREAD_POOL-1][LuceneHelperImpl:1186]
Unable to load index for company 10157
com.liferay.portal.kernel.exception.SystemException:
java.net.ConnectException: Connection refused at
com.liferay.portal.search.lucene.LuceneHelperImpl.getLoadIndexesInputStreamFromCluster(LuceneHelperImpl.java:488)
at
com.liferay.portal.search.lucene.LuceneHelperImpl$LoadIndexClusterResponseCallback.callback(LuceneHelperImpl.java:1176)
at
com.liferay.portal.cluster.ClusterExecutorImpl$ClusterResponseCallbackJob.run(ClusterExecutorImpl.java:614)
at
com.liferay.portal.kernel.concurrent.ThreadPoolExecutor$WorkerTask._runTask(ThreadPoolExecutor.java:682)
at
com.liferay.portal.kernel.concurrent.ThreadPoolExecutor$WorkerTask.run(ThreadPoolExecutor.java:593)
at java.lang.Thread.run(Thread.java:745) Caused by:
java.net.ConnectException: Connection refused at
java.net.PlainSocketImpl.socketConnect(Native Method) at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at
java.net.Socket.connect(Socket.java:579) at
sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:625) at
sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:160)
at sun.net.NetworkClient.doConnect(NetworkClient.java:180) at
sun.net.www.http.HttpClient.openServer(HttpClient.java:432) at
sun.net.www.http.HttpClient.openServer(HttpClient.java:527) at
sun.net.www.protocol.https.HttpsClient.(HttpsClient.java:275)
at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:371)
We verified that the jgroups multicast works outside of liferay by running the following commands and using a downloaded copy of the jgroups.jar and replacing with the 5 multicast ips and ports.
Testing with JGROUPS
1) McastReceiver -
java -cp ./jgroups.jar org.jgroups.tests.McastReceiverTest -mcast_addr 224.10.10.10 -port 5555
ex. java -cp jgroups-final.jar org.jgroups.tests.McastReceiverTest -mcast_addr 224.10.10.10 -port 5555
2) McastSender -
java -cp ./jgroups.jar org.jgroups.tests.McastSenderTest -mcast_addr 224.10.10.10 -port 5555
ex. java -cp jgroups-final.jar org.jgroups.tests.McastSenderTest -mcast_addr 224.10.10.10 -port 5555
From there, typing things into the McastSender will result in the Receiver printing it out.
Thanks!
After a lot of troubleshooting and help from various folks in my team and at liferay support, we switched to using unicast and it worked a lot better.
Here is what we did:
Extracted jgroups.jar from the tomcat home/webappts/ROOT/WEB_INF/lib, saved locally.
Unzipped the jgroups.jar file and extracted and save the tcp.xml from the jar's WEB_INF folder
As a base line test, changed the section in the tcp.xml and saved
TCPPING timeout="3000"
initial_hosts="${jgroups.tcpping.initial_hosts:servername1[7800],servername2[7800]}"
port_range="1"
num_initial_members="10"
Copy the tcp.xml to the liferay home on both the nodes
Change the portal-ext.properties to remove the mutlicast properties and add the following lines.
cluster.link.channel.properties.control=${liferay.home}/tcp.xml
cluster.link.channel.properties.transport.0=${liferay.home}/tcp.xml
Start node 1
start node 2
check logs
Do the cluster cache test:
Moving a portlet on node 1, shows up on node 2
Under control panel -> License manager both the nodes show up with valid licenses.
searching for user on node 2 after adding in node 1 in control panel -> user and organizations.
All of the above tests worked.
So we shutdown servers and changed the tcp.xml to use jdbc rather than the tcpping so we don't have to specify node names manually.
Step for the jdbc config:
Create the table in the liferay database manually.
CREATE TABLE JGROUPSPING (own_addr varchar(200) not null, cluster_name varchar(200) not null, ping_data blob default null, primary key (own_addr, cluster_name))
change tcp.xml and remove the tcpping section and add the following.
Note: Please replace the leading \ with less than symbol in the following code block. There are issues with the leading less than sign in the SO editor/parser hiding whatever comes after it:
\JDBC_PING datasource_jndi_name="java:comp/env/jdbc/LiferayPool"
initialize_sql="" />
Save and push the file manually to both the nodes.
Start the servers and repeat tests above.
It should work seamlessly.
It was invaluable to have the debug logging on for jgroups mentioned in the following the post:
https://bitsofinfo.wordpress.com/2014/05/21/clustering-liferay-globally-across-data-centers-gslb-with-jgroups-and-relay2/
tomcat home/webapps/ROOT/WEB-INF/classes/META-INF/portal-log4j-ext.xml file I used to triage various issues on bootup related to clustering.
<?xml version="1.0"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
<category name="com.liferay.portal.cluster">
<priority value="TRACE" />
</category>
<category name="com.liferay.portal.license">
<priority value="TRACE" />
</category>
We also found that the Lucene cluster replication startup errors were fixed in a fix pack and are getting a patch for it.
https://issues.liferay.com/browse/LPS-51714
https://issues.liferay.com/browse/LPS-51428
We added the following portal instance properties for lucene replication to work better between the 2 nodes:
portal.instance.http.port=port that the app servers listen on ex. 8080
portal.instance.protocol=http
Hope this helps someone.
Update
The lucene index load in a cluster issue was resolved by a Liferay 6.2 EE patch from support for the LPS's mentioned above.

Magento Soap Error - Premature end of data in tag definitions line 2

My client is using Unleashedsoftware.com to connect to a Magento Store. But it gives this error.
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Body>
<SOAP-ENV:Fault>
<faultcode>WSDL</faultcode>
<faultstring>
SOAP-ERROR: Parsing WSDL: Couldn't load from 'http://www.domain.com/index.php/api/v2_soap/index/wsdl/1/' : Premature end of data in tag definitions line 2
</faultstring>
</SOAP-ENV:Fault>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
When browsing http://www.domain.com/index.php/api/v2_soap/index/ Firebug gives me “500 Internal Service Error”.
When I browse http://www.domain.com/index.php/api/v2_soap/index/wsdl/1/, I am getting valid XML data.
I checked the server log files and it seems like:
[Thu Aug 30 22:22:25 2012] [warn] [client 92.92.92.92] mod_fcgid: stderr: in /home/doaminuser/public_html/lib/Zend/Soap/Server.php on line 762
I been searching for couple of days now and today I tried to duplicate the entire site to another test server, and it seems to be working! So that seems to be a server issue.
Please, anybody got any idea what could be the issue?
Is there any better way of debugging this issue, any sample code or debugging tips.
Magento version is 1.6.2
Thank you.
There's lots of times where Magento's SOAP API fails due to problems your Magento server has communicating with itself.
That is, PHP's SOAP implementation requires that the SOAP server itself fetch the WSDL file via http, and a local network configuration issue gets in the way of Magento fetching it's own WSDL.
You can debug this by SSHing into your Magento server, and running the following command
curl -l 'http://www.example.com/index.php/api/v2_soap/index/wsdl/1/' > /tmp/wsdl.xml
and then examining the wsdl.xml file. Because you're performing this from your web-server, you may get different results than when you're performing it from your local browser.
I had a similar problem when calling the URL
http://www.store.com/index.php/api/v2_soap/?wsdl
After some time I received the message 500 - Internal Server Error and a Premature end of script headers message in the apache error log.
After a whole day of research I figured out, that the Timeout-Directive of the Apache module (configured in httpd.conf on a Linux environment) was set to "20" which caused the server to send the 500 error after 20 seconds. The problem is, that in my case the Magento system needs a longer time to "crawl" through all wsdl.xml files in order to build the WSDL-output (if you are using Magento SOAPv2).
Maybe you should check your Timeout Directive..hope that helps.
"I have memories of this. What worked for me was to put the hostname
in /etc/hosts on the server plus the www alias on 127.0.0.1 However,
in this instance the server was in the building rather than in some
ISP place and the LAN had Windows computers on it. Windows users had
downloaded lots of trojan-virus-porn things that were spending the
whole time spamming the network so the real problem was with the
Windows computers on the network, not with the server or with Magento.
After fdisking the PC's the problem was solved."
Thank You I've been struggling for 2 days with this on magento 1.6 and Windows Server 2008 adding this line to the hosts file (C:\Windows\System32\drivers\etc) solved the issue for me:
127.0.0.1 www.Domain.com
also remember to fix your magento soap (role) because the Roles Resources doesn't save in 1.6 unless you fix this file:
MagentoRoot\app\code\core\Mage\Adminhtml\Block\Api\Tab\Rolesedit.php
replace this:
if (array_key_exists(strtolower($item->getResource_id()), $resources) && $item->getPermission() == 'allow') {
with this:
if (array_key_exists(strtolower($item->getResource_id()), $resources) && $item->getApiPermission() == 'allow') {
In my case the issue was the Mod_Security rule "PHP Easter Egg Access" was enabled.
Rule ID: 380800
Once disabled, the api access worked.
An indicator was in the Apache log file:
Jun 19 09:15:52 httpd[1024961]: [error] [client xyz.xyz.xyz.xyz] ModSecurity: [file "/usr/local/apache/conf/modsec/99_asl_jitp.conf"] [line "116"] [id "380800"] [rev "1"] [msg "Atomicorp.com WAF Rules - Virtual Just In Time Patch: PHP Easter Egg Access"] [data "phpe9568f35-d428-11d2-a769-00aa001acf42"] [severity "CRITICAL"] Access denied with code 403 (phase 2). Pattern match "php(?:e9568f3[56]-d428-11d2-a769-00aa001acf42|b8b5f2a0-3c92-11d3-a3a9-4c7b08c10000)" at REQUEST_URI. [hostname "www.yoursever.com"]...
Magento version: 1.7.0.2
PHP version: 5.3.26
More information about the PHP Easter Egg Access rule:
http://www.atomicorp.com/forums/viewtopic.php?f=3&t=5057
http://www.0php.com/php_easter_egg.php
For those wanting a quick test script to replicate the issue (useful when trying to convince your hosting provider that it's a problem on their end), use:
<?php
$server = new SoapServer("http://<url to your magento shop>/index.php/api/v2_soap/index/wsdl/1/");
?>
This is the line in /lib/Zend/Soap/Server.php that triggers the error.
In my case if you browsed to:
http://< url to your magento shop >/index.php/api/v2_soap/index/wsdl/1/
the xml was fine, but if you ran the above php script on the server, the error was given.
This error most often appeared for me while omitting www for domain given in Magento SOAP url. Url has to match base url specified in the Magento config.

configure Zend server CE if a machine doesn't support ipv6

I have installed latest Zend Server CE in my notebook (it has Win 7 installed) and then
I installed it on my desktop machine. But when you start server controller there is an alert saying Connection refused (or timed out)
What's the problem?
When I start server http://localhost:10081/ZendServer/
the following message appears
Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, admin#example.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
and apache works well in the following address http://localhost:81
when i telnet localhost 10081 there is a reply !!!
I found from the inet http://forums.zend.com/viewtopic.php?f=8&t=7518 that it's because of not having ipv6 support, I am running windows xp3 on my desktop
how to disable default ipv6 support in Zend conf?
how to fix this problem or should I switch to win 7 (my desktop)?
Thanks in advance!
I had a similar problem, on Debian Squeeze after an upgrade to the currently latest version of Zend from the zend apt repo, [deb http://repos.zend.com/zend-server/deb server non-free] . The Zend gui fails.
zend-server-ce-php-5.3 5.5.0+b63
Log in /usr/local/zend/gui/lighttpd/logs/php.log throws this exception.
[10-Jan-2012 11:20:03] PHP Fatal error: Uncaught exception 'Exception' with message 'Unknown application version' in /usr/local/zend/gui/application/CheckDependencies.php:28
Stack trace:
#0 /usr/local/zend/gui/html/index.php(16): CheckDependencies::getChecker('INSTALLATION_PL...')
#1 {main}
thrown in /usr/local/zend/gui/application/CheckDependencies.php on line 28
The problem is the zend-server.ini file in /usr/local/zend/gui/application/data/zend-server.ini But after doing the changes shown below it seems to work ok again.
--- zend-server.ini 2012-01-05 16:45:29.000000000 +0100
+++ zend-server.ini 2012-01-10 12:01:37.000000000 +0100
## -3,7 +3,7 ##
[zendServer]
version = 5.6.0
-edition = INSTALLATION_PLACEHOLDER_GUI_EDITION
+edition = CE
devEnv=0
This is Zend's default error message, meaning you have some error in the php script you are running. By default, your php.ini file in Zend is configured to report errors, but not specifically state what errors had occurred, which makes this case hard to diagnose. My recommendation is that if you are using this server for debugging purposes, go to your Zend php.ini file (typically C:\Program Files\Zend\ZendServer\etc) and search for "display_errors" and change its default value from "Off" to "On". This will allow you to see the specific error PHP is encountering rather than getting the generic Zend error message.