Apache2 crashes and there's no logging in the error.log, not even after setting the Loglevel to debug in the apache2.conf.
Owner of the error.log is root
No updates or changes were made on the Server
Tried to downgrade from Apache (apt-get install --reinstall apache2=2.4.18-2ubuntu3 apache2-bin=2.4.18-2ubuntu3 apache2-data=2.4.18-2ubuntu3) but that doesn't help.
service apache2 start:
Job for apache2.service failed because the control process exited with error code. See "systemctl status apache2.service" and "journalctl -xe" for details.
systemctl status apache2.service:
apache2.service - LSB: Apache2 web server
Loaded: loaded (/etc/init.d/apache2; bad; vendor preset: enabled)
Drop-In: /lib/systemd/system/apache2.service.d
└─apache2-systemd.conf
Active: failed (Result: exit-code) since Tue 2017-04-04 16:06:03 UTC; 1min 26s ago
Docs: man:systemd-sysv-generator(8)
Process: 11071 ExecStart=/etc/init.d/apache2 start (code=exited, status=1/FAILURE)
Apr 04 16:05:43 bpi-iot-ros systemd[1]: Starting LSB: Apache2 web server...
Apr 04 16:05:43 bpi-iot-ros apache2[11071]: * Starting Apache httpd web server apache2
Apr 04 16:06:03 bpi-iot-ros apache2[11071]: *
Apr 04 16:06:03 bpi-iot-ros apache2[11071]: * The apache2 instance did not start within 20 seconds. Please read the log files to discover problems
Apr 04 16:06:03 bpi-iot-ros systemd[1]: apache2.service: Control process exited, code=exited status=1
Apr 04 16:06:03 bpi-iot-ros systemd[1]: Failed to start LSB: Apache2 web server.
Apr 04 16:06:03 bpi-iot-ros systemd[1]: apache2.service: Unit entered failed state.
Apr 04 16:06:03 bpi-iot-ros systemd[1]: apache2.service: Failed with result 'exit-code'.
● service.service
Loaded: not-found (Reason: No such file or directory)
Active: inactive (dead)
journalctl -xe:
Apr 04 16:09:22 bpi-iot-ros apache2[11191]: *
Apr 04 16:09:22 bpi-iot-ros apache2[11191]: * The apache2 instance did not start within 20 seconds. Please read the log files to discover problems
Apr 04 16:09:22 bpi-iot-ros systemd[1]: apache2.service: Control process exited, code=exited status=1
Apr 04 16:09:22 bpi-iot-ros systemd[1]: Failed to start LSB: Apache2 web server.
-- Subject: Unit apache2.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit apache2.service has failed.
--
-- The result is failed.
Apr 04 16:09:22 bpi-iot-ros systemd[1]: apache2.service: Unit entered failed state.
Apr 04 16:09:22 bpi-iot-ros systemd[1]: apache2.service: Failed with result 'exit-code'.
Error log, last lines from Monday:
[Mon Apr 03 08:56:07.530514 2017] [:error] [pid 30594] [client 192.168.2.185:52276] PHP Fatal error: Uncaught Doctrine\DBAL\DBALException: Failed to connect to the database: An exception occured in driver: SQLSTATE[HY000] [2002] No such file or directory in /var/www/nextcloud/lib/private/DB/Connection.php:60\nStack trace:\n#0 /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(429): OC\DB\Connection->connect()\n#1 /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(389): Doctrine\DBAL\Connection->getDatabasePlatformVersion()\n#2 /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(328): Doctrine\DBAL\Connection->detectDatabasePlatform()\n#3 /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(621): Doctrine\DBAL\Connection->getDatabasePlatform()\n#4 /var/www/nextcloud/lib/private/DB/Connection.php(147): Doctrine\DBAL\Connection->setTransactionIsolation(2)\n#5 /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DriverManager.php(172): OC\DB\Connection->__construct(Array, Object(Doctrine\DBAL\Driver\PDOMySql\Driver), Obj in /var/www/nextcloud/lib/private/DB/Connection.php on line 60
[Mon Apr 03 08:56:12.034222 2017] [mpm_prefork:notice] [pid 625] AH00169: caught SIGTERM, shutting down
Edit:
apache2ctl configtest no Syntax errors.
apache2ctl -X
Child Process seems to hang up, can reach the SSL Warning (selfsigned) when attempting to reach the sitestarting
did a strace of Apache starting, some Childprocess exited with Errorcode 1 (huge File so i only copied the last Lines)
11028 open("/lib/arm-linux-gnueabihf/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
11028 read(3, "\177ELF\1\1\1\3\0\0\0\0\0\0\0\0\3\0(\0\1\0\0\0\331i\1\0004\0\0\0"..., 512) = 512
11028 lseek(3, 894128, SEEK_SET) = 894128
11028 read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 2960) = 2960
11028 lseek(3, 888324, SEEK_SET) = 888324
11028 read(3, "A2\0\0\0aeabi\0\1(\0\0\0\0057-A\0\6\n\7A\10\1\t\2\n\3\f"..., 51) = 51
11028 fstat64(3, {st_mode=S_IFREG|0755, st_size=897088, ...}) = 0
11028 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb6f72000
11028 mmap2(NULL, 963928, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb6e3e000
11028 mprotect(0xb6f14000, 65536, PROT_NONE) = 0
11028 mmap2(0xb6f24000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xd6000) = 0xb6f24000
11028 mmap2(0xb6f27000, 9560, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb6f27000
11028 close(3) = 0
11028 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb6f71000
11028 set_tls(0xb6f714c0, 0xb6f71b98, 0xb6f7c050, 0xb6f714c0, 0xb6f7c050) = 0
11028 mprotect(0xb6f24000, 8192, PROT_READ) = 0
11028 mprotect(0xb6f51000, 8192, PROT_READ) = 0
11028 mprotect(0x21000, 4096, PROT_READ) = 0
11028 mprotect(0xb6f7b000, 4096, PROT_READ) = 0
11028 munmap(0xb6f73000, 21672) = 0
11028 ioctl(1, TCGETS, 0xbedb2554) = -1 EINVAL (Invalid argument)
11028 brk(NULL) = 0x1d23000
11028 brk(0x1d44000) = 0x1d44000
11028 stat64("/root/.terminfo", 0x1d23160) = -1 ENOENT (No such file or directory)
11028 stat64("/etc/terminfo", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
11028 stat64("/lib/terminfo", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
11028 stat64("/usr/share/terminfo", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
11028 gettimeofday({1491321517, 321307}, NULL) = 0
11028 access("/etc/terminfo/x/xterm", R_OK) = -1 ENOENT (No such file or directory)
11028 access("/lib/terminfo/x/xterm", R_OK) = 0
11028 open("/lib/terminfo/x/xterm", O_RDONLY|O_LARGEFILE) = 3
11028 fstat64(3, {st_mode=S_IFREG|0644, st_size=3361, ...}) = 0
11028 read(3, "\32\1)\0&\0\17\0\235\1u\5xterm|xterm-debian|X"..., 4096) = 3361
11028 read(3, "", 4096) = 0
11028 close(3) = 0
11028 gettimeofday({1491321517, 324975}, NULL) = 0
11028 ioctl(2, TCGETS, {B38400 opost isig icanon echo ...}) = 0
11028 ioctl(2, TCGETS, {B38400 opost isig icanon echo ...}) = 0
11028 ioctl(2, TIOCGWINSZ, {ws_row=54, ws_col=189, ws_xpixel=0, ws_ypixel=0}) = 0
11028 fstat64(1, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
11028 write(1, "\33[39;49m", 8) = 8
10941 <... read resumed> "\33[39;49m", 128) = 8
11028 exit_group(0) = ?
10941 read(3, <unfinished ...>
11028 +++ exited with 0 +++
10941 <... read resumed> "", 128) = 0
10941 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=11028, si_uid=0, si_status=0, si_utime=0, si_stime=1} ---
10941 sigreturn({mask=[]}) = 0
10941 close(3) = 0
10941 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 11028
10941 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb6fae418) = 11029
11029 close(12 <unfinished ...>
10941 wait4(-1, <unfinished ...>
11029 <... close resumed> ) = 0
11029 close(11) = 0
11029 close(10) = 0
11029 execve("/bin/echo", ["/bin/echo", "-e", " \33[31mfailed!\33[39;49m"], [/* 19 vars */]) = 0
11029 brk(NULL) = 0x318000
11029 uname({sysname="Linux", nodename="bpi-iot-ros", ...}) = 0
11029 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
11029 mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb6f2e000
11029 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
11029 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
11029 fstat64(3, {st_mode=S_IFREG|0644, st_size=21672, ...}) = 0
11029 mmap2(NULL, 21672, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb6f28000
11029 close(3) = 0
11029 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
11029 open("/lib/arm-linux-gnueabihf/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
11029 read(3, "\177ELF\1\1\1\3\0\0\0\0\0\0\0\0\3\0(\0\1\0\0\0\331i\1\0004\0\0\0"..., 512) = 512
11029 lseek(3, 894128, SEEK_SET) = 894128
11029 read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 2960) = 2960
11029 lseek(3, 888324, SEEK_SET) = 888324
11029 read(3, "A2\0\0\0aeabi\0\1(\0\0\0\0057-A\0\6\n\7A\10\1\t\2\n\3\f"..., 51) = 51
11029 fstat64(3, {st_mode=S_IFREG|0755, st_size=897088, ...}) = 0
11029 mmap2(NULL, 963928, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb6e1d000
11029 mprotect(0xb6ef3000, 65536, PROT_NONE) = 0
11029 mmap2(0xb6f03000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xd6000) = 0xb6f03000
11029 mmap2(0xb6f06000, 9560, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb6f06000
11029 close(3) = 0
11029 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb6f27000
11029 set_tls(0xb6f27870, 0xb6f27f48, 0xb6f31050, 0xb6f27870, 0xb6f31050) = 0
11029 mprotect(0xb6f03000, 8192, PROT_READ) = 0
11029 mprotect(0x24000, 4096, PROT_READ) = 0
11029 mprotect(0xb6f30000, 4096, PROT_READ) = 0
11029 munmap(0xb6f28000, 21672) = 0
11029 brk(NULL) = 0x318000
11029 brk(0x339000) = 0x339000
11029 fstat64(1, {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 0), ...}) = 0
11029 write(1, " \33[31mfailed!\33[39;49m\n", 22) = 22
11029 close(1) = 0
11029 close(2) = 0
11029 exit_group(0) = ?
11029 +++ exited with 0 +++
10941 <... wait4 resumed> [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 11029
10941 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=11029, si_uid=0, si_status=0, si_utime=0, si_stime=0} ---
10941 sigreturn({mask=[]}) = 11029
10941 close(12) = 0
10941 close(11) = 0
10941 exit_group(1) = ?
10941 +++ exited with 1 +++
(Posted on behalf of the OP).
I dont know exactly how, but it's solved... maybe reinstalling libapache2-mod-php7.0 was the failure.
Related
The Question
How do I best execute memory-intensive pipelines in Apache Beam?
Background
I've written a pipeline that takes the Naemura Bird dataset and converts the images and annotations to TF Records with TF Examples of the required format for the TF object detection API.
I tested the pipeline using DirectRunner with a small subset of images (4 or 5) and it worked fine.
The Problem
When running the pipeline with a bigger data set (day 1 of 3, ~21GB) it crashes after a while with a non-descriptive SIGKILL.
I do see a memory peak before the crash and assume that the process is killed because of a too high memory load.
I ran the pipeline through strace. These are the last lines in the trace:
[pid 53702] 10:00:09.105069 poll([{fd=10, events=POLLIN}, {fd=11, events=POLLIN}, {fd=12, events=POLLIN}, {fd=13, events=POLLIN}, {fd=14, events=POLLIN}, {fd=15, events=POLLIN}, {fd=16, events=POLLIN}, {fd=17, events=POLLIN}, {fd=18, events=POLLIN}, {fd=19, events=POLLIN}, {fd=20, events=POLLIN}], 11, 100) = 0 (Timeout)
[pid 53702] 10:00:09.205826 poll([{fd=10, events=POLLIN}, {fd=11, events=POLLIN}, {fd=12, events=POLLIN}, {fd=13, events=POLLIN}, {fd=14, events=POLLIN}, {fd=15, events=POLLIN}, {fd=16, events=POLLIN}, {fd=17, events=POLLIN}, {fd=18, events=POLLIN}, {fd=19, events=POLLIN}, {fd=20, events=POLLIN}], 11, 100 <unfinished ...>
[pid 53534] 10:00:09.259806 mmap(NULL, 63082496, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3aa43d7000
[pid 53694] 10:00:09.297140 <... clock_nanosleep resumed>NULL) = 0
[pid 53694] 10:00:09.297273 clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=0, tv_nsec=200000000}, <unfinished ...>
[pid 53702] 10:00:09.306409 <... poll resumed>) = 0 (Timeout)
[pid 53702] 10:00:09.306478 poll([{fd=10, events=POLLIN}, {fd=11, events=POLLIN}, {fd=12, events=POLLIN}, {fd=13, events=POLLIN}, {fd=14, events=POLLIN}, {fd=15, events=POLLIN}, {fd=16, events=POLLIN}, {fd=17, events=POLLIN}, {fd=18, events=POLLIN}, {fd=19, events=POLLIN}, {fd=20, events=POLLIN}], 11, 100) = 0 (Timeout)
[pid 53702] 10:00:09.406866 poll([{fd=10, events=POLLIN}, {fd=11, events=POLLIN}, {fd=12, events=POLLIN}, {fd=13, events=POLLIN}, {fd=14, events=POLLIN}, {fd=15, events=POLLIN}, {fd=16, events=POLLIN}, {fd=17, events=POLLIN}, {fd=18, events=POLLIN}, {fd=19, events=POLLIN}, {fd=20, events=POLLIN}], 11, 100 <unfinished ...>
[pid 53710] 10:03:55.844910 <... futex resumed>) = ?
[pid 53709] 10:03:57.797618 <... futex resumed>) = ?
[pid 53708] 10:03:57.797737 <... futex resumed>) = ?
[pid 53707] 10:03:57.797793 <... futex resumed>) = ?
[pid 53706] 10:03:57.797847 <... futex resumed>) = ?
[pid 53705] 10:03:57.797896 <... futex resumed>) = ?
[pid 53704] 10:03:57.797983 <... futex resumed>) = ?
[pid 53703] 10:03:57.798035 <... futex resumed>) = ?
[pid 53702] 10:03:57.798085 +++ killed by SIGKILL +++
[pid 53701] 10:03:57.798124 <... futex resumed>) = ?
[pid 53700] 10:03:57.798173 <... futex resumed>) = ?
[pid 53699] 10:03:57.798224 <... futex resumed>) = ?
[pid 53698] 10:03:57.798272 <... futex resumed>) = ?
[pid 53697] 10:03:57.798321 <... accept4 resumed> <unfinished ...>) = ?
[pid 53694] 10:03:57.798372 <... clock_nanosleep resumed> <unfinished ...>) = ?
[pid 53693] 10:03:57.798426 <... futex resumed>) = ?
[pid 53660] 10:03:57.798475 <... futex resumed>) = ?
[pid 53641] 10:03:57.798523 <... futex resumed>) = ?
[pid 53640] 10:03:57.798572 <... futex resumed>) = ?
[pid 53639] 10:03:57.798620 <... futex resumed>) = ?
[pid 53710] 10:03:57.798755 +++ killed by SIGKILL +++
[pid 53709] 10:03:57.798792 +++ killed by SIGKILL +++
[pid 53708] 10:03:57.798828 +++ killed by SIGKILL +++
[pid 53707] 10:03:57.798864 +++ killed by SIGKILL +++
[pid 53706] 10:03:57.798900 +++ killed by SIGKILL +++
[pid 53705] 10:03:57.798937 +++ killed by SIGKILL +++
[pid 53704] 10:03:57.798973 +++ killed by SIGKILL +++
[pid 53703] 10:03:57.799008 +++ killed by SIGKILL +++
[pid 53701] 10:03:57.799044 +++ killed by SIGKILL +++
[pid 53700] 10:03:57.799079 +++ killed by SIGKILL +++
[pid 53699] 10:03:57.799116 +++ killed by SIGKILL +++
[pid 53698] 10:03:57.799152 +++ killed by SIGKILL +++
[pid 53697] 10:03:57.799187 +++ killed by SIGKILL +++
[pid 53694] 10:03:57.799245 +++ killed by SIGKILL +++
[pid 53693] 10:03:57.799282 +++ killed by SIGKILL +++
[pid 53660] 10:03:57.799318 +++ killed by SIGKILL +++
[pid 53641] 10:03:57.799354 +++ killed by SIGKILL +++
[pid 53640] 10:03:57.799390 +++ killed by SIGKILL +++
[pid 53639] 10:03:57.910349 +++ killed by SIGKILL +++
10:03:57.910381 +++ killed by SIGKILL +++
Multiple things could cause this behaviour, because the pipeline runs fine with less Data, analysing what has changed could lead us to a resolution.
Option 1 : clean your input data
The third line of the logs you provide might indicate that you're processing unclean data in your bigger pipeline mmap(NULL, could mean that | "Get Content" >> beam.Map(lambda x: x.read_utf8()) is trying to read a null value.
Is there an empty file somewhere ? Are your files utf8 encoded ?
Option 2 : use smaller files as input
I'm guessing using the fileio.ReadMatches() will try to load into memory the whole file, if your file is bigger than your memory, this could lead to errors. Can you split your data into smaller files ?
Option 3 : use a bigger infrastructure
If files are too big for your current machine with a DirectRunner you could try to use an on-demand infrastructure using another runner on the Cloud such as DataflowRunner
My application is based on Micronaut and GraalVM (java 11) and does a simple call to http://httpbin.org/get:
#Controller("/api")
class HelloWorld(
#Client("http://httpbin.org")
private val httpClient: RxHttpClient
) {
private val logger = LoggerFactory.getLogger(javaClass)
#Get("/hello")
fun hello(): String {
return "Hello World!"
}
#Get("/fb")
fun fb(): Flowable<String> {
logger.info("Trying to call FB")
logger.info("Using url http://httpbin.org/get")
try {
return httpClient.retrieve("/get")
.doOnError { logger.error("Error calling fb api flowable", it) }
.doFinally { logger.info("Finished calling FB api flowable") }
} catch (ex: Exception) {
logger.error("Error calling fb api", ex)
throw ex
} finally {
logger.info("Finished calling fb api")
}
}
}
When I build a docker image of the app using this Dockerfile:
FROM maven:3.6.3-jdk-11 as maven
COPY . /home/app
WORKDIR /home/app
RUN mvn package
FROM oracle/graalvm-ce:19.3.1-java11 as graalvm
COPY --from=maven /home/app/target/app-*.jar /home/app/
WORKDIR /home/app
RUN gu install native-image
RUN native-image --no-server --enable-http --enable-https -cp app-*.jar
FROM debian:stretch
EXPOSE 8080
COPY --from=graalvm /home/app/app .
#RUN apt-get update && apt-get -y install strace
ENTRYPOINT ["./app"]
Everything works in my local environment.
But when I push the image to Google Cloud Repository, deploy it to Cloud Run and try to access the endpoint /api/fb, it crashes the container with 503 Service Unavailable.
The error in the logs is "The request failed because the HTTP connection to the instance had an error.". When I enable strace, these are the logs:
A 2020-02-07T12:04:27.443115Z [pid 18] <... futex resumed> ) = -1 ETIMEDOUT (Connection timed out)
A 2020-02-07T12:04:27.443125Z [pid 18] futex(0x3e61dc018a80, FUTEX_WAKE_PRIVATE, 1) = 0
A 2020-02-07T12:04:27.443357Z [pid 18] futex(0x3e62040009c4, FUTEX_CMP_REQUEUE_PRIVATE, 1, 2147483647, 0x3e6204000990, 234) = 1
A 2020-02-07T12:04:27.443369Z [pid 15] <... futex resumed> ) = 0
A 2020-02-07T12:04:27.443440Z [pid 18] futex(0x3e61dc018ab4, FUTEX_WAIT_BITSET_PRIVATE, 157, {tv_sec=3624, tv_nsec=391761056}, 0xffffffff <unfinished ...>
A 2020-02-07T12:04:27.443478Z [pid 15] futex(0x3e6204000990, FUTEX_WAKE_PRIVATE, 1) = 0
A 2020-02-07T12:04:27.959629Z [pid 20] epoll_wait(17, <unfinished ...>
A 2020-02-07T12:04:27.959658Z [pid 8] <... epoll_wait resumed> [{EPOLLIN, {u32=53, u64=53}}], 1024, -1) = 1
A 2020-02-07T12:04:27.959865Z [pid 8] accept(53, {sa_family=AF_INET6, sin6_port=htons(36294), inet_pton(AF_INET6, "::ffff:169.254.8.129", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, [28]) = 4
A 2020-02-07T12:04:27.959884Z [pid 8] fcntl(4, F_GETFL) = 0x2 (flags O_RDWR)
A 2020-02-07T12:04:27.959945Z [pid 8] getsockname(4, {sa_family=AF_INET6, sin6_port=htons(8080), inet_pton(AF_INET6, "::ffff:169.254.8.130", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, [28]) = 0
A 2020-02-07T12:04:27.960036Z [pid 8] getsockname(4, {sa_family=AF_INET6, sin6_port=htons(8080), inet_pton(AF_INET6, "::ffff:169.254.8.130", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, [28]) = 0
A 2020-02-07T12:04:27.960085Z [pid 8] fcntl(4, F_GETFL) = 0x2 (flags O_RDWR)
A 2020-02-07T12:04:27.960128Z [pid 8] fcntl(4, F_SETFL, O_RDWR|O_NONBLOCK) = 0
A 2020-02-07T12:04:27.960211Z [pid 8] setsockopt(4, SOL_TCP, TCP_NODELAY, [1], 4) = 0
A 2020-02-07T12:04:27.960322Z [pid 8] getsockopt(4, SOL_SOCKET, SO_SNDBUF, [1048576], [4]) = 0
A 2020-02-07T12:04:27.960440Z [pid 8] mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x3e60f0e00000
A 2020-02-07T12:04:27.960564Z [pid 8] mprotect(0x3e60f0e00000, 4096, PROT_NONE) = 0
A 2020-02-07T12:04:27.960793Z [pid 8] clone(/usr/bin/strace: Process 22 attached
A 2020-02-07T12:04:27.960875Z <unfinished ...>
A 2020-02-07T12:04:27.965711Z [pid 22] set_robust_list(0x3e60f16009e0, 24 <unfinished ...>
A 2020-02-07T12:04:27.965738Z [pid 8] <... clone resumed> child_stack=0x3e60f15fffb0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x3e60f16009d0, tls=0x3e60f1600700, child_tidptr=0x3e60f16009d0) = 22
A 2020-02-07T12:04:27.965753Z [pid 22] <... set_robust_list resumed> ) = -1 ENOSYS (Function not implemented)
A 2020-02-07T12:04:27.965761Z [pid 8] epoll_wait(29, <unfinished ...>
A 2020-02-07T12:04:27.965770Z [pid 22] sched_getaffinity(22, 32, [0, 1]) = 8
A 2020-02-07T12:04:27.965783Z [pid 22] futex(0x3b4c0c4, FUTEX_CMP_REQUEUE_PRIVATE, 1, 2147483647, 0x3b4c068, 30) = 1
A 2020-02-07T12:04:27.965790Z [pid 3] <... futex resumed> ) = 0
A 2020-02-07T12:04:27.965799Z [pid 3] futex(0x3b4c068, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...>
A 2020-02-07T12:04:27.965807Z [pid 22] futex(0x3b4c068, FUTEX_WAKE_PRIVATE, 1) = 1
A 2020-02-07T12:04:27.965816Z [pid 22] mmap(NULL, 2097152, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0 <unfinished ...>
A 2020-02-07T12:04:27.965824Z [pid 3] <... futex resumed> ) = 0
A 2020-02-07T12:04:27.965833Z [pid 3] futex(0x3b4c068, FUTEX_WAKE_PRIVATE, 1 <unfinished ...>
A 2020-02-07T12:04:27.965842Z [pid 22] <... mmap resumed> ) = 0x3e60f0c00000
A 2020-02-07T12:04:27.965850Z [pid 3] <... futex resumed> ) = 0
A 2020-02-07T12:04:27.965858Z [pid 3] futex(0x3b4c0c4, FUTEX_WAIT_PRIVATE, 31, NULL <unfinished ...>
A 2020-02-07T12:04:27.965866Z [pid 22] munmap(0x3e60f0d00000, 1048576) = 0
A 2020-02-07T12:04:27.965874Z [pid 22] prctl(PR_SET_NAME, "ntLoopGroup-1-7"...) = 0
A 2020-02-07T12:04:27.965882Z [pid 22] epoll_wait(20, [], 1024, 0) = 0
A 2020-02-07T12:04:27.965891Z [pid 22] mmap(NULL, 2097152, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x3e60f0a00000
A 2020-02-07T12:04:27.965903Z [pid 22] munmap(0x3e60f0b00000, 1048576) = 0
A 2020-02-07T12:04:27.966731Z [pid 22] epoll_ctl(20, EPOLL_CTL_ADD, 4, {EPOLLIN, {u32=4, u64=4}}) = 0
A 2020-02-07T12:04:27.966836Z [pid 22] epoll_wait(20, [{EPOLLIN, {u32=4, u64=4}}], 1024, 299996) = 1
A 2020-02-07T12:04:27.967306Z [pid 22] mmap(NULL, 2097152, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x3e60f0800000
A 2020-02-07T12:04:27.967325Z [pid 22] munmap(0x3e60f0900000, 1048576) = 0
A 2020-02-07T12:04:27.967493Z [pid 22] mmap(NULL, 16781312, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x3e60ef600000
A 2020-02-07T12:04:27.984977Z [pid 22] read(4, "GET /api/fb HTTP/1.1\r\nhost: app"..., 1024) = 864
A 2020-02-07T12:04:27.985474Z [36m12:04:27.985[0;39m [1;30m[nioEventLoopGroup-1-7][0;39m [34mINFO [0;39m [35mcom.roihunter.app.HelloWorld[0;39m - Trying to call FB
A 2020-02-07T12:04:27.985581Z [36m12:04:27.985[0;39m [1;30m[nioEventLoopGroup-1-7][0;39m [34mINFO [0;39m [35mcom.roihunter.app.HelloWorld[0;39m - Using url /act_984750788289990/insights
A 2020-02-07T12:04:27.985645Z [pid 22] write(1, "\33[36m12:04:27.985\33[0;39m \33[1;30m"..., 143) = 143
A 2020-02-07T12:04:27.985666Z [pid 22] write(1, "\33[36m12:04:27.985\33[0;39m \33[1;30m"..., 165) = 165
A 2020-02-07T12:04:27.985766Z [36m12:04:27.985[0;39m [1;30m[nioEventLoopGroup-1-7][0;39m [34mINFO [0;39m [35mcom.roihunter.app.HelloWorld[0;39m - Finished calling fb api
A 2020-02-07T12:04:27.985836Z [pid 22] write(1, "\33[36m12:04:27.985\33[0;39m \33[1;30m"..., 149) = 149
A 2020-02-07T12:04:27.986363Z [pid 22] write(4, "HTTP/1.1 200 OK\r\ntransfer-encodi"..., 143) = 143
E 2020-02-07T12:04:27.988626Z GET 503 546 B 31 ms curl/7.66.0 https://app-5phkf6s3jq-ez.a.run.app/api/fb GET 503 546 B 31 ms curl/7.66.0
A 2020-02-07T12:04:27.989104Z [pid 22] writev(4, [{iov_base="1\r\n", iov_len=3}, {iov_base="[", iov_len=1}, {iov_base="\r\n", iov_len=2}], 3) = 6
A 2020-02-07T12:04:27.989513Z [pid 22] mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x3e60eec00000
A 2020-02-07T12:04:27.989534Z [pid 22] mprotect(0x3e60eec00000, 4096, PROT_NONE) = 0
A 2020-02-07T12:04:27.989732Z [pid 22] clone(/usr/bin/strace: Process 23 attached
A 2020-02-07T12:04:27.989753Z <unfinished ...>
A 2020-02-07T12:04:27.989873Z [pid 23] set_robust_list(0x3e60ef4009e0, 24) = -1 ENOSYS (Function not implemented)
A 2020-02-07T12:04:27.989884Z [pid 23] sched_getaffinity(23, 32, [0, 1]) = 8
A 2020-02-07T12:04:27.989964Z [pid 23] futex(0x3b4c0c4, FUTEX_CMP_REQUEUE_PRIVATE, 1, 2147483647, 0x3b4c068, 32) = 1
A 2020-02-07T12:04:27.990286Z [pid 23] mmap(NULL, 2097152, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0 <unfinished ...>
A 2020-02-07T12:04:27.990303Z [pid 22] <... clone resumed> child_stack=0x3e60ef3fffb0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x3e60ef4009d0, tls=0x3e60ef400700, child_tidptr=0x3e60ef4009d0) = 23
A 2020-02-07T12:04:27.990311Z [pid 23] <... mmap resumed> ) = 0x3e60eea00000
A 2020-02-07T12:04:27.990320Z [pid 3] <... futex resumed> ) = 0
A 2020-02-07T12:04:27.990327Z [pid 3] futex(0x3b4c068, FUTEX_WAKE_PRIVATE, 1) = 0
A 2020-02-07T12:04:27.990335Z [pid 3] futex(0x3b4c0c4, FUTEX_WAIT_PRIVATE, 33, NULL <unfinished ...>
A 2020-02-07T12:04:27.990345Z [pid 23] munmap(0x3e60eeb00000, 1048576) = 0
A 2020-02-07T12:04:27.990423Z [pid 22] socket(AF_INET6, SOCK_STREAM, IPPROTO_IP <unfinished ...>
A 2020-02-07T12:04:27.990435Z [pid 23] prctl(PR_SET_NAME, "ionThreadPool-1"... <unfinished ...>
A 2020-02-07T12:04:27.990446Z [pid 22] <... socket resumed> ) = 80
A 2020-02-07T12:04:27.990668Z [pid 22] setsockopt(80, SOL_IPV6, IPV6_V6ONLY, [0], 4) = 0
A 2020-02-07T12:04:27.990684Z [pid 23] <... prctl resumed> ) = 0
I tried using a different library to make the call, Java 11 Http client, and it works. But whatever I do, it doesn't work when I use Micronaut Http client, and there is no helpful message.
I also tried using Java 8, a different linux distribution in the docker image (fedora:latest, ubuntu:latest), but it didn't help.
Do you have any idea what could be causing this?
Thanks for you answer in advance.
ok. Since this is the only question that comes up whenever someone searches for cloud run 503 error, I would like to share my horrible experience and solution.
I deployed my spring boot app on cloud run and configured pub-sub push subscription trigger for it. I tested it on my local system. Then i tested it on my private GCP project. And then when i deployed it on my organization's GCP account, I started to get 503 error.
After this, below is what i tried.
I tried changing the roles/permissions of the default service account (which was used by the run instance) by giving max permissions, but it was of no use.
Tried changing the concurrency to 1 (as suggested in GCP docs) but no help.
I started comparing each and every config of my run instance with my private GCP run instance, And then i found out that I mistakenly had checked below option which was causing this issue.
So i unchecked it, redeployed my instance, and it worked.
In my case I simply made a syntax error in my python code inside my Cloud Run instance and I got the 503 Service Unavailable error.
This showed up in the logs. Go to your Cloud Run Service and click "Logs" for more details.
So it looks like the core of the issue is that Cloud Run currently does not support HTTP streaming:
https://cloud.google.com/run/docs/issues#grpc_websocket
And when I return Flowable with Micronauts, it is opening a streamable HTTP connection. So the solution is not to use Flowable (or any other Publisher that is unbounded e.g. Flow, Flux etc) as a response.
I'm trying to run autossh (on a VM running CentOS6), but it's exiting immediately with the help message. I think this is a system issue because when I run it with the exact same parameters on another computer (running Ubuntu 14.04) it works fine. It's also fine when I run the same command but with ssh instead of autossh. So I tried strace to see if anything's wrong there. But if there is I'm not sure what it is. Any ideas?
Here's the autossh command: autossh -oStrictHostKeyChecking=no -oServerAliveInterval=15 -oServerAliveCountMax=4 -L 3130:localhost:3130 -N -i /path/to/some.pem user#remotehost
Here's the strace output (note myserver is an entry in .ssh/config that contains the same parameters as the previous command):
execve("/usr/local/bin/autossh", ["autossh", "myserver"], [/* 55 vars */]) = 0
brk(0) = 0xefc000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f26193cc000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=36751, ...}) = 0
mmap(NULL, 36751, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f26193c3000
close(3) = 0
open("/lib64/libnsl.so.1", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p#\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=113432, ...}) = 0
mmap(NULL, 2198192, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f2618f95000
mprotect(0x7f2618fab000, 2093056, PROT_NONE) = 0
mmap(0x7f26191aa000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15000) = 0x7f26191aa000
mmap(0x7f26191ac000, 6832, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f26191ac000
close(3) = 0
open("/lib64/libc.so.6", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\356\1\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1920936, ...}) = 0
mmap(NULL, 3750152, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f2618c01000
mprotect(0x7f2618d8b000, 2097152, PROT_NONE) = 0
mmap(0x7f2618f8b000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x18a000) = 0x7f2618f8b000
mmap(0x7f2618f90000, 18696, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f2618f90000
close(3) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f26193c2000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f26193c1000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f26193c0000
arch_prctl(ARCH_SET_FS, 0x7f26193c1700) = 0
mprotect(0x7f2618f8b000, 16384, PROT_READ) = 0
mprotect(0x7f26191aa000, 4096, PROT_READ) = 0
mprotect(0x7f26193cd000, 4096, PROT_READ) = 0
munmap(0x7f26193c3000, 36751) = 0
write(2, "usage: autossh [-V] [-M monitor_"..., 69usage: autossh [-V] [-M monitor_port[:echo_port]] [-f] [SSH_OPTIONS]
) = 69
write(2, "\n", 1
) = 1
write(2, " -M specifies monitor port. M"..., 238 -M specifies monitor port. May be overridden by environment
variable AUTOSSH_PORT. 0 turns monitoring loop off.
Alternatively, a port for an echo service on the remote
machine may be specified. (Normally port 7.)
) = 238
write(2, " -f run in background (autoss"..., 85 -f run in background (autossh handles this, and does not
pass it to ssh.)
) = 85
write(2, " -V print autossh version and"..., 39 -V print autossh version and exit.
) = 39
write(2, "\n", 1
) = 1
write(2, "Environment variables are:\n", 27Environment variables are:
) = 27
write(2, " AUTOSSH_GATETIME - how lo"..., 259 AUTOSSH_GATETIME - how long must an ssh session be established
before we decide it really was established
(in seconds). Default is 30 seconds; use of -f
flag sets this to 0.
) = 259
write(2, " AUTOSSH_LOGFILE - file t"..., 107 AUTOSSH_LOGFILE - file to log to (default is to use the syslog
facility)
) = 107
write(2, " AUTOSSH_LOGLEVEL - level "..., 49 AUTOSSH_LOGLEVEL - level of log verbosity
) = 49
write(2, " AUTOSSH_MAXLIFETIME - set th"..., 65 AUTOSSH_MAXLIFETIME - set the maximum time to live (seconds)
) = 65
write(2, " AUTOSSH_MAXSTART - max ti"..., 69 AUTOSSH_MAXSTART - max times to restart (default is no limit)
) = 69
write(2, " AUTOSSH_MESSAGE - messag"..., 74 AUTOSSH_MESSAGE - message to append to echo string (max 64 bytes)
) = 74
write(2, " AUTOSSH_PATH - path t"..., 53 AUTOSSH_PATH - path to ssh if not default
) = 53
write(2, " AUTOSSH_PIDFILE - write "..., 49 AUTOSSH_PIDFILE - write pid to this file
) = 49
write(2, " AUTOSSH_POLL - how of"..., 70 AUTOSSH_POLL - how often to check the connection (seconds)
) = 70
write(2, " AUTOSSH_FIRST_POLL - time b"..., 71 AUTOSSH_FIRST_POLL - time before first connection check (seconds)
) = 71
write(2, " AUTOSSH_PORT - port t"..., 61 AUTOSSH_PORT - port to use for monitor connection
) = 61
write(2, " AUTOSSH_DEBUG - turn l"..., 104 AUTOSSH_DEBUG - turn logging to maximum verbosity and log to
stderr
) = 104
write(2, "\n", 1
) = 1
exit_group(1) = ?
+++ exited with 1 +++
I had exactly the same problem with autossh 1.4e on CentOS Linux 7. autossh stopped immediately and printed the help, without even trying to connect to SSH.
The solution was to specify -M 0 on the command line:
autossh -M 0 \
-oStrictHostKeyChecking=no \
-oServerAliveInterval=15 \
-oServerAliveCountMax=4 \
-L 3130:localhost:3130 \
-N -i /path/to/some.pem user#remotehost
Come on ... the autossh exits with
write(2, "usage: autossh [-V] [-M monitor_"..., 69usage: autossh [-V] [-M monitor_port[:echo_port]] [-f] [SSH_OPTIONS]
which means that you specified wrong arguments and it is trying to explain you how the syntax looks like. To see what is wrong, there are several things you can try:
Use -vvv switches to ssh to see more verbose log from the ssh.
Use AUTOSSH_DEBUG environment variable to get some debug logs from autossh.
I successfully installed apache on my machine,but when I try to run it throws core dump.
I ran it again using strace,it throws core dump at write system call.
Below is the extract from output of stace. What can I infer from this error ?
write(2, "/tools/abc/local/bin/apachectl: line 79: 14372 Segmentation fault (core dump"..., 120/tools/abc/local/bin/apachectl: line 79: 14372 Segmentation fault (core dumped) $HTTPD -k $ARGV
) = 120
EDIT
Output of strace -s 100 httpd -k start
execve("/tools/abc/local/bin/httpd", ["httpd", "-k", "start"], [/* 38 vars */]) = 0
brk(0) = 0xe7e000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ffa0aece000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ffa0aecd000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ffa0aecc000
arch_prctl(ARCH_SET_FS, 0x7ffa0aecd680) = 0
--- SIGSEGV (Segmentation fault) # 0 (0) ---
+++ killed by SIGSEGV (core dumped) +++
Memory fault(coredump)
I have 6 machines: 3 running RHEL 3 (Taroon Update 9) and 3 other running RHEL 5.6 (Tikanga).
They all share a NFS volume containing
an Apache 2.2.4 installation
that Apache's configuration files
the PHP scripts and other content served by that Apache
On each machine, that shared Apache is started with the "-D $hostname" option to distinguish from the Apache instances running on the other machines. (Where $hostname is the hostname of the machine).
There is a load balancer (another Apache) in front of these 6 machines, balancing the requests between them.
The problem:
I've placed a .htaccess file in a directory containing these rules:
order deny,allow
deny from all
allow from my.very.own.ip
(I've tried the same rules in the configuration file between , with the same result)
Any request from any IP other than my.very.own.ip is denied by all 6 Apaches.
Requests from my.very.own.ip that hit the first 3 Apaches (running on RHEL 3) are accepted, which is normal.
Requests from my.very.own.ip that hit the last 3 Apaches (running on RHEL 5.6) are denied (HTTP 403, client denied by server configuration), which is not normal.
I've ran a strace on all the Apache servers (and their forked children) and this is how they look like:
- on a "good" Apache with request coming from my.very.own.ip
19553 read(7, "GET /my/website"..., 8000) = 1761
19553 gettimeofday({1343736878, 168708}, NULL) = 0
19553 gettimeofday({1343736878, 168787}, NULL) = 0
19553 gettimeofday({1343736878, 168835}, NULL) = 0
19553 stat64("/shark/apps/apache1/conf/www/maps/stationlist.txt", {st_mode=S_IFREG|0644, st_size=5557, ...}) = 0
19553 stat64("/shark/apps/apache1/conf/somedir/stationkidsurls.txt", {st_mode=S_IFREG|0644, st_size=6200, ...}) = 0
19553 stat64("/shark/www_docs/my/website/php/script.php", {st_mode=S_IFREG|0644, st_size=2449, ...}) = 0
19553 open("/shark/www_docs/.htaccess", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
19553 open("/shark/www_docs/my/.htaccess", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
19553 open("/shark/www_docs/my/website/.htaccess", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
19553 open("/shark/www_docs/my/website/php/.htaccess", O_RDONLY|O_LARGEFILE) = 10
19553 fstat64(10, {st_mode=S_IFREG|0644, st_size=51, ...}) = 0
19553 read(10, " order deny,allow\n deny f"..., 4096) = 51
19553 read(10, "", 4096) = 0
19553 close(10) = 0
19553 open("/shark/www_docs/my/website/php/script.php/.htaccess", O_RDONLY|O_LARGEFILE) = -1 ENOTDIR (Not a directory)
19553 getpid() = 19553
19553 setitimer(ITIMER_PROF, {it_interval={0, 0}, it_value={60, 0}}, NULL) = 0
19553 rt_sigaction(SIGPROF, {0x1280014, [PROF], SA_RESTORER|SA_RESTART, 0x3c10d8}, {0x1280014, [PROF], SA_RESTORER|SA_RESTART, 0x3c10d8}, 8) = 0
19553 rt_sigprocmask(SIG_UNBLOCK, [PROF], NULL, 8) = 0
19553 getcwd("/", 4095) = 2
19553 chdir("/shark/www_docs/my/website/php") = 0
- on a "bad" Apache with request coming from my.very.own.ip
1723 read(9, "GET /my/website"..., 8000) = 1761
1723 gettimeofday({1343736621, 548677}, NULL) = 0
1723 gettimeofday({1343736621, 548735}, NULL) = 0
1723 gettimeofday({1343736621, 548771}, NULL) = 0
1723 stat64("/shark/apps/apache1/conf/www/maps/stationlist.txt", {st_mode=S_IFREG|0644, st_size=5557, ...}) = 0
1723 stat64("/shark/apps/apache1/conf/somedir/stationkidsurls.txt", {st_mode=S_IFREG|0644, st_size=6200, ...}) = 0
1723 stat64("/shark/www_docs/my/website/php/script.php", {st_mode=S_IFREG|0644, st_size=2449, ...}) = 0
1723 open("/shark/www_docs/.htaccess", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
1723 open("/shark/www_docs/my/.htaccess", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
1723 open("/shark/www_docs/my/website/.htaccess", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
1723 open("/shark/www_docs/my/website/php/.htaccess", O_RDONLY|O_LARGEFILE) = 12
1723 fstat64(12, {st_mode=S_IFREG|0644, st_size=51, ...}) = 0
1723 read(12, " order deny,allow\n deny f"..., 4096) = 51
1723 read(12, "", 4096) = 0
1723 close(12) = 0
1723 open("/shark/www_docs/my/website/php/script.php/.htaccess", O_RDONLY|O_LARGEFILE) = -1 ENOTDIR (Not a directory)
1723 gettimeofday({1343736621, 550606}, NULL) = 0
1723 write(10, "[Tue Jul 31 08:10:21 2012] [erro"..., 159) = 159
729 <... read resumed> "[Tue Jul 31 08:10:21 2012] [erro"..., 65536) = 159
729 gettimeofday({1343736621, 550743}, NULL) = 0
729 gettimeofday({1343736621, 550786}, NULL) = 0
What do you think?
The only difference between these machines in the RHEL version. That may also mean that the NFS share ( /shark ) is mounted differently...
I've modified the .htaccess an changed the allow,deny rule with a Rewrite (requests not coming from my.very.own.ip are redirected to somewhere else). But... I'm still wondering why the same rules behave differently on different machines (same Apache, same configuration files).
Thank you!