I have been using python-libevent (https://pypi.org/project/python-libevent/) with multiple
versions of Python without any issues. Now, when I am trying to migrate to Python-3.8,
I see that a event_add() from my C code segfaults! and I am not getting much
from the backtrace/gdb!
I tried with python3.8-dbg and rebuilding libraries with debug symbols, still not getting
any clues!
(gdb) bt
#0 0x0000000000000000 in ?? ()
#1 0x00007fe4bc2f804e in event_add (ev=0x7fe4bba0d000, tv=0x7fe4bba0c020) at event.c:2443
where
2443: EVBASE_ACQUIRE_LOCK(ev->ev_base, th_base_lock);
Can anyone please help me on how to debug this further! I checked that I have enabled
pthreads and debug:
evthread_use_pthreads();
evthread_enable_lock_debugging();
event_enable_debug_mode();
event_enable_debug_logging(EVENT_DBG_ALL)
Has anything changed in the usage of libpython APIs - related to
memory-handling/capsules between 3.6 and 3.8 ? (my code works up until Py3.6)
libevent version: 2.1.8-stable
Py3.8 version: 3.8.0
Any help/pointers to debug much appreciated.
--More info--
After debugging with valgrind (helgrind tool) I see :
==13558== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==13558== Bad permissions for mapped region at address 0x0
==13558== at 0x0: ???
In my module, when I call event_add()!
Is there any restriction on
adding event to a event-base owned by a different module ?
Related
I'm using valgrind to try and locate the cause of a violating memory access in a C-cum-C++ program. Even with this access averted (i.e. when everything runs fine), valgrind tells me:
==11436== Memcheck, a memory error detector
==11436== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==11436== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==11436== Command: bin/monetdb-bp-reader /home/eyalroz/dbfarms/monetdb/tpch-sf-1
==11436==
--11436-- WARNING: Serious error when reading debug info
--11436-- When reading debug info from /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.24:
--11436-- Ignoring non-Dwarf2/3/4 block in .debug_info
--11436-- WARNING: Serious error when reading debug info
--11436-- When reading debug info from /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.24:
--11436-- Last block truncated in .debug_info; ignoring
--11436-- WARNING: Serious error when reading debug info
--11436-- When reading debug info from /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.24:
--11436-- parse_CU_Header: is neither DWARF2 nor DWARF3 nor DWARF4
--11436-- WARNING: Serious error when reading debug info
--11436-- When reading debug info from /lib/x86_64-linux-gnu/libgcc_s.so.1:
--11436-- Ignoring non-Dwarf2/3/4 block in .debug_info
--11436-- WARNING: Serious error when reading debug info
--11436-- When reading debug info from /lib/x86_64-linux-gnu/libgcc_s.so.1:
--11436-- Last block truncated in .debug_info; ignoring
--11436-- WARNING: Serious error when reading debug info
--11436-- When reading debug info from /lib/x86_64-linux-gnu/libgcc_s.so.1:
--11436-- parse_CU_Header: is neither DWARF2 nor DWARF3 nor DWARF4
What should I do about these warnings? Should I...
Fix them somehow? It's in the system libraries, not in my code; is it even possible to do something about them?
Suppress them somehow (but without suppressing any bona-fide error message)?
Ignore them - if I can neither fix nor suppress?
Notes: I use GNU/Linux Mint 18.4 with kernel 4.13.0-32 on an X86_64 machine.
This could be due to enabled compressed debug symbols in libstdc++.so, see relevant debian bug report: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=810295.
Valgrind did not support compressed debug symbols in 3.11 version, but it does support them now, see https://bugs.kde.org/show_bug.cgi?id=303877. I would suggest to update Valgrind to a latest version or build it from sources: http://valgrind.org/downloads/repository.html.
Iam facing JVM Crash cosistently while enabling hotdeploy (USING below java options on starting up JAVA_OPTS -Xmx4096m -XX:MetaspaceSize=512m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=crash -XX:ThreadStackSize=512 -XX:+UseConcMarkSweepGC -XX:ParallelGCThreads=5 -XX:NewRatio=2 -XX:+UnlockDiagnosticVMOptions -XX:-UseLoopPredicate -Xdebug -Xrunjdwp:transport=dt_socket,address=$DEBUG_PORT,server=y,suspend=n -XX:NewRatio=2 -Dspringloaded.synchronize=true JAVA_OPTS=`echo $JAVA_OPTS -Dspringloaded.synchronize=true -javaagent:springloaded-1.2.1.jar -noverify
)
Environment : JDK 1.8 U 66, RHEL 6.7
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007faee9a1e27c, pid=27208, tid=140379827795712
#
# JRE version: Java(TM) SE Runtime Environment (8.0_66-b17) (build 1.8.0_66-b17)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.66-b17 mixed mode linux-amd64 )
# Problematic frame:
# V [libjvm.so+0x35027c] Canonicalizer::do_If(If*)+0x1c
#
# Core dump written. Default location: core.27208
#
# An error report file with more information is saved as:
# hs_err_pid27208.log
# [ timer expired, abort... ]
I've noticed both -javaagent and -noverify in Java options list.
It looks like springloaded agent generates invalid bytecode, while the bytecode verification is explicitly turned off. No surprise, this may lead to unpredictable results including JVM crash.
This is not a JVM problem, but most likely a bug in springloaded agent. Try to remove -noverify option.
-XX:-TieredCompilation may also work around this particular problem, but don't expect application to work correctly if the bytecode fails to pass verification. It's better to stay away from the buggy agent libraries.
4.2.1 Crash in HotSpot Compiler Thread or Compiled Code
If the fatal error log indicates that the crash occurred in a compiler
thread, then it is possible (but not always the case) that you have
encountered a compiler bug. Similarly, if the crash is in compiled
code then it is possible that the compiler has generated incorrect
code.
In the case of the HotSpot Client VM (-client option), the compiler
thread appears in the error log as CompilerThread0. With the HotSpot
Server VM there are multiple compiler threads and these appear in the
error log file as CompilerThread0, CompilerThread1, and AdapterThread.
Below is a fragment of an error log for a compiler bug that was
encountered and fixed during the development of J2SE 5.0. The log file
shows that the HotSpot Server VM is used and the crash occurred in
CompilerThread1. In addition, the log file shows that the Current
CompileTask was the compilation of the java.lang.Thread.setPriority
method.
An unexpected error has been detected by HotSpot Virtual Machine:
:
Java VM: Java HotSpot(TM) Server VM (1.5-internal-debug mixed mode) :
--------------- T H R E A D ---------------
Current thread (0x001e9350): JavaThread "CompilerThread1" daemon
[_thread_in_vm, id=20]
Stack: [0xb2500000,0xb2580000), sp=0xb257e500, free space=505k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code,
C=native code) V [libjvm.so+0xc3b13c] :
Current CompileTask: opto: 11 java.lang.Thread.setPriority(I)V
(53 bytes)
--------------- P R O C E S S ---------------
Java Threads: ( => current thread ) 0x00229930 JavaThread "Low
Memory Detector" daemon [_thread_blocked, id=21]
=>0x001e9350 JavaThread "CompilerThread1" daemon [_thread_in_vm, id=20] :
In this case there are two potential workarounds:
The brute force approach: change the configuration so that the application is run with the -client option to specify the HotSpot
Client VM.
Assume that the bug only occurs during the compilation of the setPriority method and exclude this method from compilation.
The first approach (to use the -client option) might be trivial to
configure in some environments. In others, it might be more difficult
if the configuration is complex or if the command line to configure
the VM is not readily accessible. In general, switching from the
HotSpot Server VM to the HotSpot Client VM also reduces the peak
performance of an application. Depending on the environment, this
might be acceptable until the actual issue is diagnosed and fixed.
The second approach (exclude the method from compilation) requires
creating the file .hotspot_compiler in the working directory of the
application. Below is an example of this file:
exclude java/lang/Thread setPriority
In general the format of this file is exclude CLASS METHOD, where
CLASS is the class (fully qualified with the package name) and METHOD
is the name of the method. Constructor methods are specified as
and static initializers are specified as .
Note - The .hotspot_compiler file is an unsupported interface. It is
documented here solely for the purposes of troubleshooting and finding
a temporary workaround.
Once the application is restarted, the compiler will not attempt to
compile any of the methods listed as excluded in the .hotspot_compiler
file. In some cases this can provide temporary relief until the root
cause of the crash is diagnosed and the bug is fixed.
In order to verify that the HotSpot VM correctly located and processed
the .hotspot_compiler file that is shown in the example above, look
for the following log information at runtime. Note that the file name
separator is a dot, not a slash.
Excluding compile: java.lang.Thread::setPriority
Source
Agree with #apangin, In the program you are doing bytecode intrumentation (-agent) but specifies -noverify. When verification is turned off, you may end up such crashes.
You should not use -noverify or -Xverify:none during byte code intrumentation.
For those of you unfamiliar with bytecode verification, it is simply part of the JVM's classloading process that checks the code for certain dangerous and disallowed behavior. You can (but shouldn't) disable this protection on many JVMs by adding -Xverify:none or -noverify to the Java command line. https://blogs.oracle.com/buck/entry/never_disable_bytecode_verification_in
I have a 2 node OpenStack Mitaka environment consisting of a controller/compute node and a compute node.
I've followed the setup guide to enable instance live migration using LVM block storage. I.e.: There's no shared storage backend, just local LVM block storage.
Using OpenStack Horizon to perform the live migration a success message is displayed, however, the migration is far from successful. This worked pretty much out-of-the-box with our Juno installation. I've exhausted Google and cannot find any other instances of people facing the same problem. I thought it might have been a time synchronisation problem so have set both nodes to UTC. Still the problems persists.
Source machine /var/log/nova/nova-compute.log
2016-08-12 15:56:42.120 2230 ERROR nova.virt.libvirt.driver [req-b71ea7b0-5fa8-4b57-92d2-4edec62135c2 b017d86d1143461a92a267d4b912c104 88c686f09e1b427fb750f5c00716f84e - - -] [instance: 5763b6b6-370c-448c-8e8f-8b71eafaa8f1] Migration operation has aborted
2016-08-12 15:56:42.470 2230 ERROR nova.virt.libvirt.driver [req-b71ea7b0-5fa8-4b57-92d2-4edec62135c2 b017d86d1143461a92a267d4b912c104 88c686f09e1b427fb750f5c00716f84e - - -] [instance: 5763b6b6-370c-448c-8e8f-8b71eafaa8f1] Live Migration failure: internal error: unable to execute QEMU command 'migrate': Migration disabled: failed to allocate shared memory
Target node /var/log/libvirt/libvirtd.log
2016-08-12 15:56:41.864+0000: 2170: error : qemuMonitorJSONGetMigrationStatsReply:2443 : internal error: info migration reply was missing return status
2016-08-12 15:56:41.864+0000: 2170: error : virNetClientProgramDispatchError:177 : Cannot open log file: '/var/log/libvirt/qemu/instance-0000006a.log': Device or resource busy
There are no other events captured in the source or target nova or libvirt logs.
I should also note that I am trying to use qemu+tcp (libvirt listening enabled, default tcp port, no auth) rather than qemu+ssh in order to keep things simple while testing. In fact, I intend to only use qemu+tcp anyway.
Which version of ubuntu did you deploy?
I had the same error with ubuntu 14.04 and mitaka version.
And I figured out that default kernel (3.13) makes this problem.
I upgraded the kernel from 3.13 to 4.40 and this problem is gone now.
I hope my experience help you solve this problem out.
Thanks
I've been trying to build Mono 3.2.3 on Solaris 11 with no luck. I've made a few minor code changes and turned off configuration features to get to this point but now I'm stuck with mono crashing while trying to build System.dll. Any ideas?
MONO_PATH="./../../class/lib/basic:$MONO_PATH" /home/axsadm/mono-3.2.3/runtime/mono-wrapper ./../../class/lib/basic/basic.exe /codepage:65001 -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -nowarn:1699 -nostdlib -lib:./../../class/lib/build -r:mscorlib.dll -optimize /noconfig -nowarn:618 -d:CONFIGURATION_2_0 -unsafe -resource:resources/Asterisk.wav -resource:resources/Beep.wav -resource:resources/Exclamation.wav -resource:resources/Hand.wav -resource:resources/Question.wav -target:library -out:../../class/lib/build/tmp/System.dll #System.dll.sources
+ r=/home/axsadm/mono-3.2.3
+ MONO_CFG_DIR=/home/axsadm/mono-3.2.3/runtime/etc
+ PATH=/home/axsadm/mono-3.2.3/runtime/_tmpinst/bin:/usr/bin:/usr/sbin
+ MONO_SHARED_DIR=/home/axsadm/mono-3.2.3/runtime
+ export MONO_CFG_DIR MONO_SHARED_DIR PATH
+ [ -n '' ]
+ exec /home/axsadm/mono-3.2.3/libtool '--mode=execute' /home/axsadm/mono-3.2.3/mono/mini/mono --config /home/axsadm/mono-3.2.3/runtime/etc/mono/config ./../../class/lib/basic/basic.exe /codepage:65001 -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -nowarn:1699 -nostdlib -lib:./../../class/lib/build -r:mscorlib.dll -optimize /noconfig -nowarn:618 -d:CONFIGURATION_2_0 -unsafe -resource:resources/Asterisk.wav -resource:resources/Beep.wav -resource:resources/Exclamation.wav -resource:resources/Hand.wav -resource:resources/Question.wav -target:library -out:../../class/lib/build/tmp/System.dll #System.dll.sources
* Assertion at threads.c:1001, condition `info' not met
Native stacktrace:
/home/axsadm/mono-3.2.3/mono/mini/mono-boehm'mono_handle_native_sigsegv+0x1b8 [0x187c58]
/home/axsadm/mono-3.2.3/mono/mini/mono-boehm'sigabrt_signal_handler+0xa0 [0x1ed97c]
/lib/libc.so.1'__sighndlr+0xc [0xff0254f0]
/lib/libc.so.1'call_user_handler+0x370 [0xff018e50]
/lib/libc.so.1'sigacthandler+0x58 [0xff019040]
/lib/libc.so.1'_lwp_kill+0x8 [0xff029fa0]
/lib/libc.so.1'abort+0xc8 [0xfefaac2c]
/home/axsadm/mono-3.2.3/mono/mini/mono-boehm'monoeg_g_logv+0x174 [0x3d5454]
/home/axsadm/mono-3.2.3/mono/mini/mono-boehm'monoeg_assertion_message+0x38 [0x3d54e8]
/home/axsadm/mono-3.2.3/mono/mini/mono-boehm'mono_thread_attach_full+0x2bc [0x2dc650]
/home/axsadm/mono-3.2.3/mono/mini/mono-boehm'mono_thread_attach+0x10 [0x2dc37c]
/home/axsadm/mono-3.2.3/mono/mini/mono-boehm'mono_runtime_init+0x23c [0x314034]
/home/axsadm/mono-3.2.3/mono/mini/mono-boehm'mini_init+0x1a60 [0x77158]
/home/axsadm/mono-3.2.3/mono/mini/mono-boehm'mono_main+0x232c [0x1457a4]
/home/axsadm/mono-3.2.3/mono/mini/mono-boehm'mono_main_with_options+0x48c [0x5fa30]
/home/axsadm/mono-3.2.3/mono/mini/mono-boehm'main+0x2c [0x5fa74]
/home/axsadm/mono-3.2.3/mono/mini/mono-boehm'_start+0x5c [0x5f3e4]
=================================================================
Got a SIGABRT while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.
=================================================================
gmake[8]: *** [../../class/lib/build/tmp/System.dll] Abort (core dumped)
Configuration:
Engine:
GC: included Boehm
TLS: pthread
SIGALTSTACK: no
Engine: Building and using the JIT
oprofile: no
BigArrays: no
DTrace: no
LLVM Back End: no (dynamically loaded: no)
Libraries:
.NET 2.0/3.5: yes
.NET 4.0: yes
.NET 4.5: yes
MonoDroid: no
MonoTouch: no
JNI support: IKVM Native
libgdiplus: assumed to be installed
zlib: system zlib
This might be a bit late, but for me I had more luck using the external Boehm GC as opposed to the bundled/included one.
./autogen.sh --prefix=/usr --disable-dtrace --with-sgen=no --with-gc=boehm
This seemed to work OK for Solaris 10 and 11 on Intel. I'm still working on SPARC!
Good luck,
Steve
we are using NIOFSDirectory instead of MMapDirectory in an intensive test load for indexing. But we are still getting a SIGSEV with MMapDirectory. Is there anything else I should be looking for? This is a Linux 64 bit machine. Head of the stack trace follows:
Please let me know what more info would be helpful.
Thanks...
A fatal error has been detected by the Java Runtime Environment:
SIGSEGV (0xb) at pc=0x00002aaaab285ea7, pid=2408, tid=1128642880
JRE version: 6.0_25-b06
Java VM: Java HotSpot(TM) 64-Bit Server VM (20.0-b11 mixed mode linux-amd64 compressed >oops)
Problematic frame:
J org.apache.lucene.store.MMapDirectory$MMapIndexInput.readByte()B
You are most likely closing your IndexReader while other threads are still using it?
Lucene tries to catch this mis-use but it's detection is best-effort only.
You could also try calling MMapDirectory.setUseUnmap(false) to turn off unmapping; this should prevent the SIGSEGV.