AIX Missing Symbols ap_cleanup_scoreboard and ap_accept_lock_mech for Apache Process. However, the symbols do exist - mod-wsgi

I am having a problem whereby apache is not able to find certain symbols referenced from a library (mod_wsgi) loaded within the apache process.
When i start the apache process, i get this error.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
root [zibal]% ./usr/local/apache2/bin/apachectl restart httpd: Syntax error on line 53 of /usr/local/apache2/conf/httpd.conf: Cannot load /usr/local/apache2/modules/mod_wsgi.so into server:
rtld:0712-001 Symbol ap_cleanup_scoreboard was referenced from module /usr/local/apache2/modules/mod_wsgi.so(), but a runtime definition of the symbol was not found.
rtld: 0712-001 Symbol ap_accept_lock_mech was referenced from module /usr/local/apache2/modules/mod_wsgi.so(), but a runtime definition of the symbol was not found
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
I am pasting the output of nm command on the apache executable and those symbols seem to exist.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
root [zibal]% nm ./usr/local/apache2/bin/httpd | grep ap_accept_lock_mech
ap_accept_lock_mech D 536880332
ap_accept_lock_mech d 536900392 4
ap_accept_lock_mech:G879 - 0
root [zibal]% nm ./usr/local/apache2/bin/httpd | grep ap_cleanup_scoreboard
.ap_cleanup_scoreboard T 268613428 212
ap_cleanup_scoreboard D 536890068
ap_cleanup_scoreboard d 536890068 12
ap_cleanup_scoreboard d 536899972 4
ap_cleanup_scoreboard:F385 - 2976 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Please guide.

And this is how I was able to solve this issue (I had to do apply two solutions as indicated below).
Solution 1 (Thanks to Graham Dumpleton)
Don't change anything in Apache Code.
Go into mod_wsgi.c source code and change:
/*
* Cleanup the Apache scoreboard to ensure that any
* shared memory segments or memory mapped files not
* available to code in daemon processes.
*/
ap_cleanup_scoreboard(0);
to
#if 0
/*
* Cleanup the Apache scoreboard to ensure that any
* shared memory segments or memory mapped files not
* available to code in daemon processes.
*/
ap_cleanup_scoreboard(0);
#endif
and then look for:
#if !defined(AP_ACCEPT_MUTEX_TYPE)
sconfig->lock_mechanism = ap_accept_lock_mech;
#else
sconfig->lock_mechanism = APR_LOCK_DEFAULT;
#endif
and change it to:
#define AP_ACCEPT_MUTEX_TYPE 1
#if !defined(AP_ACCEPT_MUTEX_TYPE)
sconfig->lock_mechanism = ap_accept_lock_mech;
#else
sconfig->lock_mechanism = APR_LOCK_DEFAULT;
#endif
Then build mod_wsgi
Solution 2 (Thanks to Jeff Trawick from Apache)
Extract the Source tar ball in a directory.....cd into that directory
Locate the file include/mpm_common.h
Change
extern apr_lockmech_e ap_accept_lock_mech;
to
AP_DECLARE_DATA extern apr_lockmech_e ap_accept_lock_mech;
Locate the file include/scoreboard.h
Change
apr_status_t ap_cleanup_scoreboard(void *d);
to
AP_DECLARE(apr_status_t) ap_cleanup_scoreboard(void *d);
And then issue the following commands
./configure
make
After this step, there will be a httpd.exp file create in the server
directory and a httpd file created in the present directory
remove httpd
edit server/httpd.exp and add one line for ap_accept_lock_mech
make
make install (as root)
Then build mod_wsgi 3.3 from Source Tarball
I hope this helps others facing the same problem.

Related

Cross Compile OpenSceneGraph for arm processor running embedded linux?

I am trying to build OpenSceneGraph(Version 3.6.5) for arm processor running embedded linux.
cmake throws different errors, can someone help ? What needs to be added in CMakeLists.txt for these errors specifically ?
1. /mnt/sdk8_2/OpenSceneGraph/build/CMakeFiles/CMakeTmp/src.cxx:2:17: fatal error: cstdlib: No such file or directory
2 | #include
but it is there at /opt/arago/sysroots/aarch64-linux/usr/include/c++/9.2.1/cstdlib
2. atomic.h: No such file or directory
but following files are there
/opt/arago/sysroots/aarch64-linux/usr/include/tbb/atomic.h
/opt/arago/sysroots/aarch64-linux/lib/modules/5.10.41-g4c2eade9f7/build/arch/arm64/include/asm/atomic.h
/opt/arago/sysroots/aarch64-linux/lib/modules/5.10.41-g4c2eade9f7/build/include/linux/atomic.h
3. /mnt/sdk8_2/OpenSceneGraph/build/CMakeFiles/CMakeTmp/src.cxx:2:17: fatal error: windows.h: No such file or directory
But it's there at
/opt/arago/sysroots/aarch64-linux/usr/include/boost/predef/os/windows.h
/opt/arago/sysroots/aarch64-linux/lib/modules/5.10.41-g4c2eade9f7/build/include/asm-generic/atomic.h
/opt/arago/sysroots/aarch64-linux/lib/modules/5.10.41-g4c2eade9f7/build/include/asm-generic/bitops/atomic.h
/opt/arago/sysroots/aarch64-linux/lib/modules/5.10.100-g7a7a3af903/build/arch/arm64/include/asm/atomic.h
/opt/arago/sysroots/aarch64-linux/lib/modules/5.10.100-g7a7a3af903/build/include/linux/atomic.h
/opt/arago/sysroots/aarch64-linux/lib/modules/5.10.100-g7a7a3af903/build/include/asm-generic/atomic.h
/opt/arago/sysroots/aarch64-linux/lib/modules/5.10.100-g7a7a3af903/build/include/asm-generic/bitops/atomic.h

How to build AWS C++ SDK on Solaris?

I am trying to build the AWS C++ SDK on Solaris, but I cannot do so successfully.
I found this open issue on the AWS C++ SDK page that says it is possible, but there is no guide on it and I am hoping somebody here can help.
Here is the command I use to build it:
$ cmake ../aws-sdk-cpp/ -DCMAKE_BUILD_TYPE=Debug -DBUILD_ONLY="s3"
Here is the output:
-- TARGET_ARCH not specified; inferring host OS to be platform compilation target
-- Building AWS libraries as shared objects
-- Generating linux build config
-- Building project version: 1.7.134
-- Configuring done
-- Generating done
-- Build files have been written to: /workspace/dmoini/sdk_build/.deps
gmake: Warning: File 'Makefile' has modification time 267 s in the future
gmake[1]: Warning: File 'CMakeFiles/Makefile2' has modification time 267 s in the future
gmake[2]: Warning: File 'CMakeFiles/AwsCCommon.dir/progress.make' has modification time 267 s in the future
gmake[2]: warning: Clock skew detected. Your build may be incomplete.
gmake[2]: Warning: File 'CMakeFiles/AwsCCommon.dir/progress.make' has modification time 267 s in the future
[ 4%] Performing build step for 'AwsCCommon'
[ 1%] Building C object CMakeFiles/aws-c-common.dir/source/array_list.c.o
In file included from /usr/include/stdio.h:37:0,
from /workspace/dmoini/sdk_build/.deps/build/src/AwsCCommon/include/aws/common/common.h:22,
from /workspace/dmoini/sdk_build/.deps/build/src/AwsCCommon/include/aws/common/array_list.h:18,
from /workspace/dmoini/sdk_build/.deps/build/src/AwsCCommon/source/array_list.c:16:
/opt/gcc-5.1.0/lib/gcc/i386-pc-solaris2.11/5.1.0/include-fixed/sys/feature_tests.h:405:2: error: #error "Compiler or options invalid for pre-UNIX 03 X/Open applications and pre-2001 POSIX applications"
#error "Compiler or options invalid for pre-UNIX 03 X/Open applications \
^
gmake[5]: *** [CMakeFiles/aws-c-common.dir/build.make:63: CMakeFiles/aws-c-common.dir/source/array_list.c.o] Error 1
gmake[4]: *** [CMakeFiles/Makefile2:484: CMakeFiles/aws-c-common.dir/all] Error 2
gmake[3]: *** [Makefile:139: all] Error 2
gmake[2]: *** [CMakeFiles/AwsCCommon.dir/build.make:112: build/src/AwsCCommon-stamp/AwsCCommon-build] Error 2
gmake[1]: *** [CMakeFiles/Makefile2:68: CMakeFiles/AwsCCommon.dir/all] Error 2
gmake: *** [Makefile:84: all] Error 2
CMake Error at CMakeLists.txt:193 (message):
Failed to build third-party libraries.
Additionally, here is my system information:
$ uname -a
SunOS bld-dmoini-01-sv4b 5.11 omnios-r151020-4151d05 i86pc i386 i86pc
Any and all help/guidance is greatly appreciated.
I've successfully completed compiling the AWS C++ SDK on a stock install of Solaris 11.4, and found several issues that could cause the problems noted.
Start with a clean source tree.
Remove -Werror
The first thing do to is remove the -Werror compiler options. The version of OpenSSL installed by default on Solaris 11.4 has quite a few deprecated functions, and the -Werror option causes the build to fail when it runs into those deprecations. I used this find command run from the topmost directory of the AWS SDK source tree to remove all the -Werror options:
vi `find . | xargs grep -l Werror`
You'll get about three or four files, only two of which are actually setting the -Werror as a compiler option. Just remove the "-Werror" strings from those files.
Fix the POSIX defines
Then run cmake . in the topmost directory. It will fail because the cmake files that it downloads will have improper POSIX command-line options - -D_POSIX_C_SOURCE=200809L -D_XOPEN_SOURCE=500. That 500 is wrong. _POSIX_C_SOURCE=200809L corresponds to _XOPEN_SOURCE=700. _XOPEN_SOURCE=500 is SUSv2, circa 1997. It's not proper to compile a SUSv2 application with C99.
Per 2.2.1 Strictly Conforming POSIX Application, paragraph 8:
For the C programming language, shall define _POSIX_C_SOURCE to be 200809L before any header is included
and 2.2.4 Strictly Conforming XSI Application, paragraph 8:
For the C programming language, shall define _XOPEN_SOURCE to be 700 before any header is included
Per the Illumos sys/feature_tests.h file (based on OpenSolaris, which was also the basis for Solaris 11):
* Feature Test Macro Specification
* ------------------------------------------------ -------------
* _XOPEN_SOURCE XPG3
* _XOPEN_SOURCE && _XOPEN_VERSION = 4 XPG4
* _XOPEN_SOURCE && _XOPEN_SOURCE_EXTENDED = 1 XPG4v2
* _XOPEN_SOURCE = 500 XPG5
* _XOPEN_SOURCE = 600 (or POSIX_C_SOURCE=200112L) XPG6
* _XOPEN_SOURCE = 700 (or POSIX_C_SOURCE=200809L) XPG7
The files cmake downloads via git need to be edited:
vi `find .deps | xargs grep -l XOPEN_SOURCE`
Change any -D_XOPEN_SOURCE=500 to -D_XOPEN_SOURCE=700 and rerun cmake .. It should complete successfully this time.
Then run gmake. (I find gmake works much better on Solaris for just about all open source projects, as many open source projects use GNU-specific make extensions.)
Now you get to fix any broken source code you run into.
Fix broken source code
1
The file aws-sdk-cpp/aws-cpp-sdk-core/source/platform/linux-shared/OSVersionInfo.cpp has the following wrong code:
Aws::String ComputeOSVersionString()
{
utsname name;
int32_t success = uname(&name);
Per POSIX, the correct type is struct utsname, not just utsname:
int uname(struct utsname *name);
The AWS code needs to be:
Aws::String ComputeOSVersionString()
{
struct utsname name;
int success = uname(&name);
And no, I'm most certainly not impressed with the quality of the AWS code, given this, umm, laugher:
while (!feof(outputStream))
Yes, an actual while (!feof()) loop...
2
The file aws-sdk-cpp/aws-cpp-sdk-mediaconvert/include/aws/mediaconvert/model/M2tsSegmentationMarkers.h uses an enumeration with the value EBP, which conflicts with the EBP register #define in /usr/include/sys/regset.h.
I just changed it to EBP_HASH as that seems to match the code somewhat:
vi `find . | xargs grep -l EBP`
3
The file aws-sdk-cpp/aws-cpp-sdk-route53domains/include/aws/route53domains/model/CountryCode.h creates an enumeration value ES that conflicts with the ES register #define in /usr/include/sys/regset.h. I just added
#ifdef ES
#undef ES
#endif
and the compile continued. I don't know if that #undef could have broken anything.
4
The file aws-sdk-cpp/aws-cpp-sdk-waf/include/aws/waf/model/GeoMatchConstraintValue.h has ES, GS, and SS enumeration value that conflict with the ES, GS, and SS register #define's in /usr/include/sys/regset.h.
Again, I just added a few more #undef's:
#ifdef ES
#undef ES
#endif
#ifdef GS
#undef GS
#endif
#ifdef SS
#undef SS
#endif
I'm really wondering why sys/regset.h is being #include'd in just about everything in the AWS SDK.
5
Same problem in aws-sdk-cpp/aws-cpp-sdk-waf-regional/include/aws/waf-regional/model/GeoMatchConstraintValue.h. Same fix, add:
#ifdef ES
#undef ES
#endif
#ifdef GS
#undef GS
#endif
#ifdef SS
#undef SS
#endif
Note that compiling on SPARC hardware means the #define value from sys/regset.h will be completely different, and any errors will be completely different.
6
The file aws-sdk-cpp/aws-cpp-sdk-core-tests/utils/FileSystemUtilsTest.cpp incorrectly assumes the POSIX NAME_MAX value is defined. Per the POSIX Pathname Variable Values standard (bolding mine):
Pathname Variable Values
The values in the following list may be constants within an
implementation or may vary from one pathname to another. For example,
file systems or directories may have different characteristics.
A definition of one of the symbolic constants in the following list
shall be omitted from the <limits.h> header on specific
implementations where the corresponding value is equal to or greater
than the stated minimum, but where the value can vary depending on the
file to which it is applied. The actual value supported for a specific
pathname shall be provided by the pathconf() function.
Again: the "definition ... shall be omitted ... where the value can vary".
The AWS code wrongly assumes NAME_MAX must be #define'd.
I just hardcoded a value of 255 to get past this point, although using something like _POSIX_NAME_MAX or _XOPEN_NAME_MAX is probably better.
7
File aws-sdk-cpp/ws-cpp-sdk-core-tests/http/HttpClientTest.cpp seems to be incorrectly assuming a std::shared_ptr will be 8 bytes. This question and answer provides a good example of how that's wrong.
I just ignored this error as it's just a test and continued with gmake -i, which completed successfully outside of this one error.

Undefined symbol for HDF5 when installing with cmake in Linux environment

I have been able to install and start a program with CMake (with HDF5) but once I access the "about" drop-down link for said program it crashes with the following error:
python: symbol lookup error: /nobackup/<user id>/program-devel/dist_linux/bin_linux/libprogramDLL.so: undefined symbol: h5lib_MP_h5get_libversion_f_
I believe it is an issue with linking static libraries but I am, unfortunately, quite new to CMake and unable to isolate the root problem. I know that this "symbol" is tied somehow to a libhdf5_fortran.a and this is listed in my Cache with:
$ grep -rnw '/nobackup/<user id>/program-devel/build' -e "libhdf5_fortran"
/nobackup/<user id>/program-devel/build/CMakeCache.txt:234:ToolkitLib_LIB_DEPENDS:STATIC=general;SomeLib;general;libz.a;general;libhdf5.a;general;libhdf5_fortran.a;
I’m not sure if this might be where the problem is or not but this is from the ToolKit file - CMakeLists.txt.
if (${USE_HDF5})
#link_directories (${HDF5_DIRECTORY}/lib)
if (WIN32)
target_link_libraries(ToolkitLib libszip libzlib libhdf5 libhdf5_f90cstub libhdf5_fortran)
else ()
# doesn't seem to work on Linux for some reason....
#target_link_libraries(ToolkitLib libsz libz libhdf5 libhdf5_fortran)
# ... try this ... seems to be getting the shared libs...not the static ones...
# set(HDF5_LINK_FLAGS "-L${HDF5_DIRECTORY}/lib -lz -lhdf5_fortran -lhdf5")
# SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${HDF5_LINK_FLAGS}")
# SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} ${HDF5_LINK_FLAGS}")
# ... doesn't work either ...
target_link_libraries(ToolkitLib libz.a libhdf5.a libhdf5_fortran.a)
endif()
endif()
The last 4 "target_link_library" elements are in directories I've added to my LD_LIBRARY_PATH (although, I've heard this isn't the preferred approach). There is also a final output line when I issue gmake install that might be relevant:
-- Set runtime path of "/nobackup/<user id>/program-devel/dist_linux/bin_linux/libProgramDLL.so" to "$ORIGIN/"
It seems I fixed my problem by adding a --with-pic on the hdf5 installation bash scripts. I also had to remove a -standard-semantics from the CMAKE_Fortran_FLAGS in the src directory. Those two things along with the help you provided fixed my problem. Thanks a lot Pierre de Buyl!

Undefined symbol unixd_config

Whenever I try to load the libmod_sm22.so or libmodsm_20.so module into apache-2.4.6, I am getting error as shown below :
httpd: Syntax error on line 65 of httpd.conf: Cannot load /opt/software/siteminder/waR12cr009/webagent/bin/libmod_sm22.so into server: /opt/software/siteminder/waR12cr009/webagent/bin/libmod_sm22.so: undefined symbol: unixd_config
From the apache documentation I found that apache-2.4.6 API has changed ‘unixd_config’ to ‘ap_unixd_config’!
But the libmod_sm20.so / libmod_sm22.so is still looking for ‘unixd_config’
In the apache’s include folder I have found references for the unixd_config symbol as shown below
============================================================================
[apache#VDCLL1828 include]$ cat ap_mmn.h |grep 'unixd_config'
* 20071108.9 (2.3.0-dev) Add chroot support to unixd_config
* 20101106.2 (2.3.9-dev) Add suexec_disabled_reason field to ap_unixd_config
* add ap_unixd_config.group_name
[apache#VDCLL1828 include]$ cat unixd.h |grep 'unixd_config'
} unixd_config_rec;
AP_DECLARE_DATA extern unixd_config_rec ap_unixd_config;
===========================================================
I didn’t try to change these references as these are not completely related to what I am looking for & in turn this may break other working things if I mess up.
At this point I have two options to make siteminder working in apache-2.4.6
1) Prepare the libmod_sm22.so that it looks for ap_unixd_config instead of unixd_config (If possible)
2) Look out for a patch to apache 2.4.6 api .
Any thoughts around this are highly appreciated.
Thanks,
Kiran

Linking Free Pascal programs to include dependencies

I have two Free Pascal units that I would like to use from a C program on linux.
Here is what I do:
$ fpc -fPIC base64.pas queueutils.pas
Warning: Only one source file supported
Free Pascal Compiler version 2.2.2 [2008/11/05] for x86_64
Copyright (c) 1993-2008 by Florian Klaempfl
Target OS: Linux for x86-64
Compiling queueutils.pas
queueutils.pas(2088,11) Warning: Symbol "Socket" is deprecated
queueutils.pas(2097,10) Warning: Symbol "Connect" is deprecated
queueutils.pas(2104,3) Warning: Symbol "Sock2Text" is deprecated
2432 lines compiled, 0.5 sec
4 warning(s) issued
$ ppumove -o queueutils -e ppl *.ppu
PPU-Mover Version 2.1.1
Copyright (c) 1998-2007 by the Free Pascal Development Team
Processing base64.ppu... Done.
Processing queueutils.ppu... Done.
Linking queueutils.o base64.o
Done.
Seems fine so far, libqueueutils.so is created:
$ file libqueueutils.so
libqueueutils.so: ELF 64-bit LSB shared object, AMD x86-64, version 1 (SYSV), not stripped
$ ldd libqueueutils.so
ldd: warning: you do not have execution permission for `./libqueueutils.so'
statically linked
However when the C program tries to use the library this way:
libqueue = dlopen("./libqueueutils.so", RTLD_LAZY);
if (!libqueue) {
fprintf (stderr, "%s\n", dlerror());
}
it yields an error message:
$ ./tmbrkr
./libqueueutils.so: undefined symbol: VMT_PROCESS_TPROCESS
This VMT_PROCESS_TPROCESS-related error is resolved if I add process.o and process.ppu to the linking process done by ppumove. However after doing so another unit is missing and after that another... You get it.
Is there a way to somehow link all the necessary units together in one .so file so that the C program can dlopen() the library properly?
Just like a normal binary (exe) is from a "program" source file , a .so/dll is created from a ''library'' sourcefile.
For the rest is the model is the same. You simply build the library mainprogram, and the compiler collects all units necessary and stuffs them in the .so.
With the exports keyword you can define what symbols to export.
library testdll;
uses x,y,z;
// define exportable symbols here
// some examples of symbol exports
exports
P1 index 1, // dll based on index
P2 name 'Proc2', // normal export with alternate external symbol
P3, // just straight export.
P4 resident // for some MCU use
;
begin
// startup code
end.
Also look up $soname $libsuffix and $libprefix in the manual.
Though I would recommend just using most recent 2.6.0, not some 5 year old 2.2.2
It might require recompiling FPC first with PIC though.