I moved my Gem5 simulations from my system to a server. My system does not have HDF5 libraries, but the server has, and I am met with this error:
/usr/local/lib/python2.7/config/libpython2.7.a(posixmodule.o): In function `posix_tmpnam':
/space/src/Python-2.7/./Modules/posixmodule.c:7275: warning: the use of `tmpnam_r' is dangerous, better use `mkstemp'
/usr/local/lib/python2.7/config/libpython2.7.a(posixmodule.o): In function `posix_tempnam':
/space/src/Python-2.7/./Modules/posixmodule.c:7230: warning: the use of `tempnam' is dangerous, better use `mkstemp'
build/X86/base/lib.o.partial: In function `Stats::Hdf5::addMetaData(H5::DataSet&, char const*, double)':
/net/nasstore/students/GRAD/ECE/febinps/home/Paper3/gem5/build/X86/base/stats/hdf5.cc:312: undefined reference to `H5::H5Object::createAttribute(char const*, H5::DataType const&, H5::DataSpace const&, H5::PropList const&) const'
build/X86/base/lib.o.partial: In function `Stats::Hdf5::addMetaData(H5::DataSet&, char const*, std::vector<char const*, std::allocator<char const*> > const&)':
/net/nasstore/students/GRAD/ECE/febinps/home/Paper3/gem5/build/X86/base/stats/hdf5.cc:280: undefined reference to `H5::H5Object::createAttribute(char const*, H5::DataType const&, H5::DataSpace const&, H5::PropList const&) const'
build/X86/base/lib.o.partial: In function `Stats::Hdf5::addMetaData(H5::DataSet&, char const*, std::string const&)':
/net/nasstore/students/GRAD/ECE/febinps/home/Paper3/gem5/build/X86/base/stats/hdf5.cc:302: undefined reference to `H5::H5Object::createAttribute(char const*, H5::DataType const&, H5::DataSpace const&, H5::PropList const&) const'
collect2: error: ld returned 1 exit status
scons: *** [build/X86/gem5.opt] Error 1
scons: building terminated because of errors.
How can I fix this? Is there an yway I can avoid the build from using HDF5? I cannot do much at the server as I do not have admin access.
After the resolution of: https://askubuntu.com/questions/1187343/installation-of-gem5-on-ubuntu-19-10-on-my-laptop-i5-cpu-m520-64bit in gem5 b383997d4a9c642dd4356bfc4554ac7ae183ae62 (March 2020) the Ubuntu 19.10 (GCC 9) build is working for me:
scons -j `nproc` build/ARM/gem5.opt
If you find any further build bugs, do open an issue on the Jira issue tracker.
After https://gem5-review.googlesource.com/c/public/gem5/+/34777 gem5 should be building every time with HDF5 on the upstream testing system see also https://askubuntu.com/questions/350475/how-can-i-install-gem5/1275773#1275773 so I don't think it will break again easily.
I don't know the root cause of this issue, some one who is working on Gem5 could possibly answer that.
But as a workaround, since I have no admin access to the server ,and even then removing a library just for the sake of one build doesn't feel right, I edited the SConstruct file in gem5/, where the environment variables are gathered and passed:
have_hdf5 = 0 #check_hdf5() #line number 951 in the SConstruct file
This seems to work for the build.
In case you run on older system like Debian 10 or Ubuntu 16.04, the errors are due to the fact that the hdf5 library path is not in default system library path. I solved it by manually (brutally) linking gem5 ...
The link flags added are:
-L/usr/lib/x86_64-linux-gnu/hdf5/serial/ -lhdf5_cpp -lhdf5
g++ -o /gem5/default/build/X86/gem5.opt \
-Wl,--as-needed -fuse-ld=gold -L/usr/lib/python2.7/config-x86_64-linux-gnu -L/usr/lib \
-Xlinker -export-dynamic \
-Wl,-O1 -Wl,-Bsymbolic-functions -z origin -O3 \
/gem5/default/build/X86/sim/main.o \
/gem5/default/build/X86/dev/net/lib.o.partial \
/gem5/default/build/X86/base/lib.o.partial \
/gem5/default/build/X86/dev/i2c/lib.o.partial \
/gem5/default/build/X86/cpu/testers/traffic_gen/lib.o.partial \
/gem5/default/build/X86/mem/cache/tags/indexing_policies/lib.o.partial \
/gem5/default/build/X86/mem/ruby/slicc_interface/lib.o.partial \
/gem5/default/build/X86/mem/probes/lib.o.partial \
/gem5/default/build/X86/mem/ruby/network/simple/lib.o.partial \
/gem5/default/build/X86/dev/x86/lib.o.partial \
/gem5/default/build/X86/mem/ruby/network/fault_model/lib.o.partial \
/gem5/default/build/X86/systemc/utils/lib.o.partial \
/gem5/default/build/X86/systemc/dt/int/lib.o.partial \
/gem5/default/build/X86/cpu/kvm/lib.o.partial \
/gem5/default/build/X86/cpu/simple/probes/lib.o.partial \
/gem5/default/build/X86/base/filters/lib.o.partial \
/gem5/default/build/X86/dev/serial/lib.o.partial \
/gem5/default/build/X86/sim/power/lib.o.partial \
/gem5/default/build/X86/mem/cache/tags/lib.o.partial \
/gem5/default/build/X86/arch/x86/bios/lib.o.partial \
/gem5/default/build/X86/systemc/dt/fx/lib.o.partial \
/gem5/default/build/X86/mem/ruby/common/lib.o.partial \
/gem5/default/build/X86/mem/ruby/network/garnet2.0/lib.o.partial \
/gem5/default/build/X86/mem/ruby/structures/lib.o.partial \
/gem5/default/build/X86/cpu/testers/garnet_synthetic_traffic/lib.o.partial \
/gem5/default/build/X86/mem/cache/prefetch/lib.o.partial \
/gem5/default/build/X86/cpu/trace/lib.o.partial \
/gem5/default/build/X86/sim/probe/lib.o.partial \
/gem5/default/build/X86/sim/lib.o.partial \
/gem5/default/build/X86/mem/ruby/protocol/lib.o.partial \
/gem5/default/build/X86/systemc/tlm_core/2/quantum/lib.o.partial \
/gem5/default/build/X86/cpu/simple/lib.o.partial \
/gem5/default/build/X86/base/vnc/lib.o.partial \
/gem5/default/build/X86/mem/ruby/system/lib.o.partial \
/gem5/default/build/X86/mem/cache/lib.o.partial \
/gem5/default/build/X86/arch/x86/lib.o.partial \
/gem5/default/build/X86/dev/storage/lib.o.partial \
/gem5/default/build/X86/mem/protocol/lib.o.partial \
/gem5/default/build/X86/systemc/core/lib.o.partial \
/gem5/default/build/X86/systemc/tlm_core/2/generic_payload/lib.o.partial \
/gem5/default/build/X86/cpu/testers/directedtest/lib.o.partial \
/gem5/default/build/X86/mem/ruby/profiler/lib.o.partial \
/gem5/default/build/X86/arch/x86/regs/lib.o.partial \
/gem5/default/build/X86/dev/pci/lib.o.partial \
/gem5/default/build/X86/cpu/o3/probe/lib.o.partial \
/gem5/default/build/X86/mem/cache/compressors/lib.o.partial \
/gem5/default/build/X86/cpu/lib.o.partial \
/gem5/default/build/X86/learning_gem5/part2/lib.o.partial \
/gem5/default/build/X86/mem/cache/replacement_policies/lib.o.partial \
/gem5/default/build/X86/dev/virtio/lib.o.partial \
/gem5/default/build/X86/proto/lib.o.partial \
/gem5/default/build/X86/cpu/testers/rubytest/lib.o.partial \
/gem5/default/build/X86/mem/qos/lib.o.partial \
/gem5/default/build/X86/cpu/pred/lib.o.partial \
/gem5/default/build/X86/python/lib.o.partial \
/gem5/default/build/X86/arch/generic/lib.o.partial \
/gem5/default/build/X86/systemc/tlm_bridge/lib.o.partial \
/gem5/default/build/X86/dev/lib.o.partial \
/gem5/default/build/X86/kern/lib.o.partial \
/gem5/default/build/X86/mem/lib.o.partial \
/gem5/default/build/X86/cpu/testers/memtest/lib.o.partial \
/gem5/default/build/X86/systemc/dt/misc/lib.o.partial \
/gem5/default/build/X86/systemc/tlm_utils/lib.o.partial \
/gem5/default/build/X86/cpu/o3/lib.o.partial \
/gem5/default/build/X86/mem/ruby/network/lib.o.partial \
/gem5/default/build/X86/systemc/dt/bit/lib.o.partial \
/gem5/default/build/X86/dev/ps2/lib.o.partial \
/gem5/default/build/X86/unittest/lib.o.partial \
/gem5/default/build/X86/systemc/channel/lib.o.partial \
/gem5/default/build/X86/systemc/dt/lib.o.partial \
/gem5/default/build/X86/base/date.o \
-L/gem5/default/build/libelf -L/gem5/default/build/fputils -L/gem5/default/build/libfdt -L/gem5/default/build/drampower -L/gem5/default/build/iostream3 -L/gem5/default/build/nomali -L/gem5/default/build/googletest -L/usr/lib/x86_64-linux-gnu/hdf5/serial/ \
-lpython2.7 -lpthread -ldl -lutil -lm -lz -lprotobuf -lrt -ltcmalloc -lnomali -liostream3 \
-ldrampower -lfdt -lfputils -lelf -lhdf5_cpp -lhdf5
Related
I am using Cmake as below:
**build.sh**
cmake -S . -B $BUILD_DIR_PREFIX$BUILD_TYPE \
-DCMAKE_C_COMPILER=$C_COMPILER \
-DCMAKE_CXX_COMPILER=$CXX_COMPILER \
-DCMAKE_BUILD_TYPE=$BUILD_TYPE \
-GNinja \
-DCMAKE_CXX_STANDARD_LIBRARIES="-lcurl -lssl -lmylib" \
$SANITIZER \
${CMAKE_PARAMS[#]}
I have created a shared library mylib.so in path /home/sama/mylib.so
how can I link my shared library as curl or ssl?
I have done export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/sama/ before but still i get error /usr/bin/ld: cannot find -lmylib
I downloaded the file first using:
!curl -L -O https://github.com/huggingface/transformers/blob/master/examples/legacy/question-answering/run_squad.py
Then used following code:
!python run_squad.py \
--model_type bert \
--model_name_or_path bert-base-uncased \
--output_dir models/bert/ \
--data_dir data/squad \
--overwrite_output_dir \
--overwrite_cache \
--do_train \
--train_file /content/train.json \
--version_2_with_negative \
--do_lower_case \
--do_eval \
--predict_file /content/val.json \
--per_gpu_train_batch_size 2 \
--learning_rate 3e-5 \
--num_train_epochs 2.0 \
--max_seq_length 384 \
--doc_stride 128 \
--threads 10 \
--save_steps 5000
Also tried following:
!python run_squad.py \
--model_type bert \
--model_name_or_path bert-base-cased \
--do_train \
--do_eval \
--do_lower_case \
--train_file /content/train.json \
--predict_file /content/val.json \
--per_gpu_train_batch_size 12 \
--learning_rate 3e-5 \
--num_train_epochs 2.0 \
--max_seq_length 584 \
--doc_stride 128 \
--output_dir /content/
The error says in both the codes:
File "run_squad.py", line 7
^ SyntaxError: invalid syntax
What exactly is the issue? How can I run the .py file?
SOLVED: It was giving error because I was downloading the github link rather than the script in github. Once I copied and used 'Raw' link to download the script, the code ran.
I'm getting the next error with llc when trying to to link some IR files:
LLVM ERROR: Cannot select: intrinsic %llvm.objc.clang.arc.use
I discovered that if I disabled clang optimization (O0) during compilation the error doesn't pop up, but I don't want to do that
I attach a sample of how to reproduce it, just cd the 'issue' folder and run the 'issue.sh' to reproduce the error. You must have Xcode (12.x preferred) tu run it
https://drive.google.com/file/d/1j0TvMZI2A7QxRbv_Q7aydtlNYaiTPYBm/view?usp=sharing
This is what the issue.sh contains, that is a reduced version of my real implementation:
echo "clang..."
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -x objective-c \
-target arm64-apple-ios12.0 \
-fobjc-arc -fmodules \
-S -Os -flto=thin \
-isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS14.1.sdk \
-fembed-bitcode \
-c ${PWD}/GTMSessionFetcher.m \
-o ${PWD}/GTMSessionFetcher.ll
echo "llc..."
${PWD}/llc \
GTMSessionFetcher.ll \
-stats -filetype=obj \
-code-model=small \
-enable-machine-outliner=never \
--mtriple=arm64-apple-ios12.0 \
-o ${PWD}/GTMSessionFetcher.o
As I mentioned before, if you change -Os for -O0, the error disappears, but I do need the size optimization.
The objective-c file that I included is a file from a pod from a Google library
https://github.com/google/gtm-session-fetcher
I found a similar question, but it doesn't have any answer yet
LLVM Error Cannot select intrinsic %llvm.coro.begin
I thought it was bug, but it wasn't
As Akira pointed me in LLVM bug report I was able to fix it by adding an intermediate optimizer OPT step with -objc-arc-contract flag:
clang
...
<path to your LLVM tools>/OPT \
<Your IR file> \
-objc-arc-contract \
-o <Output file>
llc
...
I have a script for building a project that I need to upgrade from using configure to cmake. The original configure command is
CFLAGS="$SLKCFLAGS" \
CXXFLAGS="$SLKCFLAGS" \
./configure \
--with-clang \
--prefix=$PREFIX \
--libdir=$PREFIX/lib${LIBDIRSUFFIX} \
--incdir=$PREFIX/include \
--mandir=$PREFIX/man/man1 \
--etcdir=$PREFIX/etc/root \
--docdir=/usr/doc/$PRGNAM-$VERSION \
--enable-roofit \
--enable-unuran \
--disable-builtin-freetype \
--disable-builtin-ftgl \
--disable-builtin-glew \
--disable-builtin-pcre \
--disable-builtin-zlib \
--disable-builtin-lzma \
$GSL_FLAGS \
$FFTW_FLAGS \
$QT_FLAGS \
--enable-shared \
--build=$ARCH-slackware-linux
I am not familiar enough with cmake to know how to do the equivalent. I would prefer a command line option but am open to modifying the CMakeLists.txt file as well.
PS: Question is at the end, the following is just background information that might help.
I'm trying to compile tensorflow, but getting this error:
bazel-out/host/bin/_solib_local/_U#local_Uconfig_Usycl_S_Ssycl_Csyclrt___Uexternal_Slocal_Uconfig_Usycl_Ssycl_Slib/libComputeCpp.so:
undefined reference to `clSetUserEventStatus#OPENCL_1.1'
By using "strace" tool, i've isolated the exact statement that's failing to be:
(
cd /home/rh/.cache/bazel/_bazel_rh/81919f16ea125cb9f08993f06569f022/execroot/tensorflow && \
exec env - PATH=/home/rh/.nvm/versions/node/v6.9.5/bin:/home/rh/perl5/bin:/home/rh/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/rh/bin:/usr/local/java/jdk1.8.0_74/bin:/home/rh/.local/bin:/home/rh/myscripts \
/usr/bin/clang++-3.6 \
-o \
bazel-out/host/bin/tensorflow/python/gen_functional_ops_py_wrappers_cc \
-Lbazel-out/host/bin/_solib_local/_U#local_Uconfig_Usycl_S_Ssycl_Csyclrt___Uexternal_Slocal_Uconfig_Usycl_Ssycl_Slib \
-Wl,-rpath,$ORIGIN/../../_solib_local/_U#local_Uconfig_Usycl_S_Ssycl_Csyclrt___Uexternal_Slocal_Uconfig_Usycl_Ssycl_Slib \
-pthread \
-Wl,-no-as-needed \
-B/usr/bin/ \
-Wl,-no-as-needed \
-Wl,--build-id=md5 \
-Wl,--hash-style=gnu \
-Wl,-S \
-Wl,#bazel-out/host/bin/tensorflow/python/gen_functional_ops_py_wrappers_cc-2.params \
-Wl,--no-undefined
)
strace also confirms that this command IS reading /usr/local/computecpp/lib/libComputeCpp.so, which contains the aforementioned clSetUserEventStatus symbol.
I verified that libComputeCpp.so contains clSetUserEventStatus by checking the output of the nm command:
nm /usr/local/computecpp/lib/libComputeCpp.so | grep clSetUserEventStatus
>> U clSetUserEventStatus##OPENCL_1.1
So here is my question: clang (and/or its linker) is reading the file that contains the symbol that it's complaining about... why is it "missing something" and complaining that it's undefined?
How can I get this compile to work?