I am new to Lua.
Several days ago, I complished using python in ExecuteScript of NIFI. I set Python Module path to /usr/local/lib/python2.7/dist-packages. Everything goes well.
But doing the same thing with Lua is so difficult for me. Always Error of cannot find the Module!!!
I use Luarock to install modules.
Could you please tell me how to set the lua module path or some useful infomation about it?
Here comes some info of my Luarocks setting:
lbh#es-2:~/install/nifi-1.1.1/lua_modules$ luarocks list
Installed rocks:
----------------
lua-cjson
2.1.0-1 (installed) - /usr/local/lib/luarocks/rocks
luasocket
3.0rc1-2 (installed) - /usr/local/lib/luarocks/rocks
redis-lua
2.0.4-1 (installed) - /usr/local/lib/luarocks/rocks
An example of lua-cjson.
lbh#es-2:~/install/nifi-1.1.1/lua_modules$ luarocks show lua-cjson
lua-cjson 2.1.0-1 - A fast JSON encoding/parsing module
The Lua CJSON module provides JSON support for Lua. It features: - Fast,
standards compliant encoding/parsing routines - Full support for JSON with
UTF-8, including decoding surrogate pairs - Optional run-time support for
common exceptions to the JSON specification (infinity, NaN,..) - No
dependencies on other libraries
License: MIT
Homepage: http://www.kyne.com.au/~mark/software/lua-cjson.php
Installed in: /usr/local
Modules:
cjson
lua2json
json2lua
cjson.util
Directory info of /usr/local/lib/luarocks/rocks
lbh#es-2:~/install/nifi-1.1.1/lua_modules$ ls -l /usr/local/lib/luarocks/rocks
total 16
drwxr-xr-x 3 root root 4096 Mar 17 11:13 lua-cjson
drwxr-xr-x 3 root root 4096 Mar 17 11:18 luasocket
-rw-r--r-- 1 root root 3653 Mar 17 11:18 manifest
drwxr-xr-x 3 root root 4096 Mar 17 11:18 redis-lua
Related
We are facing an issue on the Databrick filesyste that considers files as directory and we are unable to read files with Pandas. The files exist in the Azure Storage Explorer, and are considered as files as seen here :
We have mounted the storage with oAuth 2.0.
On Databricks,
%sh ls -al '<path_to_files>'
returns the following :
total 1127
drwxrwxrwx 2 root root 4096 Jan 29 09:26 .
drwxrwxrwx 2 root root 4096 Jan 9 13:47 ..
drwxrwxrwx 1 root root 136705 Jan 28 16:35 AAAA_2019-10-01_2019-12-27.csv
drwxrwxrwx 1 root root 183098 Jan 28 16:35 BBBB_2019-10-01_2019-12-27.csv
-rwxrwxrwx 1 root root 313120 Jan 28 16:35 CCCC_2019-10-01_2019-12-27.csv
-rwxrwxrwx 1 root root 212935 Jan 29 09:26 df_cube.csv
-rwxrwxrwx 1 root root 298228 Jan 29 09:26 df_other_cube.csv
The thing is, the two first csv files are not directories at all. We can download them and read them as csv, but we cannot load them into a Pandas dataframe.
df = pd.read_csv(rootname_source_test + r'AAAA_2019-10-01_2019-12-27.csv',header=0,sep="|",engine='python')
>>> IsADirectoryError: [Errno 21] Is a directory: '/dbfs/mnt/<path>/AAA_2019-10-01_2019-12-27.csv'
They are generated the same way the 3rd csv is generated, and the 3rd on is loadable in pandas. Sometimes they appear as files, sometimes as directories and we are having trouble recreating and solving this consistently.
Cluster config : Runtime 6.2 ML (includes Apache Spark 2.4.4, Scala 2.11)
Any help will be very appreciated.
Currently I am creating a simple project which will install a utility shared library.
Here's my CMakeLists:
cmake_minimum_required (VERSION 2.6)
project(MathLibs CXX)
add_library (${PROJECT_NAME} SHARED
fact.cpp
fibo.cpp
isPrime.cpp
)
install (TARGETS ${PROJECT_NAME}
RUNTIME DESTINATION ${PROJECT_NAME}/bin
LIBRARY DESTINATION ${PROJECT_NAME}/lib
ARCHIVE DESTINATION ${PROJECT_NAME}/lib)
Since I do not have root privileges I cannot install the shared library in system lib folder. I override the CMAKE_INSTALL_PREFIX to $HOME/apps.
When I build the shared library it had the executable permissions.
Here's the build folder with the shared library:
-rw-rw-r-- 1 ameya ameya 9714 Jun 18 20:02 CMakeCache.txt
drwxrwxr-x 5 ameya ameya 4096 Jun 18 20:02 CMakeFiles
-rw-rw-r-- 1 ameya ameya 2701 Jun 18 20:02 cmake_install.cmake
-rw-rw-r-- 1 ameya ameya 84 Jun 18 20:02 install_manifest.txt
-rwxrwxr-x 1 ameya ameya 6808 Jun 18 20:02 libMathLibs.so
-rw-rw-r-- 1 ameya ameya 7748 Jun 18 20:02 Makefile
drwxrwxr-x 3 ameya ameya 4096 Jun 18 20:02 test
After installing the executable permissions disappears.
Here's the install folder location:
-rw-r--r-- 1 ameya ameya 6808 Jun 18 20:02 libMathLibs.so
What am I missing in the CMakeLists.txt to correct this?
They said that CMake doesn't set execute permissions on installed library because on Linux libraries don't need to be executable.
As for library's permissions in build tree, these are set not by CMake but by the linker.
If you want executable permissions of installed library for some reason, use PERMISSIONS option in install() command.
After looking for more details online I found this referenced in the CMake bugs report.
The handling of shared library on different systems is different I tried using Ubuntu and a Fedora workstation.
On Ubuntu system the system installed shared libraries do not have the executable bit set, but on the Fedora Workstation the same library had the executable bit set.
One can have look at the ${CMAKE_ROOT}/cmake/Modules/Platform/Linux.cmake,
which has the CMAKE_INSTALL_SO_NO_EXE macro defined(sorry for the typo in my earlier reply).
Flag "-fprofile-instr-use" generates error given below.
This issue occurs even if we build llvm,clang and compiler-rt using cmake or configure.
Please let me know your inputs to resolve this issue
error: Could not read profile: Invalid file format (bad magic)
Thanks,
Steps to reproduce this issue:
$ clang -O2 -fprofile-instr-generate hello.c -o c1.out
$ ls -rlt
-rw-r--r-- 1 root root 70 Jul 11 10:10 hello.c
-rwxr-xr-x 1 root root 15793 Jul 11 10:10 c1.out
-rw-r--r-- 1 root root 12203204 Jul 11 10:10 gmon.out
$ ./c1.out
Hello world
$ ls -rlt
-rw-r--r-- 1 root root 70 Jul 11 10:10 hello.c
-rwxr-xr-x 1 root root 15793 Jul 11 10:10 c1.out
-rw-r--r-- 1 root root 12203204 Jul 11 10:10 gmon.out
-rw-r--r-- 1 root root 104 Jul 11 10:10 default.profraw
$ clang -O2 -fprofile-instr-use=default.profraw hello.c -o c2.out
error: Could not read profile: Invalid file format (bad magic)
1 error generated.
Clang version (July 10th-2014 build from stage):
$ clang -v
clang version 3.5.0 (llvm.org/git/clang.git 5f9d646cba20f309bb69c6c358996d71912c54cd) (llvm.org/git/llvm.git dc90a3ab8ffc841a442888940635306de6131d2f)
Target: x86_64-unknown-linux-gnu
Thread model: posix
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.8
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.8.2
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9.0
Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.8
Candidate multilib: .;#m64
Candidate multilib: 32;#m32
Selected multilib: .;#m64
OS: Ubuntu 14.04
LLVM configure: ../llvm/configure --enable-profiling --enable-optimized --enable-shared --disable-debug-runtime --enable-targets=x86
It turns out that step 3 outlined here: http://clang.llvm.org/docs/UsersManual.html#profiling-with-instrumentation
is required even if you only have 1 output file you are using. "Combine profiles from multiple runs and convert the “raw” profile format to the input expected by clang" makes it sound like you should only do this if you have multiple profiles, but you need to do it unconditionally.
So I just installed Arch and most things are working fine, but when I try to use pacman or curl, I get the error:
pacman: error while loading shared libraries: /usr/lib/libcurl.so.4: invalid ELF header
Also, I can't seem to run anything pacman-related for now... not even a pacman --help
Not sure if useful, but ls -l /usr/lib | grep libcurl gives:
-rw-r--r-- 1 root root 594016 Jun 22 12:21 libcurl.a
lrwxrwxrwx 1 root root 16 Jun 22 12:21 libcurl.so -> libcurl.so.4.3.0
lrwxrwxrwx 1 root root 16 Jun 22 12:21 libcurl.so.4 -> libcurl.so.4.3.0
-rwxr-xr-x 1 root root 408324 Jun 22 12:21 libcurl.s0.4.3.0
Thanks in advance!
Update: running ./curl-config gives the error, "cannot execute binary file". This makes me wonder if maybe I have a 64 bit version, whilst I'm running Arch i686. What is the best way to handle this?
maybe I have a 64 bit version, whilst I'm running Arch i686
That would do it. Run file ./curl-config. If it says ELF 64-bit LSB executable,... reinstall curl from correct packages.
I am running Centos 5.8 on a production server. I have an application that needs to use the keyctl command, but everytime the app calls (or I call) the command, I have some errors.
The first error was this:
root#server [~] keyctl show
segmentation fault
Then, I re-installed the keyutils binaries using yum. These are the keyutils packages I have on the server:
root#server [~]# rpm -qa | grep keyutils
keyutils-libs-1.2-1.el5
keyutils-libs-1.2-1.el5
keyutils-1.2-1.el5
keyutils-libs-devel-1.2-1.el5
And now, I have another different error:
root#server [~]# keyctl show
keyctl: symbol lookup error: /lib64/libkeyutils.so.1: undefined symbol: dlopen
I checked the libraries of keyctl, and libdl is not there.
root#server [~]# ldd /bin/keyctl
linux-vdso.so.1 => (0x00007fffcc5fd000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00000033df000000)
libc.so.6 => /lib64/libc.so.6 (0x0000003d7ae00000)
/lib64/ld-linux-x86-64.so.2 (0x0000003d7aa00000)
All libraries that uses are fine.
root#server [~]# ls -al /lib64/libkeyutils*
-rwxr-xr-x 1 root root 9472 Jan 6 2007 /lib64/libkeyutils-1.2.so*
lrwxrwxrwx 1 root root 18 Nov 21 07:56 /lib64/libkeyutils.so.1 -> libkeyutils.so.1.9*
-rwxr-xr-x 1 root root 34584 Jan 6 2007 /lib64/libkeyutils.so.1.9*
root#server [~]# ls -al /lib64/libdl*
-rwxr-xr-x 1 root root 23360 Aug 27 08:59 /lib64/libdl-2.5.so*
lrwxrwxrwx 1 root root 12 Nov 16 02:01 /lib64/libdl.so.2 -> libdl-2.5.so*
root#server [~]#
Have you ever seen this problem before? I tried run the same version on others distros and it works.
I would like to re-install this server, but I can't because it is a production server.
Is there a way I can add or link a shared library to a binary already linked to others .so libraries?
Look at: http://blog.solidshellsecurity.com/2013/02/08/sshd-spam-rootkit-lib64libkeyutils-so-1-9/
It appears that there is no such legitimate file as libkeyutils.so.1.9
It is a rootkit, the latest legitimate version of this library is libkeyutils.so.1.3 on CentOS 6.3 (final).
rm -f /lib64/libkeyutils.so.1.9
ldconfig
/etc/init.d/sshd restart
There's also a suspected (as of now) unpatched user escalation priviledge flaw in all CentOS and RedHat kernels: https://access.redhat.com/security/cve/CVE-2013-0871 and http://blog.configserver.com/index.php?itemid=716
You may also need to reinstall SSH:
https://serverfault.com/questions/476954/remove-shared-library-from-sshd
https://forums.cpanel.net/f185/sshd-rootkit-323962.html
LD_PRELOAD=/lib64/libdl-2.5.so keyctl show