ERROR : Called singularity_config_get_value on uninitialized config subsystem when building Singularity container - singularity-container

I tried to build a singularity container from Singularity Hub as such:
sudo singularity pull shub://marcc-hpc/pytorch
And I got
ERROR : Called singularity_config_get_value on uninitialized config subsystem.
So, I copy pasted the recipe into a file called 'recipe', and executed
sudo singularity build --sandbox myimage recipe
It hung forever at 33.3% done, as shown here:
Using container recipe deffile: recipe
Sanitizing environment
Adding base Singularity environment to container
tar: ./.exec: implausibly old time stamp -9223372036854775808
tar: ./.run: implausibly old time stamp -9223372036854775808
tar: ./.shell: implausibly old time stamp -9223372036854775808
tar: ./.singularity.d/actions/exec: implausibly old time stamp -9223372036854775808
tar: ./.singularity.d/actions/run: implausibly old time stamp -9223372036854775808
tar: ./.singularity.d/actions/shell: implausibly old time stamp -9223372036854775808
tar: ./.singularity.d/actions/start: implausibly old time stamp -9223372036854775808
tar: ./.singularity.d/actions/test: implausibly old time stamp -9223372036854775808
tar: ./.singularity.d/actions: implausibly old time stamp -9223372036854775808
tar: ./.singularity.d/env/01-base.sh: implausibly old time stamp -9223372036854775808
tar: ./.singularity.d/env/90-environment.sh: implausibly old time stamp -9223372036854775808
tar: ./.singularity.d/env/95-apps.sh: implausibly old time stamp -9223372036854775808
tar: ./.singularity.d/env/99-base.sh: implausibly old time stamp -9223372036854775808
tar: ./.singularity.d/env: implausibly old time stamp -9223372036854775808
tar: ./.singularity.d/libs: implausibly old time stamp -9223372036854775808
tar: ./.singularity.d/runscript: implausibly old time stamp -9223372036854775808
tar: ./.singularity.d/startscript: implausibly old time stamp -9223372036854775808
tar: ./.singularity.d: implausibly old time stamp -9223372036854775808
tar: ./.test: implausibly old time stamp -9223372036854775808
tar: ./dev: implausibly old time stamp -9223372036854775808
tar: ./environment: implausibly old time stamp -9223372036854775808
tar: ./etc/hosts: implausibly old time stamp -9223372036854775808
tar: ./etc/resolv.conf: implausibly old time stamp -9223372036854775808
tar: ./etc: implausibly old time stamp -9223372036854775808
tar: ./home: implausibly old time stamp -9223372036854775808
tar: ./proc: implausibly old time stamp -9223372036854775808
tar: ./root: implausibly old time stamp -9223372036854775808
tar: ./singularity: implausibly old time stamp -9223372036854775808
tar: ./sys: implausibly old time stamp -9223372036854775808
tar: ./tmp: implausibly old time stamp -9223372036854775808
tar: ./var/tmp: implausibly old time stamp -9223372036854775808
tar: ./var: implausibly old time stamp -9223372036854775808
tar: .: implausibly old time stamp -9223372036854775808
Docker image path: index.docker.io/marcchpc/pytorch_cuda9:latest
Cache folder set to /root/.singularity/docker
[7/21] |===========|-----------------------| 33.3%
Then, I figured I'd try to just get the hello world container:
singularity pull --name meatballs.img shub://vsoch/hello-world
...and I got the same error message.
Does anyone know what this error means, and why I can't get this running?

I've found the solution: the singularity version used is too old!
I got this version, which is too old:
$ singularity --version
2.4.2-dist
Download this script with content:
#!/bin/bash
VERSION=2.5.2
wget https://github.com/singularityware/singularity/releases/download/$VERSION/singularity-$VERSION.tar.gz
tar xvf singularity-$VERSION.tar.gz
cd singularity-$VERSION
./configure --prefix=/usr/local
make
sudo make install
calling it:
sudo ./install_singularity
to get to v2.5.2
$ singularity --version
2.5.2-dist
and everything works as expected.

Same problem here. I notice if I run
sudo singularity build hello_world.simg shub://singularityhub/ubuntu
on Travis (see log) everythings works as expected:
Cache folder set to /root/.singularity/shub
Progress |===================================| 100.0%
Building from local image: /root/.singularity/shub/singularityhub-ubuntu-master-latest.simg
Building Singularity image...
Singularity container built: hello_world.simg
Cleaning up...
Running it locally, however, results in different behavior:
richel#sonic:~/GitHubs/singularity_example_1$ sudo singularity build hello_world.simg shub://singularityhub/ubuntu
[sudo] password for richel:
Cache folder set to /root/.singularity/shub
Progress |===================================| 100.0%
ERROR : Called singularity_config_get_value on uninitialized config subsystem
ABORT : Retval = 255
ERROR: Unsupported file type: /root/.singularity/shub/singularityhub-ubuntu-master-latest.simg
Cleaning up...
I know the versions between the one on Travis (2.5.2-HEAD.ddf3e31) and on my local computer (2.4.2-dist) differ, but I think that should be no problem [edit: I was wrong, see answer].
I hope this helps in finding the solution.
OS:
richel#sonic:~/GitHubs/singularity_example_1$ cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04.2 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.2 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

Related

How did you install nvm?

I wasn't able to install using the curl command.
Here's the transcript:
$ curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.2/install.sh | bash
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 13527 100 13527 0 0 13527 0 0:00:01 --:--:-- 0:00:01 48138
=> Downloading nvm from git to '/cygdrive/c/MikesStuff/Pers/Dropbox/Personal/home/.nvm'
=> Cloning into '/cygdrive/c/MikesStuff/Pers/Dropbox/Personal/home/.nvm'...
remote: Enumerating objects: 288, done.
remote: Counting objects: 100% (288/288), done.
remote: Compressing objects: 100% (258/258), done.
Receiving objects: 87% (251/288)sed 95 (delta 18), pack-reused 0Receiving objects: 8
Receiving objects: 100% (288/288), 146.70 KiB | 2.72 MiB/s, done.
Resolving deltas: 100% (35/35), done.
error: invalid path 'test/fast/Aliases/"nvm alias" should not accept aliases with slashes'
fatal: unable to checkout working tree
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'
Failed to clone nvm repo. Please report this!

Symlink error in google colab while installing meshroom

After running below command
!wget -N https://github.com/alicevision/meshroom/releases/download/v2019.2.0/Meshroom-2019.2.0-linux.tar.gz
!mkdir meshroom
!tar xzf Meshroom-2019.2.0-linux.tar.gz -C ./meshroom
I get the below error in google colab please provide an alternative or solution for it.
Meshroom-2019.2.0-l 100%[===================>] 409.38M 14.1MB/s in 32s
2019-12-14 09:50:21 (12.6 MB/s) - ‘Meshroom-2019.2.0-linux.tar.gz’ saved [429271659/429271659]
tar: Meshroom-2019.2.0/qtPlugins/qml/AliceVision/libqtAliceVisionPlugin.so: Cannot create symlink to ‘libqtAliceVisionPlugin.so.0’: Operation not supported
tar: Meshroom-2019.2.0/qtPlugins/qml/AliceVision/libqtAliceVisionPlugin.so.0: Cannot create symlink to ‘libqtAliceVisionPlugin.so.0.1’: Operation not supported
tar: Meshroom-2019.2.0/qtPlugins/qml/AlembicEntity/libalembicEntityQmlPlugin.so: Cannot create symlink to ‘libalembicEntityQmlPlugin.so.0’: Operation not supported
tar: Meshroom-2019.2.0/qtPlugins/qml/AlembicEntity/libalembicEntityQmlPlugin.so.0: Cannot create symlink to ‘libalembicEntityQmlPlugin.so.0.1’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_exportPMVS: Cannot create symlink to ‘aliceVision_exportPMVS-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_cameraInit: Cannot create symlink to ‘aliceVision_cameraInit-2.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_computeStructureFromKnownPoses: Cannot create symlink to ‘aliceVision_computeStructureFromKnownPoses-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_convertLDRToHDR: Cannot create symlink to ‘aliceVision_convertLDRToHDR-0.1’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_globalSfM: Cannot create symlink to ‘aliceVision_globalSfM-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_utils_sfmLocalization: Cannot create symlink to ‘aliceVision_utils_sfmLocalization-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_meshFiltering: Cannot create symlink to ‘aliceVision_meshFiltering-3.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_cameraCalibration: Cannot create symlink to ‘aliceVision_cameraCalibration-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_utils_sfmTransform: Cannot create symlink to ‘aliceVision_utils_sfmTransform-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_imageMatching: Cannot create symlink to ‘aliceVision_imageMatching-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_utils_sfmColorHarmonize: Cannot create symlink to ‘aliceVision_utils_sfmColorHarmonize-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_exportMVSTexturing: Cannot create symlink to ‘aliceVision_exportMVSTexturing-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_utils_voctreeQueryUtility: Cannot create symlink to ‘aliceVision_utils_voctreeQueryUtility-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_exportTracks: Cannot create symlink to ‘aliceVision_exportTracks-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_rigCalibration: Cannot create symlink to ‘aliceVision_rigCalibration-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_incrementalSfM: Cannot create symlink to ‘aliceVision_incrementalSfM-2.1’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_utils_qualityEvaluation: Cannot create symlink to ‘aliceVision_utils_qualityEvaluation-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_exportMatlab: Cannot create symlink to ‘aliceVision_exportMatlab-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_meshResampling: Cannot create symlink to ‘aliceVision_meshResampling-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_featureMatching: Cannot create symlink to ‘aliceVision_featureMatching-2.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_exportMVE2: Cannot create symlink to ‘aliceVision_exportMVE2-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_utils_frustumFiltering: Cannot create symlink to ‘aliceVision_utils_frustumFiltering-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_meshDecimate: Cannot create symlink to ‘aliceVision_meshDecimate-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_utils_voctreeStatistics: Cannot create symlink to ‘aliceVision_utils_voctreeStatistics-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_convertSfMFormat: Cannot create symlink to ‘aliceVision_convertSfMFormat-2.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_cameraLocalization: Cannot create symlink to ‘aliceVision_cameraLocalization-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_exportMatches: Cannot create symlink to ‘aliceVision_exportMatches-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_depthMapFiltering: Cannot create symlink to ‘aliceVision_depthMapFiltering-2.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_prepareDenseScene: Cannot create symlink to ‘aliceVision_prepareDenseScene-2.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_exportMeshroomMaya: Cannot create symlink to ‘aliceVision_exportMeshroomMaya-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_rigLocalization: Cannot create symlink to ‘aliceVision_rigLocalization-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_utils_split360Images: Cannot create symlink to ‘aliceVision_utils_split360Images-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_exportMeshlab: Cannot create symlink to ‘aliceVision_exportMeshlab-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_utils_sfmAlignment: Cannot create symlink to ‘aliceVision_utils_sfmAlignment-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_exportColoredPointCloud: Cannot create symlink to ‘aliceVision_exportColoredPointCloud-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_utils_voctreeCreation: Cannot create symlink to ‘aliceVision_utils_voctreeCreation-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_texturing: Cannot create symlink to ‘aliceVision_texturing-3.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_exportCameraFrustums: Cannot create symlink to ‘aliceVision_exportCameraFrustums-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_exportAnimatedCamera: Cannot create symlink to ‘aliceVision_exportAnimatedCamera-2.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_meshing: Cannot create symlink to ‘aliceVision_meshing-3.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_utils_rigTransform: Cannot create symlink to ‘aliceVision_utils_rigTransform-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_exportKeypoints: Cannot create symlink to ‘aliceVision_exportKeypoints-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_convertFloatDescriptorToUchar: Cannot create symlink to ‘aliceVision_convertFloatDescriptorToUchar-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_depthMapEstimation: Cannot create symlink to ‘aliceVision_depthMapEstimation-2.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_convertRAW: Cannot create symlink to ‘aliceVision_convertRAW-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_utils_keyframeSelection: Cannot create symlink to ‘aliceVision_utils_keyframeSelection-2.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_meshDenoising: Cannot create symlink to ‘aliceVision_meshDenoising-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/bin/aliceVision_featureExtraction: Cannot create symlink to ‘aliceVision_featureExtraction-1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libgfortran.so.3: Cannot create symlink to ‘libgfortran.so.3.0.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_multiview.so.2: Cannot create symlink to ‘libaliceVision_multiview.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libcolamd.so.2: Cannot create symlink to ‘libcolamd.so.2.9.6’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libgomp.so.1: Cannot create symlink to ‘libgomp.so.1.0.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/liblapack.so.3: Cannot create symlink to ‘liblapack.so.3.8.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libopencv_imgcodecs.so.4.1: Cannot create symlink to ‘libopencv_imgcodecs.so.4.1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libopencv_imgproc.so.4.1: Cannot create symlink to ‘libopencv_imgproc.so.4.1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_gpu.so.2: Cannot create symlink to ‘libaliceVision_gpu.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libceres.so.1: Cannot create symlink to ‘libceres.so.1.14.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libIlmThread-2_3.so.24: Cannot create symlink to ‘libIlmThread-2_3.so.2.3.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_sfmData.so.2: Cannot create symlink to ‘libaliceVision_sfmData.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_matching.so.2: Cannot create symlink to ‘libaliceVision_matching.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libraw_r.so.19: Cannot create symlink to ‘libraw_r.so.19.0.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_kvld.so.2: Cannot create symlink to ‘libaliceVision_kvld.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libIlmImf-2_3.so.24: Cannot create symlink to ‘libIlmImf-2_3.so.2.3.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libcamd.so.2: Cannot create symlink to ‘libcamd.so.2.4.6’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libtiff.so.5: Cannot create symlink to ‘libtiff.so.5.4.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libsuitesparseconfig.so.4: Cannot create symlink to ‘libsuitesparseconfig.so.4.5.6’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_localization.so.2: Cannot create symlink to ‘libaliceVision_localization.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libz.so.1: Cannot create symlink to ‘libz.so.1.2.11’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libAlembic.so.1.7: Cannot create symlink to ‘libAlembic.so.1.7.11’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_numeric.so.2: Cannot create symlink to ‘libaliceVision_numeric.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libopencv_videoio.so.4.1: Cannot create symlink to ‘libopencv_videoio.so.4.1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libHalf-2_3.so.24: Cannot create symlink to ‘libHalf-2_3.so.2.3.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_fuseCut.so.2: Cannot create symlink to ‘libaliceVision_fuseCut.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_lInftyComputerVision.so.2: Cannot create symlink to ‘libaliceVision_lInftyComputerVision.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_sensorDB.so.2: Cannot create symlink to ‘libaliceVision_sensorDB.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libccolamd.so.2: Cannot create symlink to ‘libccolamd.so.2.9.6’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libspqr.so.2: Cannot create symlink to ‘libspqr.so.2.0.8’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_depthMap.so.2: Cannot create symlink to ‘libaliceVision_depthMap.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libopencv_calib3d.so.4.1: Cannot create symlink to ‘libopencv_calib3d.so.4.1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_mvsData.so.2: Cannot create symlink to ‘libaliceVision_mvsData.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_feature.so.2: Cannot create symlink to ‘libaliceVision_feature.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_rig.so.2: Cannot create symlink to ‘libaliceVision_rig.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_colorHarmonization.so.2: Cannot create symlink to ‘libaliceVision_colorHarmonization.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_sfmDataIO.so.2: Cannot create symlink to ‘libaliceVision_sfmDataIO.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_keyframe.so.2: Cannot create symlink to ‘libaliceVision_keyframe.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libopencv_xfeatures2d.so.4.1: Cannot create symlink to ‘libopencv_xfeatures2d.so.4.1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_matchingImageCollection.so.2: Cannot create symlink to ‘libaliceVision_matchingImageCollection.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_calibration.so.2: Cannot create symlink to ‘libaliceVision_calibration.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libjpeg.so.62: Cannot create symlink to ‘libjpeg.so.62.2.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libgeogram.so.1: Cannot create symlink to ‘libgeogram.so.1.6.6’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libblas.so.3: Cannot create symlink to ‘libblas.so.3.8.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libquadmath.so.0: Cannot create symlink to ‘libquadmath.so.0.0.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libIex-2_3.so.24: Cannot create symlink to ‘libIex-2_3.so.2.3.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_sfm.so.2: Cannot create symlink to ‘libaliceVision_sfm.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libpng16.so.16: Cannot create symlink to ‘libpng16.so.16.34.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libcudart.so.7.0: Cannot create symlink to ‘libcudart.so.7.0.28’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libcublas.so.7.0: Cannot create symlink to ‘libcublas.so.7.0.28’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libImath-2_3.so.24: Cannot create symlink to ‘libImath-2_3.so.2.3.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libamd.so.2: Cannot create symlink to ‘libamd.so.2.4.6’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libgeogram_num_3rdparty.so.1: Cannot create symlink to ‘libgeogram_num_3rdparty.so.1.6.6’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_system.so.2: Cannot create symlink to ‘libaliceVision_system.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_mvsUtils.so.2: Cannot create symlink to ‘libaliceVision_mvsUtils.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libopencv_video.so.4.1: Cannot create symlink to ‘libopencv_video.so.4.1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libgeogram_num_3rdparty.so: Cannot create symlink to ‘libgeogram_num_3rdparty.so.1’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_hdr.so.2: Cannot create symlink to ‘libaliceVision_hdr.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_track.so.2: Cannot create symlink to ‘libaliceVision_track.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libopencv_features2d.so.4.1: Cannot create symlink to ‘libopencv_features2d.so.4.1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libcholmod.so.3: Cannot create symlink to ‘libcholmod.so.3.0.11’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libOpenImageIO.so.2.0: Cannot create symlink to ‘libOpenImageIO.so.2.0.10’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libopencv_flann.so.4.1: Cannot create symlink to ‘libopencv_flann.so.4.1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_image.so.2: Cannot create symlink to ‘libaliceVision_image.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_mesh.so.2: Cannot create symlink to ‘libaliceVision_mesh.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libopencv_core.so.4.1: Cannot create symlink to ‘libopencv_core.so.4.1.0’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_dataio.so.2: Cannot create symlink to ‘libaliceVision_dataio.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_voctree.so.2: Cannot create symlink to ‘libaliceVision_voctree.so.2.2’: Operation not supported
tar: Meshroom-2019.2.0/aliceVision/lib/libaliceVision_geometry.so.2: Cannot create symlink to ‘libaliceVision_geometry.so.2.2’: Operation not supported
tar: Exiting with failure status due to previous errors
I tried some possible solutions but here i am directly using my drive to do the task.
I don't know any details like this to eradicate such problems it has taken a lot of time to get my hands overit please look into the error i used the opensource meshroom tool for 3d image but results are not getting out.
I am sorry to write such things but stackoverflow is not allowing me to submit my question as it thinks there are no details here and only the messy code is present which is clueless.
Anyways solution for this please.

Error attempting CrashPlan Home restore using PlanC - "Failed to open block manifest for reading"

Background to Plan C
Code42 decided to terminate their "CrashPlan for Home" service. This means that after the shutdown date of October 22, 2018, CrashPlan will delete your backup on their servers, which is to be expected, but much more annoyingly, you will no longer be able to restore CrashPlan backups that you stored locally. Effectively, Code42 is reaching into your computer to break your backups for you.
PlanC is an open source project to enable restore from existing CrashPlan Home backups to be performed.
My Problem
However, when attempting to restore I received an error:
MacBook-Pro:CrashPlanHomeRecovery daniel$ ./plan-c-osx/plan-c --key 07B... --archive ./sg2015/642033544161964565/ --dest ./recovered/ --filename "J:/..." restore
Caching block indexes in memory...
libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: Failed to open block manifest for reading: ./sg2015/642033544161964565/cpbf0000000000017581637/cpbmf
Abort trap: 6
The file referenced in the error appears to read OK, but the reported error provided no more information.
I reported this GitHub Issue #9.
I then made a minor change to the error reporting GitHub Pull Request #10 to work out that the error was a Too many open files error:
MacBook-Pro:CrashPlanHomeRecovery daniel$ ./plan-c-osx/plan-c --key 07B... --archive ./sg2015/642033544161964565/ --dest ./recovered/ --filename "J:/..." restore
Caching block indexes in memory...
libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: Failed to open block manifest (../../sg2015/642033544161964565/cpbf0000000000017581637/cpbmf) for reading: Too many open files
Abort trap: 6
Just a note that if my pull request (only just submitted) is not merged (and a new binary released) you will need to build from my fork.
Which I then fixed with a ulimit change:
MacBook-Pro:PlanC daniel$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 256
pipe size (512 bytes, -p) 1
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 1418
virtual memory (kbytes, -v) unlimited
by increasing number of open files for the shell to 1024:
MacBook-Pro:PlanC daniel$ ulimit -S -n 1024
Recording this answer in case others have problems - backups are important after all :)

Hiveserver2 does not start after installing HDP 2.6.4.0-91 using cloudbreak on AWS

Hiveserver2 does not start after installing HDP 2.6.4.0-91 using cloudbreak on AWS.
Start the hiveserver2 in the Ambari UI and check the contents of /var/log/hive/hiveserver2.log.
Below is the error log.
Any help would be appreciated.
Contents of hiveserver2.log
2018-03-08 04:41:53,345 WARN [main-EventThread]: server.HiveServer2 (HiveServer2.java:process(343)) - This instance of HiveServer2 has been removed from the list of server instances available for dynamic service discovery. The last client session has ended - will shutdown now.
2018-03-08 04:41:53,347 INFO [main]: zookeeper.ZooKeeper (ZooKeeper.java:close(684)) - Session: 0x16203aad5af0040 closed
2018-03-08 04:41:53,347 INFO [main]: server.HiveServer2 (HiveServer2.java:removeServerInstanceFromZooKeeper(361)) - Server instance removed from ZooKeeper.
2018-03-08 04:41:53,348 INFO [main-EventThread]: server.HiveServer2 (HiveServer2.java:stop(405)) - Shutting down HiveServer2
2018-03-08 04:41:53,348 INFO [main-EventThread]: server.HiveServer2 (HiveServer2.java:removeServerInstanceFromZooKeeper(361)) - Server instance removed from ZooKeeper.
2018-03-08 04:41:53,348 INFO [main-EventThread]: zookeeper.ClientCnxn (ClientCnxn.java:run(524)) - EventThread shut down
2018-03-08 04:41:53,348 WARN [main]: server.HiveServer2 (HiveServer2.java:startHiveServer2(508)) - Error starting HiveServer2 on attempt 1, will retry in 60 seconds
org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. Application application_1520480101488_0046 failed 2 times due to AM Container for appattempt_1520480101488_0046_000002 exited with exitCode: -1000
For more detailed output, check the application tracking page: http://ip-10-0-91-7.ap-northeast-2.compute.internal:8088/cluster/app/application_1520480101488_0046 Then click on links to logs of each attempt.
Diagnostics: ExitCodeException exitCode=2: tar: Removing leading `/' from member names
tar: Skipping to next header
gzip: /hadoopfs/fs1/yarn/nodemanager/filecache/60_tmp/tmp_tez.tar.gz: invalid compressed data--format violated
tar: Exiting with failure status due to previous errors
Failing this attempt. Failing the application.
at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:699)
at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:218)
at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:116)
at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.startPool(TezSessionPoolManager.java:76)
at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:488)
at org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:87)
at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:720)
at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:593)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:233)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
I had exactly the same issue with HDP on AWS. FYI, In my case the issue was with HDP version 2.6.4.5-2. I'm going to show how I fixed using this version because it is the latest at this time.
As the error log shows the problem is with tez.tar.gz that is corrupted then YARN is unable to decompress it in the YARN container.
This tez.tar.gz file is copied from the hdfs:///hdp/apps/<hdp_version>/tez/tez.tar.gz.
To reproduce the error and confirm that this file is corrupted, you can run the following command:
sudo su
su hdfs
hdfs dfs -get /hdp/apps/2.6.4.5-2/tez.tar.gz
tar -xvzf tez.tar.gz
You will get the following error:
gzip: stdin: invalid compressed data--format violated
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now
The fix is pretty simple, you must just replace the HDFS file with the one that you have on your local file-system running the following command:
hdfs dfs -rm /hdp/apps/2.6.4.5-2/tez/tez.tar.gz
hdfs dfs -put /usr/hdp/current/tez-client/lib/tez.tar.gz /hdp/apps/2.6.4.5-2/tez/tez.tar.gz
Now restart Hive Server 2 service and done!
NOTE: If something similar happens with other services you can do the same thing. Please check the following link that has more details: https://community.hortonworks.com/articles/30096/foxing-broken-targz-and-jar-files-in-hdp-24.html
Hope this helps!

wp-content directory files into an archive file named files.tar.gz

I am trying to put wp-content directory into an archive file named files.tar.gz. I am doing this through putty on the external sever where wordpress is placed.
After command:
tar -zcvf files.tar.gz /public_html/denim4u/wp-content
I receive:
tar: Removing leading `/' from member names
tar:
/public_html/denim4u/wp-content: Cannot stat: No such file or
directory
tar: Exiting with failure status due to previous errors
What am I doing wrong?
According to this error message :
tar: Removing leading `/' from member names
tar: /public_html/denim4u/wp-content: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
The directory you want to compress is not a valid path.
Best regards