I wrote a Singularity container that works just fine on my computer. However, when a colleague of mine tries to run it, he gets the error output
FATAL: container creation failed: failed to resolve session directory /usr/local/var/singularity/mnt/session: lstat /usr/local/var: no such file or directory
In the past, he could run containers I build. In fact, he used being able to run a container with the same recipe. The change was that the version of Singularity on the machine I use to build it was upgraded.
I entered the error in a search engine, and I only found a single hit, https://forum.image.sc/t/improving-cluster-supercomputer-performance-tesla-v100-volta-16-32gb-gpu/37459/8, in which this is not resolved.
Does anybody know a way to fix this? Or what the source of the problem is? Or a workaround, preferably one that does not require me to downgrade Singularity? (The machine on which I build it is shared between several users, that's why I don't want to do that.)
Okay, this was somewhat trivial to solve, we just had the colleague create the required folder,
mkdir -p /usr/local/var/singularity/mnt/{container,final,overlay,session}
Related
I've made a fresh install of nextflow in a new computer, and I was trying to test the nf-core/rnaseq pipeline, but I am receiving the following error when executing:
Error executing process > 'NFCORE_RNASEQ:RNASEQ:INPUT_CHECK:SAMPLESHEET_CHECK (samplesheet.csv)'
Command error:
.command.sh: line 3: /media/Data/nextflow-rnaseq/rnaseq/bin/check_samplesheet.py: Permission denied
I've checked the permissions for the file, and has all the executing permissions:-rwxrwxr-x. I've also tried to execute it using both my working environment and singularity, and keep finding the same error.
I've also tested my own pipeline, with another project root folder and its own bin folder with custom scripts in there, and I'm having the same error.
Does anyone know if I'm missing something I should have done to make the scripts in bin accessible to nextflow?
Nextflow version: 22.04.4.5706
As Steve pointed out in a comment, the issue was related to how the filesystem was mounted (noexec), and fixing that solves the problem
Steps to reproduce are very easy.
Create a Dockerfile.
My Dockerfile has many more lines, but I have trimmed them so we can focus in the source of the problem.
Said that, these two lines alone (without anything more) show the problem.
FROM microsoft/iis
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue'; $VerbosePreference = 'Continue'; "]
Run docker build . and you get hcsshim::PrepareLayer - failed failed in Win32: Función incorrecta. (0x1).
Windows 10 Pro 1909 (but it happened too in 1903)
Docker version: 2.1.0.5
Engine: 19.03.5
Machine: 0.16.2
I have found the solution to the problem.
Reading all the https://github.com/docker/for-win/issues/3884 bug, some have found a simple solution: rename C:\windows\system32\driver\cbfsconnect2017.sys so it isn't loaded the next boot.
Disabling that driver enables me to do a docker build for the first time in windows containers in almost a year.
In my case Box Sync was the one using that driver.
EDIT: #GustavoTM have found that pCloud raises the same problem.
EDIT2: #VonC have noticed that some people in the issue in GitHub has solved it deleting this other file: C:\Windows\System32\drivers\cbfs6.sys. I haven't tried that, but i put it if it helps others.
The good thing is that I don't need to uninstall Box, but only rename that file.
This is still an issue (still open) with Win10.
Looks like uninstalling cloud storage providers with file system filters like Dropbox, Box, etc. as a workaround is an option for some users.
Deinstall cloud storage providers or virus scanners; if you identify which one is not working please share in https://github.com/docker/for-win/issues/3884
In my case was the problem similar but the file cbfs6.sys was placed somewhere in the rest of uninstalled application Jungle disk, somewhere in the folder c:\Program files\Jungle disk .... It's part of Callback File System signed by EldoS Corporation.
The folder could be rename only and not delete directly. So I could delete its immediately after the PC restart, before running the Docker. So it could be delete during the Docker service restart too.
I am using Hadoop 3.2.0 and trying to run a simple application in a docker container and I have made the required configuration changes both in yarn-site.xml and container-executor.cfg to choose LinuxContainerExecutor and docker runtime.
I use the example of distributed shell in one of the hortonworks blog. https://hortonworks.com/blog/trying-containerized-applications-apache-hadoop-yarn-3-1/
The problem I face here is when the application is submitted to YARN it fails with a reason related to directory creation issue with the below error
2019-02-14 20:51:16,450 INFO distributedshell.Client: Got application
report from ASM for, appId=2, clientToAMToken=null,
appDiagnostics=Application application_1550156488785_0002 failed 2
times due to AM Container for appattempt_1550156488785_0002_000002
exited with exitCode: -1000 Failing this attempt.Diagnostics:
[2019-02-14 20:51:16.282]Application application_1550156488785_0002
initialization failed (exitCode=20) with output: main : command
provided 0 main : user is myuser main : requested yarn user is
myuser Failed to create directory
/data/yarn/local/nmPrivate/container_1550156488785_0002_02_000001.tokens/usercache/myuser
- Not a directory
I have configured yarn.nodemanager.local-dirs in yarn-site.xml and I can see the same reflected in YARN web ui localhost:8088/conf
<property>
<name>yarn.nodemanager.local-dirs</name>
<value>/data/yarn/local</value>
<final>false</final>
<source>yarn-site.xml</source>
</property>
I do not understand why is it trying to create usercache dir inside the nmPrivate directory.
Note : I have verified the permissions for myuser to the directories and also have tried clearing the directories manually as suggested in a related post. But no fruit. I do not see any additional information about container launch failure in any other logs.
How do I debug why the usercache dir is not resolved properly??
Really appreciate any help on this.
Realized that this is all because of the users the services were started with and the permissions to the directories the services work on.
After making sure the required changes are done, I am able to seamlessly run the examples and other applications..
Thanks Hadoop user community for the direction. Adding the link here for more details.
http://mail-archives.apache.org/mod_mbox/hadoop-user/201902.mbox/browser
I'm trying to run this code from : https://github.com/pnnl/safekit ,using cmd on windows 10, I already installed python.
when I type the command:
tar -xjvf data_examples.tar.bz2
I keep getting the error:
tar: Error opening archive: Can't initialize filter; unable to run
program "bzip2 -d"
I have tried to download bzip2 through easy-7 zip and GnuWin32 , but it didn't work.
Can any one help me?
P.S.: I did search for the same problem before posting mine.
If you run the command in git bash instead of cmd it should work.
I've run into the same problem! My non-elegant solution so far has been to force Windows's tar.exe to use the bzip2.exe provided with my Windows Git installation. The trick is to add to your user PATH the directory where bzip2.exe is located, in my case:
C:\Program Files\Git\mingw64\bin
So, right now my PATH looks like:
Path=%USERPROFILE%\AppData\Local\Microsoft\WindowsApps;C:\Program Files\Git\mingw64\bin
Notice that I've only touched the user environment PATH, not the system PATH, and I've appended the new directory. As always, touching your PATH can be a little bit dangerous, proceed with care. If someone has a better solution, I'd be glad to hear it.
Note: I tried copying the bzip2.exe to a separate directory, but this didn't work (I presume because bzip2.exe couldn't find some dependencies).
Trying to set up Jenkins on one of my servers for the first time and think I might be missing something.
Jenkins 1.545
Phing 2.6.1
Jenkins builds give me the following output.
Building in workspace /var/www/vhosts/domain.co.uk/httpdocs
looking for '/var/www/vhosts/domain.co.uk/httpdocs/build.xml' ...
looking for '/var/www/vhosts/domain.co.uk/httpdocs/build.xml' ...
looking for 'build.xml' ...
buildfile 'build.xml' not found.
Build step 'Invoke Phing targets' marked build as failure
Finished: FAILURE
If I run my build.xml on it's own it works fine.
I'm using a custom workspace at the moment, before I tried a symlink from the default workspace to my webroot, when I did that it found the build file but failed when trying to run phing. I know it's a problem with permissions but I'm not sure exactly what.
I'm running this on a plesk web server and have tried adding the jenkins user to the psacln and psaserv groups but that didn't work either.
I use hudson but I think is the same problem.
Provide to ant job the full path (advanced settings)
${WORKSPACE}/buil.xml
Assuming the correct set of jenkins user
RUN_AS_USER=jenkins
Go to the custom workspace and
chown -R jenkins:jenkins myworkspace
if it doesn't work
chmod -R 777 myworkspace
then you will fix later.
I hope it helps.