I'd like to use Repast Simphony's batch runner in headless mode using an unrolled parameter file to avoid using a nxn experiment setup. I have managed to create a jar using the console and the parameter file, but cannot find a way to actually launch the model jar that was created, since the -run option does not appear to be valid when using custom unrolled parameter files.
Does anyone have some advice on how to proceed there?
Currently, there's no way to use a custom upf and have simphony chunk the upf and distribute and run those chunks on different hosts. You can use the -u / --upf arguments to include your custom upf in the payload and then run that on an HPC system via the slurm or PBS scheduler. There's instructions for that in the batch runs getting started doc.
Those HPC runs use a script that runs some chunk of the upf file individually. That might be a useful workaround here for you.
sed -n "$begin","$end"p "$paramFile" > localParamFile.txt
mkdir $instanceDir
cd $instanceDir
java -Xmx512m -cp "../lib/*" repast.simphony.batch.InstanceRunner \
-pxml ../scenario.rs/batch_params.xml \
-scenario ../scenario.rs \
-id $instance \
-pinput localParamFile.txt
The idea here is that sed is used to chunk the upf file beginning at line $begin and ending at line $end and write that into localParamFile.txt. Then the InstanceRunner is started and that iterates over each line in that file and performs a model run using each line as input.
You could adapt this and manually chunk your custom file and then run the InstanceRunner.
Related
I installed slurm on a workstation and it seemed to work, i can use the slurm commands, srun is working too.
But when i try to launch a job from a script using sbatch test.sh i get the following error : Batch job submission failed: I/O error writing script/environment to file even if the script is the simplest like
#!/bin/bash
srun hostname
Make sure slurmd is running as root. See the SlurmdUser parameter in slurm.conf. Its default value is root and it should be so.
Note this is different from the SlurmUser parameter, that defines the user which runs the controller processes ; this one is preferably not root.
If the configuration is correct, then you might have a faulty filesystem at the location referred to in the SlurmdSpoolDir parameter, where slurmd writes the submission script and environment for jobs assigned to the node.
having a small problem where you can help me out. On our new cluster we use LMod as environmental module system.
Creating a Module TCL Script for OpenFOAM, a system-dependent bashrc file need to be loaded.
This is the TCL script which I am using on another module system, it works fine. I am not able to execute the "source" command line in Lmod, what I am missing here?
#%Module1.0#####################################################################
##
## modules software/openfoam_v1812
##
## /opt/software/openfoam/openfoamv1812/OpenFOAM-v1812
proc ModulesHelp { } {
global version modroot
puts stderr "software/OpenFOAM-v1812 - sets the Environment for OpenFOAM-v1812 (openfoam.com)"
}
module-whatis "Sets the environment for using OpenFOAM-v1812"
# for Tcl script use only
set VERSION v1812
set OpenFOAM_PATH /opt/software/openfoam/openfoam${VERSION}/OpenFOAM-${VERSION}
set FOAM_INST_DIR /opt/software/openfoam/openfoam${VERSION}
puts stdout "source /opt/software/openfoam/openfoam${VERSION}/OpenFOAM-${VERSION}/etc/bashrc;"
I am not an expert, but I have recently come across a similar problem, in my case for activating Anaconda Python in a model. In my case, the solution was to use the 'execute' command in LMod
https://lmod.readthedocs.io/en/latest/050_lua_modulefiles.html
which has the documentation:
execute {cmd=”<any command>”,modeA={“load”}}
Run any command with a certain mode. For example execute {cmd=”ulimit
-s unlimited”,modeA={“load”}} will run the command ulimit -s unlimited as the last thing that the loading the module will do.
Hope this helps
I can't get the output of a script run through singularity.
I have a python script, at the end of which the output is saved with:
...
with open('saveOut.pkl','wb') as myFile:
pickle.dump(myTable,myFile)
I want to run this script with singularity on a distant machine. Since I am learning singularity, I made a 'sand box' debian image (not compiled into a single 'img' file yet) in the directory /tmp/debian; in this image I copied the python script test.py in /usr/src and I run it with the command:
sudo singularity exec /tmp/debian python3.5 /usr/src/test.py
The problem:
It works well as long as I have only displayed results. with the pickle example described above, I don't get any saveOut.pkl file anywhere: this file is just not written anywhere but I don't see any message. I tried to write an explicit path in the python script. For instance /usr/src/saveOut.pkl, but this is the same.
How could I write a result ?
What was your expected result i.e. in which directory did you expect
to find the output file?
I expect a file saveOutput.pkl anywhere, in the container or not, I don't care the location. Currently I don't get it at all: neither in the container's current directory, nor in the container's /usr/src/, nor on the host, nor anywhere.
Did you look for it on the host or in the container?
both, I don't see it anywhere
What's happening here is that your python script is writing the pickle file to its current location (/usr/src/ in the container). Then, since the output from your script is not persistent (due to the sandbox not being writable on execution), it gets deleted at the end of the run.
I believe you could change your script:
with open('/opt/saveOut.pkl','wb') as myFile:
pickle.dump(myTable,myFile)
and then bind the local directory and get the output you're looking for:
sudo singularity exec -B ./:/opt /tmp/debian python3.5 /usr/src/test.py
This worked for me, anyway.
There is no documentation that I can find beyond the oneliner provided by the command-line utility. It says:
-batch force batch I/O
So what's going on here if I pass it a query or multiple queries?
When the sqlite3 shell is run interactively, it shows a startup message and the sqlite> prompt, and (on Windows) tries to converts the input to UTF-8.
If the automatic console detection does not work as intended, it can be overridden with -batch or -interactive.
I want to start load testing by running JMeter from command line for more accurate test results, but how can I monitor the run and then analyze the results after the test finishes.
You can generate JTL (JMeter results) file while executing the JMX (JMeter script) file from command line. A sample command for generating JTL file will look like this..
jmeter -n -t path-to-jmeterScript.jmx -l path-to-jtlFile.jtl
After completion of script execution you can open the JMeter GUI and simply open the JTL file in any listener (as per your requirement).
Most of the listeners in JMeter have an option to save the results into a file. This file contains usually not the report itself, but the samples which are generated by the tests. If you define this filename, you can generate the reports using these saved files. For example see http://jmeter.apache.org/usermanual/component_reference.html#Summary_Report .
If you run JMeter in command-line non-GUI mode passing results file name via -l parameter it will output results there. After test finishes you will be able to open the file with the Listener of your choice and perform the analysis.
By default JMeter writes results in chunks, if you need to monitor them in real time add the following line to user.properties file (lives under /bin folder of your JMeter installation)
jmeter.save.saveservice.autoflush=true
You can use other properties which names start with jmeter.save.saveservice.* to control what metrics you need to store. The list with default values can be seen in jmeter.properties file. See Apache JMeter Properties Customization Guide for more information on various JMeter properties types and ways of working with them.
You can also consider running your JMeter test via Taurus tool - it provides some statistics as the test goes either in console mode or via web interface.