I have confirmed my host machine support branch-miss perf events by testing with perf command.
But I cannot observe any branch-miss info from the generated flamegraph when I use async-profiler.
This is the command I use: profiler.sh -f /tmp/fg.html -e branch-misses -d 120 Pid. The fg.html is empty.
If I use profiler.sh -f /tmp/fg.html -e cpu -d 120 Pid, the flamegraph is generated correctly but there is no info about branch-miss.
Do I miss anything here? Thank you.
The version of async-profiler I use is 2.8.3 (latest).
Related
I'm trying to use the --immediate-submit on a PBSPro cluster. I tried using an in-place modification of the dependencies string to adapt it to PBSPro, similar to what is done here.
snakemake --cluster "qsub -l wd -l mem={cluster.mem}GB -l ncpus={threads} -e {cluster.stderr} -q {cluster.queue} -l walltime={cluster.walltime} -o {cluster.stdout} -S /bin/bash -W $(echo '{dependencies}' | sed 's/^/depend=afterok:/g' | sed 's/ /:/g')"
This last part gets converted into, for example:
-W depend=afterok: /g/data1a/va1/dk0741/analysis/2018-03-25_marmo_test/.snakemake/tmp.cyrhf51c/snakejob.trimmomatic_pe.7.sh
There are two problems here:
How can I get the dependencies string to output job ID instead of the script path? The qsub command normally outputs the job ID to stdout, so I'm not sure why it's not doing so here.
How do I get rid of the space after afterok:? I've tried everything!
As an aside, it would be helpful if there were some option to debug the submission or not to delete the tmp.cyrhf51c directory in .snakemake -- is there some way to do this?
Thanks,
David
I suggest to use a profile for this, instead of trying to find an ad-hoc solution. This will also help with debugging. E.g., there is already a pbs-torque profile available (https://github.com/Snakemake-Profiles/pbs-torque), probably there is not much to change towards pbspro?
My problem is that I have a cluster-server with Torque PBS and want to use it to run a sequence-comparison with the program rapsearch.
The normal RapSearch command is:
./rapsearch -q protein.fasta -d database -o output -e 0.001 -v 10 -x t -z 32
Now I want to run it with 2 nodes on the cluster-server.
I've tried with: echo "./rapsearch -q protein.fasta -d database -o output -e 0.001 -v 10 -x t -z 32" | qsub -l nodes=2 but nothing happened.
Do you have any suggestions? Where I'm wrong? Help please.
Standard output (and error output) files are placed in your home directory by default; take a look. You are looking for a file named STDIN.e[numbers], it will contain the error message.
However, I see that you're using ./rapsearch but are not really being explicit about what directory you're in. Your problem is therefore probably a matter of changing directory into the directory that you submitted from. When your terminal is in the directory of the rapsearch executable, try echo "cd \$PBS_O_WORKDIR && ./rapsearch [arguments]" | qsub [arguments] to submit your job to the cluster.
Other tips:
You could add rapsearch to your path if you use it often. Then you can use it like a regular command anywhere. It's a matter of adding the line export PATH=/full/path/to/rapsearch/bin:$PATH to your .bashrc file.
Create a submission script for use with qsub. Here is a good example.
In /usr/local/nagios/etc/nrpe.cfg I added a new command check_this_process to the already pre-defined ones:
command[check_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10
command[check_load]=/usr/local/nagios/libexec/check_load -w 15,10,5 -c 30,25,20
command[check_hda1]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/$
command[check_zombie_procs]=/usr/local/nagios/libexec/check_procs -w 5 -c 10 -s$
command[check_total_procs]=/usr/local/nagios/libexec/check_procs -w 150 -c 200
command[check_this_process]=/usr/local/nagios/libexec/check_procs -w 15 -c 20 -C name
This works:
define service{
use generic-service
host_name my_host
service_description CPU Load
check_command check_nrpe!check_load
}
This doesn't:
define service{
use local-service
host_name my_host
service_description cron
check_command check_nrpe!check_this_process
}
and returns: NRPE: Command 'check_this_process' not defined
The terminology used in the supplied docs is a little confusing, but I'll put it like this:
As written in Page 10 of https://assets.nagios.com/downloads/nagioscore/docs/nrpe/NRPE.pdf, you need to modify /usr/local/nagios/etc/commands.cfg on your Nagios server and add the following to define the check_nrpe command:
define command{
command_name check_nrpe
command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
}
On your Nagios server, define your service definition as you've already done:
define service{
use local-service
host_name my_host
service_description cron
check_command check_nrpe!check_this_process
}
On your remote host to be monitored, the following is going to be different depending on whether you installed NRPE:
using the tarball and xinetd as in
https://assets.nagios.com/downloads/nagioscore/docs/nrpe/NRPE.pdf
or using a package manager like yum as in
http://sharadchhetri.com/2013/03/02/how-to-install-and-configure-nagios-nrpe-in-centos-and-red-hat/
If you used the tarball / xinetd method, your NRPE configuration file will likely be located at /usr/local/nagios/etc/nrpe.cfg on your remote-host-to-be-monitored. (To avoid typing that all the time, I'll just call it "my_host").
So, on my_host, modify /usr/local/nagios/etc/nrpe.cfg.
Add
command[check_this_process]=/usr/local/nagios/libexec/check_procs -w 15 -c 20 -C name
So that it looks like:
command[check_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10
command[check_load]=/usr/local/nagios/libexec/check_load -w 15,10,5 -c 30,25,20
command[check_hda1]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/$
command[check_zombie_procs]=/usr/local/nagios/libexec/check_procs -w 5 -c 10 -s$
command[check_total_procs]=/usr/local/nagios/libexec/check_procs -w 150 -c 200
command[check_this_process]=/usr/local/nagios/libexec/check_procs -w 15 -c 20 -C name
(Note: the above is assuming you have a process called name. If not, replace name with your real process name: i.e. crond)
Restart xinetd:
service xinetd restart
(NOTE: restarting xinted might not be necessary, but I don't use it so I'm a little fuzzy on this one.)
However, if you installed NRPE on my_host using a package manager like yum, your NRPE configuration file will likely be located at /etc/nagios/nrpe.cfg.
So, on my_host, modify /etc/nagios/nrpe.cfg.
Add
command[check_this_process]=/usr/local/nagios/libexec/check_procs -w 15 -c 20 -C name
So that it looks like:
command[check_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10
command[check_load]=/usr/local/nagios/libexec/check_load -w 15,10,5 -c 30,25,20
command[check_hda1]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/$
command[check_zombie_procs]=/usr/local/nagios/libexec/check_procs -w 5 -c 10 -s$
command[check_total_procs]=/usr/local/nagios/libexec/check_procs -w 150 -c 200
command[check_this_process]=/usr/local/nagios/libexec/check_procs -w 15 -c 20 -C name
Restart the nrpe service:
service nrpe restart
Back on your Nagios server, run a verification of your Nagios configuration settings:
nagios -v /usr/local/nagios/etc/nagios.cfg
Check the output for errors.
If there are no errors, restart Nagios:
service nagios restart
On your Nagios server you should have a check_nrpe utility installed somehwere as a result of installing the "check_nrpe plugin" on your Nagios server.
See pages 9 and 10 of: https://assets.nagios.com/downloads/nagioscore/docs/nrpe/NRPE.pdf
This check_nrpe utility will most likely be located at: /usr/local/nagios/libexec/check_nrpe
Using the host information for my_host manually test your NRPE connection from the Nagios server.
Execute the following:
/usr/local/nagios/libexec/check_nrpe -H <IP Address of my_host> -c check_this_process
If everything is setup correctly, you should get some output on the command line.
My trouble-shooting guide for 'NRPE: Command ... not found.' Ordered from most common to least common - in my environment.
Was the NRPE daemon restarted AFTER adding the new command? If it is a new command, then NRPE MUST be restarted.
Typos/spelling errors. Does the configured command name on the Nagios side, match that the one in the NRPE config?
Permissions issues. Does the USER that NRPE runs as, have READABLE and EXECUTABLE access to the actual command being ran? Did you test run the command, as the NRPE user? On that same system? TIP: Use the dash (-) when changing to the NRPE user on Linux (su - ...) so you import said users environment as well.
Path issues. Was the FULL PATH to the actual command put into the NRPE config file? Doing this will normally eliminate issues with PATHs, so don't do it any other way.
Bad commands. Does the actual command really execute? Or is it simply throwing an error and exiting? Do you have the correct version of (INSERT SOMETHING HERE) to run the command, installed on the remote system? You should be able to run any command defined in the nrpe.cfg from the command line, and all new commands should be checked BEFORE being added to the nrpe.cfg.
IF ALL THE ABOVE FAILS: Enable DEBUGGING in NRPE and check the log files (on the remote host). This is a bit of a drawn out process - described in the documentation - read it. It is important to disable DEBUGGING as soon as you get output that looks like it would be useful.
This checklist ASSUMES that you've done the needful things to the various Nagios and NRPE configs to get it working in the first place. Hopefully others will read this before posting yet another question as to why they are seeing this error.
Please help me clarify, if the pgbench tool can execute my own sql scenarios in parallel way?
Googling and local searching brought no positive result.
I run the script that execeutes with no errors. But after execution I see no signs, that my script was actually performed.
Does pgbench commits transaction with my sql script?
That's an output I get:
C:\Program Files\PostgreSQL\9.2\bin>pgbench.exe -n -h dbserverhost -U postgres -
T 10 -c 64 -j 8 bench_dbname -f c:\Dev\bench_script.sql
transaction type: TPC-B (sort of)
scaling factor: 1
query mode: simple
number of clients: 64
number of threads: 8
duration: 10 s
number of transactions actually processed: 1020
tps = 95.846561 (including connections establishing)
tps = 103.387127 (excluding connections establishing)
C:\Program Files\PostgreSQL\9.2\bin>
SQL script bench_script.sql is:
--comment here
begin;
insert into schm.log values ( 'pgbench test', current_timestamp );
end;
SOLUTION
pgBench Windows version is sensitive to the order of the arguements passed to the utility:
"bench_dbname" argument must be the last one parameter in a line.
This is the correct example of pgbench Windows version command line:
pgbench.exe -d -r -h 127.0.0.1 -U postgres -T 5 -f C:\Dev\bench_script.sql -c 64 -j 8 postgres
The most useful arguments for me were:
-T 60 (time in seconds to run script)
-t 100 (transaction amount per client)
-d print detailed debug info to the output
-r include in summary latency value calculated for every action of the script
-f run user defined sql script in benchmark mode
-c client amount
-j thread amount
pgBench official doc
PgBench, I love you! :)
Best wishes everybody ;)
The "transaction type: TPC-B (sort of)" means that it did not process the -f option to run your custom sql script, instead it ran the default query.
On Windows versions, getopt seems to stop parsing the options once it reaches the first one that does not start with a hyphen, i.e. "bench_dbname". So make sure -f comes before that.
I guess you also need the -n option as long as you are using your custom script?
-n
--no-vacuum
Perform no vacuuming before running the test.
This option is necessary if you are running a custom test scenario
that does not
include the standard tables pgbench_accounts, pgbench_branches,
pgbench_history, and pgbench_tellers.
I am setting up a Debian Etch server to host ruby and php applications with nginx. I have successfully configured inittab to start the php-cgi process on boot with the respawn action. After serving 1000 requests, the php-cgi worker processes die and are respawned by init. The inittab record looks like this:
50:23:respawn:/usr/local/bin/spawn-fcgi -n -a 127.0.0.1 -p 8000 -C 3 -u someuser -- /usr/bin/php-cgi
I initially wrote the process entry (everything after the 3rd colon) in a separate script (simply because it was long) and put that script name in the inittab record, but because the script would run its single line and die, the syslog was filled with errors like this:
May 7 20:20:50 sb init: Id "50" respawning too fast: disabled for 5 minutes
Thus, I got rid of the script file and just put the whole line in the inittab. Henceforth, no errors show up in the syslog.
Now I'm attempting the same with thin to serve a rails application. I can successfully start the thin server by running this command:
sudo thin -a 127.0.0.1 -e production -l /var/log/thin/thin.log -P /var/run/thin/thin.pid -c /path/to/rails/app -p 8010 -u someuser -g somegroup -s 2 -d start
It works apparently exactly the same whether I use the -d (daemonize) flag or not. Command line control comes immediately back (the processes have been daemonized) either way. If I put that whole command (minus the sudo and with absolute paths) into inittab, init complains (in syslog) that the process entry is too long, so I put the options into an exported environment variable in /etc/profile. Now I can successfully start the server with:
sudo thin $THIN_OPTIONS start
But when I put this in an inittab record with the respawn action
51:23:respawn:/usr/local/bin/thin $THIN_OPTIONS start
the logs clearly indicate that the environment variable is not visible to init; it's as though the command were simply "thin start."
How can I shorten the inittab process entry? Is there another file than /etc/profile where I could set the THIN_OPTIONS environment variable? My earlier experience with php-cgi tells me I can't just put the whole command in a separate script.
And why don't you call a wrapper who start thin whith your options?
start_thin.sh:
#!/bin/bash
/usr/local/bin/thin -a 127.0.0.1 -e production -l /var/log/thin/thin.log -P /var/run/thin/thin.pid -c /path/to/rails/app -p 8010 -u someuser -g somegroup -s 2 -d start
and then:
51:23:respawn:/usr/local/bin/start_thin
init.d script
Use a script in
/etc/rc.d/init.d
and set the runlevel
Here are some examples with thin, ruby, apache
http://articles.slicehost.com/2009/4/17/centos-apache-rails-and-thin
http://blog.fiveruns.com/2008/9/24/rails-automation-at-slicehost
http://elwoodicious.com/2008/07/15/nginx-haproxy-thin-fastcgi-php5-load-balanced-rails-with-php-support/
Which provide example initscripts to use.
edit:
Asker pointed out this will not allow respawning. I suggested forking in the init script and disowning the process so init doesn't hang (it might fork() the script itself, will check). And then creating an infinite loop that waits on the server process to die and restarts it.
edit2:
It seems init will fork the script. Just a loop should do it.