Run cgi program without server - testing

Is there a tool which given a cgi program, and the arguments (query string, method, file to upload, etc.) would set the correct environment variables, and execute this program (with or without a debugger.
Something like this perl script, only more solid, with a more clear output and input, support for files, etc.

Well, a new open source project was born from this question. I'm copying my answer from here.
If you just want to see your CGI running, you can use my tiny runCGI project.
All you need is to set a yaml file which looks something like this
_exec: /var/cgi-bin/myfile.cgi
method: GET
query_string:
q: s
and then run
./runCGI myyamlfile.yaml
You will see the output on the console's standard output.
You can even debug it with gdb, debug runCGI gdb runCGI, run with the correct parameters (run someyaml.yaml), issue tcatch exec (tcatch catches it only once) and then set breakpoints to your CGI file:
$ g++ a.cc -o a.out
$ cat a.yaml
method: GET
_exec: a.out
$ gdb runCGI
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu"...
(gdb) tcatch exec
Catchpoint 1 (exec)
(gdb) run a.yaml
Starting program: /home/elazar/runCGI/runCGI a.yaml
Executing new program: /home/elazar/runCGI/a.out
0x00007fc3a24a6a60 in ?? () from /lib64/ld-linux-x86-64.so.2
(gdb) tbreak main
Breakpoint 2 at 0x400577: file a.cc, line 2.
(gdb) c
Continuing.
main (argc=1, argv=0x7fff14891408) at a.cc:2
2 int a =0;
(gdb)

Related

gawk 3.1.6-1 on Windows 7 x64 Pro gets 0 return code using system() even on failed commands

I'm running gawk scripts on Windows. For a long time I've used gawk 3.1.4 on Windows XP x86 and all was OK.
My environment has changed to Windows 7 x64, and now gawk 3.1.4 frequently fails with fatal errors.
I've updated to latest available gawk 3.1.6-1 (https://sourceforge.net/projects/gnuwin32/files/gawk/) --> fatal errors are gone (yahoo), but a very strange behaviour I met: it cannot get non-zero return code on failing command.
For example, I call
print "System return test: ";
system( "gawk --version");
myReturnCode = system( "exit 0");
print "1 returned: " myReturnCode;
myReturnCode = system( "exit 1");
print "2 returned: " myReturnCode;
and the result is
System return test:
GNU Awk 3.1.6
Copyright (C) 1989, 1991-2007 Free Software Foundation.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see http://www.gnu.org/licenses/.
1 returned: 0
2 returned: 0
Why 2 returned: 0??? Previous gawk versions returns 1 as expected
System return test:
GNU Awk 3.1.4
Copyright (C) 1989, 1991-2003 Free Software Foundation.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
1 returned: 0
2 returned: 1
All my command success statuses are totally broken by this reason. I need non-zero return codes for the failed commands in gawk.
Does anybody run gawk on Windows 7 x64? Do you get something similar? Is there any ways to work this problem out?
UPD: Some notes for those who met the same problem and would like to try Cygwin
Thanks to #EdMorton with Cygwin's gawk.exe usage idea. Yes, generally speaking it works on Windows 7 x64 and system( "exit 1") returns 1 as expected (see MWE below), but the update from 3.1.6 to Cygwin is not painless. And I'm thinking should I fight against them in my current gawk-scripts-windows-world, or rewrite in in Python 3.
This is a minimal working example of Cygwin's gawk call from a Batch, two scripts:
REM awkTest.cmd
#echo off
set "exeGAWK=C:\cygwin64\bin\gawk.exe"
echo exeGAWK = "%exeGAWK%"
call "%exeGAWK%" -f "test.awk" nul
and
# test.awk
END\
{
exeGAWK = ENVIRON[ "exeGAWK" ];
print "Check version: ";
print exeGAWK
system( exeGAWK " --version");
gsub(/\\/, "/", exeGAWK)
print exeGAWK
system( exeGAWK " --version");
print "Dir test: ";
system( "dir " exeGAWK);
print "System return test: ";
myReturnCode = system( "exit 0");
print "1 returned: " myReturnCode;
myReturnCode = system( "exit 1");
print "2 returned: " myReturnCode;
}
The result is
exeGAWK = "C:\cygwin64\bin\gawk.exe"
Check version:
C:\cygwin64\bin\gawk.exe
sh: C:cygwin64bingawk.exe: command not found
C:/cygwin64/bin/gawk.exe
GNU Awk 5.0.1, API: 2.0 (GNU MPFR 4.0.2, GNU MP 6.1.2)
Copyright (C) 1989, 1991-2019 Free Software Foundation.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see http://www.gnu.org/licenses/.
Dir test:
sh: dir: command not found
System return test:
1 returned: 0
2 returned: 1
The obvious problems are
forward slashes \ in windows path should be converted to /;
cannot call Windows system dir command.
Here's an example of how you can call cygwin's awk from Windows. In Windows associate the ".bash" file suffix with "bash.exe" in the usual manner (create "test.bash" then right-click to open-with and find the bash.exe under your cygwin64 directory) and then double click on a file named "test.bash" containing this:
export HOME="/cygdrive/c/cygwin64/home/$USERNAME"
export PATH="$HOME:/usr/bin:$PATH"
. .bash_profile
# cd to the directory this script is in assuming you want it to run from there
dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
dir="C:${dir#/cygdrive/c}"
cd "$dir" || exit 1
awk 'BEGIN {
print "System return test: ";
system( "gawk --version");
myReturnCode = system( "exit 0");
print "1 returned: " myReturnCode;
myReturnCode = system( "exit 1");
print "2 returned: " myReturnCode;
}'
sleep 30
and it'll pop up a window displaying the following for 30 seconds:
System return test:
GNU Awk 5.0.1, API: 2.0 (GNU MPFR 4.0.2, GNU MP 6.1.2)
Copyright (C) 1989, 1991-2019 Free Software Foundation.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see http://www.gnu.org/licenses/.
1 returned: 0
2 returned: 1
See also how-do-i-use-awk-under-cygwin-to-print-fields-from-an-excel-spreadsheet for how to do the opposite, i.e. call a Windows command (Excel) from a cygwin bash script.

Escape characters being printed to yorick shell

I’m working with the Yorick language on Yosemite. I’ve been able to successfully install the source code and start up the Yorick shell. However, using keyboard input at the shell (e.g. delete, up arrow, etc …) prints out the escape sequence ( ^H and ^[[A, respectively) to the command line.
$ yorick
Copyright (c) 2005. The Regents of the University of California.
All rights reserved. Yorick 2.1.06 ready. For help type 'help'
> ^H # delete
> ^[[A # up arrow
Is this type of response specific to the language's interpreter? Or is there some bash configuration I can implement to get the desired response of ‘erase' and 'recall command history' for these inputs?
Very simple fix.
The readline package is incompatible with Yorick (your graphics window
goes dead when readline blocks waiting for keyboard input)
However, I was able to install rlwrap via
$ brew install rlwrap
Which acts as a GNU readline wrapper to run the shell in.
$ rlwrap yorick
# Now delete and up arrow work

Scripting LLDB to obtain a stack trace after a crash

I'm trying to add the capability of generating a stack trace from a core dump on mac automatically when one of our tests crashes.
I was able to do it pretty easily on linux with
gdb --batch --quiet -ex "thread apply all bt" -ex "quit" <binary> <core file> 2> /dev/null
But I'm having some trouble doing the same on mac (OSX 10.8) with lldb. To start off, the version of lldb I'm using is lldb-310.2.37.
My initial approach was to use the -s option and pass in a script file like so:
target create -c <core file> <binary>
thread backtrace all
quit
Initially I had some trouble which I think was caused by missing a newline at the end of the script file which caused lldb to not exit, but after that was fixed, I'm getting the following:
Executing commands in 'lldbSource'.
(lldb) target create -c <core file> <binary>
Core file '<core file>' (x86_64) was loaded.
(lldb) thread backtrace all
error: Aborting reading of commands after command #1: 'thread backtrace all' failed with error: invalid thread
Aborting after_file command execution, command file: 'lldbSource' failed.
The funny thing is, after that happens, we're still running lldb, and issuing 'thread backtrace all' manually works just fine.
So, approach #2 was to create a python script and use their python API (I tried this before figuring out the initial blocker I described was due to a missing newline).
My script:
import lldb
debugger = lldb.SBDebugger.Create()
target = debugger.CreateTarget('<binary>')
if target:
process = target.LoadCore('<core file>')
if process:
print process.exit_description
for thread in process:
print 'Thread %s:' % str(thread.GetThreadID())
print '\n'.join(str(frame) for frame in thread)
The problem I'm having with this approach is that process.exit_description is returning None (and so is every other thing I've tried; LLDB's python API documentation is almost completely useless).
The output I'm looking for from that call is something similar to the following:
Process 0 stopped
* thread #1: tid = 0x0000, 0x00007fff8aca4670 libsystem_c.dylib`strlen + 16, stop reason = signal SIGSTOP
frame #0: 0x00007fff8aca4670 libsystem_c.dylib`strlen + 16
libsystem_c.dylib`strlen + 16:
-> 0x7fff8aca4670: pcmpeqb (%rdi), %xmm0
0x7fff8aca4674: andl $0xf, %ecx
0x7fff8aca4677: shll %cl, %eax
0x7fff8aca4679: pmovmskb %xmm0, %ecx
This is output automatically by LLDB proper when loading a core file. I don't necessarily need the assembly dump, but I want at least the thread, frame and reason.
I think the first approach I used, if it could be made to work, would be ideal, but either way is OK for me. I don't have control over the LLDB version that's going to be used unfortunately, so I can't just update to latest and see if it's a bug that was fixed.
Other approaches to get the desired output are also welcome. For context, this is going to be called from a perl script.
The --batch command line argument is supported in the version of lldb that ships with Xcode 7.2 (lldb-340.4.119), and possibly earlier.
It's not documented in the man page, but it is documented in lldb --help:
-b
--batch
Tells the debugger to running the commands from -s, -S, -o & -O,
and then quit. However if any run command stopped due to a signal
or crash, the debugger will return to the interactive prompt at the
place of the crash.
These other command line options are useful for lldb automation:
-o
--one-line
Tells the debugger to execute this one-line lldb command after any
file provided on the command line has been loaded.
-k
--one-line-on-crash
When in batch mode, tells the debugger to execute this one-line
lldb command if the target crashes.
Using these facilities, I pieced together the following command:
ulimit -c unlimited && (<binary> || (lldb -c `ls -t /cores/* | head -n1` \
--batch -o 'thread backtrace all' -o 'quit' && exit 1))
This command:
Enables core dumps
Runs the executable <binary>
If it fails, runs lldb on the most recently created core dump in /cores (hopefully the right one)
Prints the backtraces of all threads
Exits with a non-zero status, so that this command can be embedded in other workflows (CI scripts, Makefiles, etc.)
I would have preferred to run lldb on <binary> directly, so that the command does not rely on guessing at the correct core file. But lldb still appears to lack a way to cause it to exit with a non-zero exit status -- an equivalent of GDB's -return-child-result option or quit 1 command -- so I would have no way of knowing if the debugged program was successful or not. I filed an issue requesting such a feature.
TOT lldb from lldb.llvm.org has a new "--batch" mode to work pretty much like the gdb batch mode, and fixes some bugs that made command-line source commands behave better. If you can build your own lldb, you can get these fixes now, otherwise you'll have to wait till the next Xcode update.
The exit_description is None because your process didn't exit, it crashed. Note that at least on OS X several threads can have had simultaneous exceptions by the time the process gets around to crashing, it isn't really useful to say the process crashed. You have to ask the threads. The stop status that lldb prints out when a thread stops is available with the GetStatus method.
stream = lldb.SBStream()
process.threads[0].GetStatus(stream)
print stream.GetData()
That doesn't seem to have a very useful help string.

Valgrind massif produces no profile output for 7-Zip

I want to use valgrind massif to figure out the heap memory used by 7-Zip. When I run massif on 7-Zip, it produces no output. Here's the command.
valgrind --tool=massif /usr/bin/7z a filename.7z filename
I would expect an output file named massif.out.pid within the current directory but no such output is produced. I should add that, using massif on other compression tools like gzip, bzip2, compress, etc. produces a massif.out.pid file.
I also used valgrind -v and there is no helpful information there either.
Any thoughts on why this doesn't work for 7-Zip?
/usr/bin/7z is a wrapper script.
#! /bin/sh
exec /usr/lib/p7zip/7z "$#"
Try running
valgrind --tool=massif /usr/lib/p7zip/7z a filename.7z filename
instead.

How do you start running the program over again in gdb with 'target remote'?

When you're doing a usual gdb session on an executable file on the same computer, you can give the run command and it will start the program over again.
When you're running gdb on an embedded system, as with the command target localhost:3210, how do you start the program over again without quitting and restarting your gdb session?
You are looking for Multi-Process Mode for gdbserver and set remote exec-file filename
Unfortunately, I don't know of a way to restart the application and still maintain your session. A workaround is to set the PC back to the entry point of your program. You can do this by either calling:
jump function
or
set $pc=address.
If you munged the arguments to main you may need set them up again.
Edit:
There are a couple of caveats with the above method that could cause problems.
If you are in a multi-threaded program jumping to main will jump the current thread to main, all other threads remain. If the current thread held a lock...then you have some problems.
Memory leaks, if you program flow allocates some stuff during initialization then you just leaked a bunch of memory with the jump.
Open files will still remain open. If you mmap some files or an address, the call will most likely fail.
So, using jump isn't the same thing as restarting the program.
"jump _start" is the usual way.
Presumably you are running gdbserver on the embedded system.
You can ask it to restart your program instead of exiting with target extended-remote
Step-by-step procedure
Remote:
# pwd contains cross-compiled ./myexec
gdbserver --multi :1234
Local:
# pwd also contains the same cross-compiled ./myexec
gdb -ex 'target extended-remote 192.168.0.1:1234' \
-ex 'set remote exec-file ./myexec' \
--args ./myexec arg1 arg2
(gdb) r
[Inferior 1 (process 1234) exited normally]
(gdb) r
[Inferior 1 (process 1235) exited normally]
(gdb) monitor exit
Tested in Ubuntu 14.04.
It is also possible to pass CLI arguments to the program as:
gdbserver --multi :1234 ./myexec arg1 arg2
and the ./myexec part removes the need for set remote exec-file ./myexec, but this has the following annoyances:
undocumented: https://sourceware.org/bugzilla/show_bug.cgi?id=21981
does not show on show args and does not persist across restarts: https://sourceware.org/bugzilla/show_bug.cgi?id=21980
Pass environment variables and change working directory without restart: How to modify the environment variables and working directory of gdbserver --multi without restarting it?
If you are running regular gdb you can type 'run' shortcut 'r' and gdb asks you if you wish to restart the program
For me the method described in 21.2 Sample GDB session startup works great. When I enter monitor reset halt later at the “(gdb)” prompt the target hardware is reset and I can re-start the application with c (= continue).
The load command can be omitted between the runs because there is no need to flash the program again and again.
You can use jump gdb command. For that, you can check your startup script.
My startup script has a symbol.
.section .text.Reset_Handler
.weak Reset_Handler
.type Reset_Handler, %function
Reset_Handler:
ldr r0, =_estack
mov sp, r0 /* set stack pointer */
I wanted to jump to start. That's why I used:
jump Reset_Handler
On EFM32 Happy Gecko none of the suggestions would work for me, so here is what I have learned from the documentation on integrating GDB into the Eclipse environment.
(gdb) mon reset 0
(gdb) continue
(gdb) continue
This puts me in the state that I would have expected when hitting reset from the IDE.