I'm trying to run a simple test with llvm-lit. I have a dedicated directory:
llvm-lit-dir
+---lit.cfg
+---llvm_lit_example.c
The cfg file is from llvm-3.8.0/utils/lit/lit/ExampleTests.ObjDir.
Here it is:
$ cat lit.cfg
config.example_obj_root = os.path.dirname(__file__)
lit.load_config(config, os.path.join(config.test_source_root,'lit.cfg'))
Here is the example I'm trying to check:
$ cat llvm_lit_example.c
// RUN: %clang -o %t0 %s
// RUN: %t0 | grep "YES"
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
int main(int argc, char **argv)
{
int i=rand()%4;
char s1[5]={'0','1','2','3', 0 };
char s2[5]={'0','1','2','3', 0 };
s1[i]='6';
if (strcmp(s1,s2) == 0) printf("NO \n");
if (strcmp(s1,s2) != 0) printf("YES\n");
}
I've tried several options but none seem to work:
$ llvm-lit llvm-lit-dir/llvm_lit_example.c
llvm-lit: TestingConfig.py:114: fatal: unable to parse config file '/home//llvm-lit-dir/lit.site.cfg', traceback: Traceback (most recent call last):
File "/home/llvm-3.8.0/llvm/utils/lit/lit/TestingConfig.py", line 101, in load_from_path
exec(compile(data, path, 'exec'), cfg_globals, None)
File "/home/llvm-lit-dir/lit.site.cfg", line 14, in <module>
lit.load_config(config, os.path.join(config.test_source_root,'lit.cfg'))
NameError: name 'lit' is not defined
What am I doing wrong? Thanks!
The following is the minimalistic "hello world" example which is now working for me:
config file: lit.cfg
import lit.formats
config.name = "LIT hello world"
config.test_format = lit.formats.ShTest("0")
test file: tests/test
; RUN: echo "Foo" | FileCheck %s
; CHECK: Foo
run it with
lit -v tests/test
To make the test fail, change it to echo "Bar".
Two additional comments:
1) Installing lit is easy:
pip3 install lit
To install FileCheck I had to download LLVM source code (LLVM Download page) and build the FileCheck target using CMake.
2) config.test_format = lit.formats.ShTest("0") is important because otherwise lit fails with the following error:
AttributeError: 'NoneType' object has no attribute 'execute'
It looks like ShTest is a default format, but a user still has to activate it manually.
This config is a python source. lit there is clearly undefined. Try importing something related to lit, like from lit.llvm import llvm_config. Take a look at test/lit.cfg.py in LLVM source.
Related
I have mnist.loader.hpp header file that opens and reads t10k-images-idx3-ubyte and t10k-labels-idx1-ubyte binary files in a given path using std::ifstream as can be seen in the snippet below:
std::string imagePath = dataDir + std::string("t10k-images-idx3-ubyte");
std::string labelPath = dataDir + std::string("t10k-labels-idx1-ubyte");
std::ifstream imageStream(imagePath, std::ios::binary);
std::ifstream labelStream(labelPath, std::ios::binary);
The path of the directory having those two binary files dataDir is embedded in mnist_caffe.cpp. It could be changed as per my setting in the std::string dataDir = "data/" line.
I set dataDir to /home/mohamed/, then compiled and statically-linked an aarch64 executable out of them and my target is to simulate it on gem5.
The problem:
gem5 always fails to read the files. It gives "Failed to read /home/mohamed/t10k-images-idx3-ubyte".
If we have a look once again at mnist_loader.hpp, we will notice that this error message is due to either the file not being open as or being opened but can't be read :
if (!imageStream.is_open())
{
std::cerr << "Failed to load " << imagePath << std::endl;
return nullptr;
}
or
imageStream.read(reinterpret_cast<char*>(&magic), sizeof(magic));
if (magic != 0x03080000)
{
std::cerr << "Failed to read " << imagePath << std::endl;
return nullptr;
}
or
if (!imageStream.good())
{
std::cerr << "Failed to read " << imagePath << std::endl;
return nullptr;
}
My guess is that gem5 can not open the file in the first place!
My trials:
(I am only interested in opening and reading t10k-images-idx3-ubyte for now)
1- Trying to pass the file to the stdin using -i option: $ ./build/ARM/gem5.opt ./configs/example/se.py -c mnist_caffe -i '/home/mohamed/t10k-images-idx3-ubyte'
2- Using -o: $ ./build/ARM/gem5.opt ./configs/example/se.py -c mnist_caffe -o "-i /home/mohamed/t10k-images-idx3-ubyte"
3- Using --redirects: $ ./build/ARM/gem5.opt ./configs/example/se.py -c /home/mohamed/NNDeploy/ML-examples/armnn-mnist/mnist_caffe --redirects /home/mohamed/t10k-images-idx3-ubyte=/home/mohamed/t10k-images-idx3-ubyte
4- Opening the binary file in the host system using xxd and then passing it to stdin: $ ./build/ARM/gem5.opt ./configs/example/se.py -c ~/NNDeploy/ML-examples/armnn-mnist/mnist_caffe -o " -i /usr/bin/xxd /home/mohamed/t10k-images-idx3-ubyte"
All the above resulted in the same failed-to-read message. I ran out of ideas and it would be great if someone could provide some suggestions!
All I can find on this topic is mentions of FSMoveObjectToTrashSync function, which is now deprecated and no alternative is listed for it.
How to do it from C or Objective-C code?
Use NSFileManager:
https://developer.apple.com/documentation/foundation/nsfilemanager
trashItemAtURL:resultingItemURL:error:
Moves an item to the trash.
In C, you can use AppleScript to move files to the trash. Here's a simple example:
#include <stdio.h>
#include <stdlib.h>
#define PATH "/tmp/"
#define NAME "delete-me.txt"
int main() {
int status;
/* Create a file */
FILE *f;
f = fopen(PATH NAME, "w");
if (!f) {
fputs("Can't create file " PATH NAME "\n", stderr);
return 1;
}
fputs("I love trash\n", f);
fclose(f);
/* Now put it in the trash */
status = system(
"osascript -e 'set theFile to POSIX file \"" PATH NAME "\"' "
"-e 'tell application \"Finder\"' "
"-e 'delete theFile' "
"-e 'end tell' "
">/dev/null"
);
if (status == 0) {
puts("Look in the trash folder for a file called " NAME);
}
else {
puts("Something went wrong. Unable to delete " PATH NAME);
}
return 0;
}
A few notes:
Multi-line scripts have to be sent as multiple -e command line options.
Since osascript insists on printing status messages to the command line console, I've redirected its output to /dev/null. But, if a file of the same name already exists in the trash, then the deleted file will be renamed. If you need to know this name, you'll have to use popen() instead of system() and parse the return string from osascript.
I am trying to trouble shoot an awk script and having trouble with the gawk64 command.
gawk64 -f awkmakearchive.txt -v SPD=abc.csv -v REF=def.csv ijk.csv
In the awkmakearchive.txt file i am getting syntax error for the following
line :
BEGIN {
FS = ","
# Load SPD
spdcount = 0
while (getline < SPD) {
for (i=0; i<96; i++) {
Pattern[$1][i] = $(i+2) ----> Syntax error
}
spdcount++
}
....
...
}
Please help.
Solved! I was using GnuWin32 to run the script instead of GnuWin 64 bit version. GnuWin32 did not have gawk64 in its bin.
I am starting a toy compiler, and I am making the simplest thing I can imagine, but it won't work.
Lex compiles, and Yacc compiles, and they link together, but the outputted program does not do what I expected.
Lex:
%{
#include <stdlib.h>
void yyerror(char *);
#include "y.tab.h"
%}
%%
a {
yylval = atoi(yytext);
return AAA;
}
. yyerror("invalid character");
%%
int yywrap(void) {
return 1;
}
Yacc:
%{
void yyerror(char *);
int yylex(void);
int sym[26];
#include <stdio.h>
%}
%token AAA
%%
daaaa:
AAA {printf("%d\n", $1);}
%%
void yyerror(char *s) {
fprintf(stderr, "%s\n", s);
}
int main(void) {
yyparse();
return 0;
}
The program I am trying to compile with this compiler is a file containing: a. that's it.
I don't know what's happened!
Clarification: What I expected the compiled compiler to do was to accept a file into it, process the file, and spit out a compiled version of that file.
Can you explain, maybe in an answer, exactly what you did, and how it worked, because as far as I can tell, and as far as I have tested the question, it shouldn't work as you say.
I took your code verbatim, creating files grammar.y and lexer.l. I then compiled the code. I'm working on Mac OS X 10.11.4, using GCC 6.1.0, Bison 2.3 (disguised as yacc) and Flex 2.5.35 (disguised as lex).
$ yacc -d grammar.y
$ lex lexer.l
$ gcc -o gl y.tab.c lex.yy.c
$ ./gl <<< 'a'
0
$
I subsequently made two changes. In grammar.y, I changed main() to:
int main(void) {
#if YYDEBUG
yydebug = 1;
#endif
yyparse();
return 0;
}
and in lexer.l, I changed the default character rule to:
\n|. yyerror("invalid character");
(The . doesn't match newline, so the newline after the a in the input was echoed by default in the original output.)
With a similar compilation, the output becomes:
$ ./gl <<< 'a'
0
invalid character
$
With the compilation specifying -DYYDEBUG too:
$ gcc -DYYDEBUG -o gl lex.yy.c y.tab.c
$
the output includes useful debugging information:
$ ./gl <<< 'a'
Starting parse
Entering state 0
Reading a token: Next token is token AAA ()
Shifting token AAA ()
Entering state 1
Reducing stack by rule 1 (line 12):
$1 = token AAA ()
0
-> $$ = nterm daaaa ()
Stack now 0
Entering state 2
Reading a token: invalid character
Now at end of input.
Stack now 0 2
Cleanup: popping nterm daaaa ()
$ ./gl <<< 'aa'
Starting parse
Entering state 0
Reading a token: Next token is token AAA ()
Shifting token AAA ()
Entering state 1
Reducing stack by rule 1 (line 12):
$1 = token AAA ()
0
-> $$ = nterm daaaa ()
Stack now 0
Entering state 2
Reading a token: Next token is token AAA ()
syntax error
Error: popping nterm daaaa ()
Stack now 0
Cleanup: discarding lookahead token AAA ()
Stack now 0
$
The second a in the input correctly triggers a syntax error (it isn't allowed by the grammar). Other characters are permitted, generate a 'invalid character' message, and are otherwise ignored (so ./gl <<< 'abc' generates 3 invalid character messages, one for the b, one for the c, and one for the newline).
Changing the assignment to yylval in lexer.l to:
yylval = 'a'; // atoi(yytext);
changes the number printed from 0 to 97, which is the character code for 'a' in ASCII, ISO 8859-1, Unicode, etc.
I've been using a here string as the source of data. It would be equally feasible to have used a file as the input:
$ echo a > program
$ cat program
a
$ ./gl < a
Starting parse
Entering state 0
Reading a token: Next token is token AAA ()
Shifting token AAA ()
Entering state 1
Reducing stack by rule 1 (line 12):
$1 = token AAA ()
97
-> $$ = nterm daaaa ()
Stack now 0
Entering state 2
Reading a token: invalid character
Now at end of input.
Stack now 0 2
Cleanup: popping nterm daaaa ()
$
If you want to read files specified by name on the command line, you have to write more code in main() to process those files.
The program does not accept a file because it was not told to.
In the Yacc program,
extern FILE *yyin; must be added in the definitions section.
I believe that's it.
I have now whittled this down to a minimal test case. Thus far I have been able to determine that this is an issue related to pseudo-terminals which come about with the pipe of ssh. Adding the '-t -t' to the ssh call improved things, in that now, it takes a second call to fgets() to cause the issue. I suspect that the stderr output of the ssh command somehow works into the issue, for now I have redirected stderr to stdout in the ssh code to execute. I do wonder if the "tcgetattr: Invalid argument" error is part of the problem, but am not sure how to get rid of that. It seems to come from the -t -t being present. I believe the -t -t is moving in the right direction, but I have to set up the pseudo terminal for stderr somehow and perhaps the test will work properly?
The Makefile:
test:
gcc -g -DBUILD_MACHINE='"$(shell hostname)"' -c -o test.o test.c
gcc -g -o test test.o
.PHONY: clean
clean:
rm -rf test.o test
The test.c source file:
#include <unistd.h>
#include <string.h>
#include <stdio.h>
int
main(int argc, char *argv[])
{
const unsigned int bufSize = 32;
char buf1[bufSize];
char buf2[bufSize];
int ssh = argv[1][0] == 'y';
const char *cmd = ssh ? "ssh -t -t " BUILD_MACHINE " \"ls\" 2>&1" : "ls";
FILE *fPtr = popen(cmd, "r");
if (fPtr == NULL) {
fprintf(stderr,"Unable to spawn command.\n");
perror("popen(3)");
exit(1);
}
printf("Command: %s\n", cmd);
if (feof(fPtr) == 0 && fgets(buf2, bufSize, fPtr) != NULL) {
printf("First result: %s\n", buf2);
if (feof(fPtr) == 0 && fgets(buf2, bufSize, fPtr) != NULL) {
printf("Second result: %s\n", buf2);
int nRead = read(fileno(stdin), buf1, bufSize);
if (nRead == 0) {
printf("???? popen() of ssh consumed the beginning of stdin ????\n");
} else if (nRead > 0) {
if (strncmp("The quick brown fox jumped", buf1, 26) != 0) {
printf("??? Failed ???\n");
} else {
printf("!!!!!!! Without ssh popen() did not consume stdin !!!!!!!\n");
}
}
}
}
}
This shows it running the passing way:
> echo "The quick brown fox jumped" | ./test n
Command: ls
First result: ARCH.linux_26_i86
Second result: Makefile
!!!!!!! Without ssh popen() did not consume stdin !!!!!!!
This shows it running the failing way:
> echo "The quick brown fox jumped" | ./test y
Command: ssh -t -t hostname "ls" 2>&1
First result: tcgetattr: Invalid argument
Second result: %backup%~ gmon.out
???? popen() of ssh consumed the beginning of stdin ????
Okay, I have got this working finally. The secret was to supply /dev/null as the input to my ssh command as follows from the test case above:
const char *cmd
= ssh ? "ssh -t -t " BUILD_MACHINE " \"ls\" 2>&1 < /dev/null" : "ls";
However, while the code works correctly, I get a nasty message which apparently I can ignore for my purposes (although I'd like to make the message go away):
tcgetattr: Inappropriate ioctl for device