Running a command on multiple files using TCL script - file-io

I am using a certain tool (Synopsys Design Vision) which I am running by using a TCL script.
The command is something like this
analyze -library work -format verilog {/user/codes/abcd.v }
Now I have a folder with 5000 .v extension files with names abcd_1, abcd_2, abcd_3.....abcd_5000. Now I want to automate the script so that the command works on each and very file.
I tried to do something like this
for { set a 1} {$a < 5001} {incr a} {
analyze -library work -format verilog {/user/codes/abcd_$a.v }
#other commands
.
.
}
However, it is not working and giving error. I have not worked with tcl scripts that much.
Kindly specify where I am going wrong and what can I do to make it work. Thank you.

To cut it short, use quotes rather than braces, as the latter prevent variable substitution from happening:
for {set a 1} {$a < 5001} {incr a} {
analyze -library work -format verilog "/user/codes/abcd_$a.v"
# ...
}
However, you might also want to check out the file command to assemble file paths.

Related

How does the variable scope in BitBake works

I use Yocto and I was wondering how the variable scope works in a BitBake recipe:
My recipe looks like:
SRC_URI += "file://something"
python do_fetch_prepend() {
d.appendVar("SRC_URI", "https://www.bla.com/resource.tar")
bb.error("SRC_URI_1: %s " % d.getVar("SRC_URI"))
d.setVar("TEST_VAR", "test")
}
python do_unpack_append() {
bb.error("SRC_URI_2: %s " % d.getVar("SRC_URI"))
bb.error("TEST_VAR: %s " % d.getVar("TEST_VAR"))
}
I run bitbake -v -c unpack myrecipe
SRC_URI_1 is printed as expected: "file://something https://www.bla.com/resource.tar"
SRC_URI_2 is printed as: "file://something"
TEST_VAR is printed as: None
I looks like setting/changing a variable in datastore (d) is only done in the scope of the do_fetch. Is this expected behaviour, because I read in the documentation that 'd' is global variable.
If this is expected behaviour, is there away to change global variables in a task of a recipe?
The reason behind the question is that I need another native-recipe before I can add the extra URI to the SRC_URI. I tried first Inline Python Variable Expansion, but the BitBake parser already expanse the variable before the native recipe is put in 'native directory'. So I try to change the SRC_URI during fetch task and I 'load' my native recipe as follow:
python () {
d.appendVarFlag('do_parse', 'depends', 'my-recipe-native:do_populate_sysroot')
}
In the do_fetch_prepend I use this native recipe which gives me the correct URL, which I wanted to append to the SRC_URI. So the fetching, unpacking, cleaning, etc works. It looks like I fetching works, but the unpacking not because the SRC_URI is not updated.
With a given task, variable changes are only local. This means do_unpack does not 'see' a change made by the do_fetch task.
This is necessary to allow some tasks to rerun when others are covered by sstate, to ensure things are deterministic.
If you really want to do what you describe, you'd need something like a prefunc for the tasks where you need to modify SRC_URI.
python myprefunc() {
d.appendVar("SRC_URI", "https://www.bla.com/resource.tar")
}
do_fetch[prefuncs] += "myprefunc"
do_unpack[prefuncs] += "myprefunc"
However note that whilst this will do some of what you want, source archives, license manifests and sstate checksums may not work correctly since you're "hiding" source data from bitbake and this data is only present at task execution time, not parse time.

Make a module out a bunch of .f source files

I downloaded some Fortran source code which contains around 50 or so .f files. Each .f file contains one subroutine or function. I'd like to conveniently stuff all of these into a Fortran module .mod file. Is there any way of doing this with gfortran (besides painstakingly writing out each sub/func prototype inside a module definition)?
This is more of a formatted comment than an answer, but you could try writing a source file something like this
module this_will_end_in_tears
contains
include 'subroutine1.f'
include 'subroutine2.f'
include 'subroutine2.f'
...
end module this_will_end_in_tears
If, as I suspect, the .f files contain fixed-form source, then you had better make sure the module is also in fixed-form.
I expect, as you may have guessed, that this will not compile the first time you try it, but it might (just might) save you a little time over doing the job right by, as you put it, painstakingly writing out each sub/func ...
Oh, and lest any of the Fortran-lovers out there cringe at this suggestion, note that I'm not saying it's a good way to proceed, just that it might save you a little time.
Good luck.
I made this bash script that takes two parameters 'the module name' and 'the file extension' then builds a module file. It's presently made for fixed-format. All the source files must be present in the same directory as the script. Of course you can modify it to be more flexible.
name=$1
ext=$2
modsrc=$name.$ext
echo -e " MODULE $name\n" > $modsrc
echo -e " implicit none\n" >> $modsrc
echo -e " contains\n" >> $modsrc
for i in *.f;
do
if [ "$i" != "$modsrc" ]; then
echo -e " include '$i'\n" >> $modsrc
fi
done
echo -e " END MODULE $name\n" >> $modsrc
I usually tackle things by compiling even .f90 code as -fixed. Then usually add in INTENT and IMPLICIT NONE. Almost always I have D-Lines for debug which seems to work easiest for me with -fixed form. Later adding in alignment pragmas and OpenMP reduction clauses and related vector stuff.

How do I escape a "$" in bitbake/yocto?

One of my recipes in Yocto need to create a file containing a very specific line, something like:
${libdir}/something
To do this, I have the recipe task:
do_install() {
echo '${libdir}/something' >/path/to/my/file
}
Keeping in mind that I want that string exactly as shown, I can't figure out how to escape it to prevent bitbake from substituting in its own value of libdir.
I originally thought the echo command with single quotes would do the trick (as it does in the bash shell) but bitbake must be interpreting the line before passing it to the shell. I've also tried escaping it both with $$ and \$ to no avail.
I can find nothing in the bitbake doco about preventing variable expansion, just stuff to do with immediate, deferred and Python expansions.
What do I need to do to get that string into the file as is?
Bitbake seems to have particular issues in preventing expansion from taking place. Regardless of whether you use single or double quotes, it appears that the variables will be expanded before being passed to the shell.
Hence, if you want them to not be expanded, you need to effectively hide them from BitBake, and this can be done with something like:
echo -e '\x24{libdir}/something' >/path/to/my/file
This uses the hexadecimal version of $ so that BitBake does not recognise it as a variable to be expanded.
You do need to ensure you're running the correct echo command however. Under some distros (like Ubuntu), it might run the sh-internal echo which does not recognise the -e option. In order to get around that, you may have to run the variant of echo that lives on the file system (and that does recognise that option):
/bin/echo -e '\x24{libdir}/something' >/path/to/my/file
By default this task will be executed as shell function via /bin/sh, but it depends on your system what it will be as you can have a symlink named /bin/sh pointing to bash. The BitBake's manual prevents from using bashism syntax though.
You can consider just adding this task in your recipe as python function:
python do_install () {
with open('/path/to/your/file', 'a') as file:
file.write('${libdir}/something')
}
'a' stands for append.
This should eliminate the problem with variable expansion.
There is no standard way to escape these sorts of expressions that I am aware of, other than to try to break up the expression - accordingly this should work:
do_install() {
echo '$''{libdir}/something' >/path/to/my/file
}
The best solution is simply this:
bitbake_function() {
command $libdir/whatever
}
Bitbake will only expand ${libdir}; $libdir is passed through verbatim.
We don't have to worry about dollar signs that are not followed by {, and in this case, there is no need for libdir to be wrapped in braces.
The only time we run into a problem with just $foo is if we have something like ${foo}bar where the braces are required as delimiters so that bar isn't included into the variable name. In that situation, there are other solutions, such as for instance generating the shell syntax "$foo"bar. This is less cryptic than resorting to \x24.
If you need to use $ in variable assignment, remember that bitbake won't evaluate $whatever so you have to escape it for the underlying shell.
For instance I set gcc/ld Rpath option to use $ORIGIN keyword this way:
TARGET_LDFLAGS_append = " -Wl,-rpath-link=\\$$ORIGIN"
https://lists.yoctoproject.org/pipermail/yocto/2017-September/037820.html
You can define a variable to be a literal dollar sign.
DOLLAR = "$"
do_install() {
echo '${DOLLAR}{libdir}/something' >/path/to/my/file
}
no extra quoting required.

In Lua, how to print the console output into a file (piping) instead of using the standard output?

I workin' with Torch7 and Lua programming languages. I need a command that redirects the output of my console to a file, instead of printing it into my shell.
For example, in Linux, when you type:
$ ls > dir.txt
The system will print the output of the command "ls" to the file dir.txt, instead of printing it to the default output console.
I need a similar command for Lua. Does anyone know it?
[EDIT] An user suggests to me that this operation is called piping. So, the question should be: "How to make piping in Lua?"
[EDIT2] I would use this # command to do:
$ torch 'my_program' # printed_output.txt
Have a look here -> http://www.lua.org/pil/21.1.html
io.write seems to be what you are looking for.
Lua has no default function to create a file from the console output.
If your applications logs its output -which you're probably trying to do-, it will only be possible to do this by modifying the Lua C++ source code.
If your internal system has access to the output of the console, you could do something similar to this (and set it on a timer, so it runs every 25ms or so):
dumpoutput = function()
local file = io.write([path to file dump here], "w+")
for i, line in ipairs ([console output function]) do
file:write("\n"..line);
end
end
Note that the console output function has to store the output of the console in a table.
To clear the console at the end, just do os.execute( "cls" ).

Zsh trouble when using echo with color/formatting characters

I'm just switch to zsh and now adapting the alias in which was printing some text (in color) along with a command.
I have been trying to use the $fg array var, but there is a side effect, all the command is printed before being executed.
The same occur if i'm just testing a echo with a color code in the terminal:
echo $fg_bold[blue] "test"
]2;echo "test" test #the test is in the right color
Why the command print itself before to do what it's supposed to do ? (I precise this doesn't happen when just printing whithout any wariable command)
Have I to set a specific option to zsh, use echo with a special parameter to get ride of that?
Execute the command first (keep its output somewhere), and then issue echo. The easiest way I can think of doing that would be:
echo $fg[red] `ls`
Edit: Ok, so your trouble is some trash before the actual output of echo. You have some funny configuration that is causing you trouble.
What to do (other than inspecting your configuration):
start a shell with zsh -f (it will skip any configuration), and then re-try the echo command: autoload colors; colors; echo $fg_bold[red] foo (this should show you that the problem is in your configuration).
Most likely your configuration defines a precmd function that gets executed before every command (which is failing in some way). Try which precmd. If that is not defined, try echo $precmd_functions (precmd_functions is an array of functions that get executed before every command). Knowing which is the code being executed would help you search for it in your configuration (which I assume you just took from someone else).
If I had to guess, I'd say you are using oh-my-zsh without knowing exactly what you turned on (which is an endless source of troubles like this).
I don't replicate your issue, which I think indicates that it's either an option (that I've set), or it's a zsh version issue:
$ echo $fg_bold[red] test
test
Because I can't replicate it, I'm sure there's an option to stop it happening for you. I do not know what that option is (I'm using heavily modified oh-my-zsh, and still haven't finished learning what all the zsh options do or are).
My suggestions:
You could try using print:
$ print $fg_bold[red] test
test
The print builtin has many more options than echo (see man zshbuiltins).
You should also:
Check what version zsh you're using.
Check what options (setopt) are enabled.
Check your ~/.zshrc (and other loaded files) to see what, if any, options and functions are being run.
This question may suggest checking what TERM you're using, but reading your question it sounds like you're only seeing this behaviour (echoing of the command after entry) when you're using aliases...?