Dynamic import in Jsonnet - dynamic

I want to get input file in Jsonnet so the following is working great for me:
local input = import './inputfile.json';
Problem is that I want to pass the file name through the Jsonnet CLI and I tried to use --ext-str or TLA but in both cases im getting the following error:
computed imports are not allowed.
I also tried to use --ext-code like here:
jsonnet -J grafonnet-lib --ext-code input=(import "./inputfile.json") createDash.jsonnet
but then I'm getting:
zsh: unknown file attribute: i
Is there any solution for this problem?

I'm guessing zsh is gobbling up either the parenthesis or the double quotes. Drop both between single quotes.
➜ tmp jsonnet -J grafonnet-lib --ext-code input='(import "./inputfile.json")' createDash.jsonnet
{ }

Related

unsupported format character error in youtube-dl

i've been trying to download a youtube playlist using youtube-dl, but i had a problem in the output template, i used the following command to get the playlist downloaded in an organised way :
youtube-dl -f mp4 -o "Desktop/mainFolder/courses/%(playlist_title)s-%(playlist_uploader)/%(title)s.%(ext)s" --embed-thumbnail --add-metadata --mark-watched https://www.youtube.com/playlist?list=PLzH6n4zXuckpfMu_4Ff8E7Z1behQks5ba
but i kept getting the following error :
ERROR: Error in output template: unsupported format character '/' (0x2f) at index 66 (encoding: 'UTF-8')
it states that i used an unsupported character which is '/', weirdly enough i used almost the same output template format in a previous download which is :
youtube-dl -f mp4 -o "Desktop/mainFolder/courses/%(playlist)s/%(playlist_index)s - %(title)s.%(ext)s" --add-metadata https://www.youtube.com/playlist?list=PL4C9296DF81B9EF13
and it worked just fine.
what did i did differently here so that the first command didn't work but the last one did ??
If you see this error it probably means one of the format expressions doesn't end with an s.
In this case, it looks like you're missing one after (playlist_uploader).
youtube-dl -f mp4 -o "Desktop/mainFolder/courses/%(playlist_title)s-%(playlist_uploader)s/%(title)s.%(ext)s" --embed-thumbnail --add-metadata --mark-watched https://www.youtube.com/playlist?list=PLzH6n4zXuckpfMu_4Ff8E7Z1behQks5ba

Snakemake - input function exception

I am trying to run snakemake code using.json file as input. While checking the dry run i got foloowing error
InputFunctionException in line 172 of /home/Snakefile_ChIPseq_pe:
KeyError: '130241_1'
Wildcards:
library=130241_1
This is the part of snakemake code
rule findPeaks:
input:
sample = os.path.join(HOMERTAG_DIR, "{library}"),
input = lambda wildcards: os.path.join(HOMERTAG_DIR, config['lib_input'][wildcards.library])
output:
os.path.join(HOMERPEAK_DIR, "{library}.all.hpeaks")
params:
config['homer_findPeaks_params']
shell:
"findPeaks {input.sample} -i {input.input} {params} -o {output}"
There is single quote around input sample which is missing in the 'lib_input' part. How to add that single quote ahead of variable?
Also library names are like 12345_1, 12345_2 etc., never had this problem before however for the first time I have libraries with "underscore" in the names.
Snakemake will first try to interpret the given value as number. Only if that fails, it will interpret the value as string. Here, it does not fail, because the underscore _ is interpreted as thousand separator.
My guess is that in your json file the library IDs are not quoted. E.g. you have this:
{
"lib_input": {1234_1: "input.txt"}
}
Instead of:
{
"lib_input": {"1234_1": "input.txt"}
}
Or maybe library 130241_1 is not in the json at all?

Using sql within shell script

I am currently trying to integrate an sql statement into a shell script, But facing major syntax issue:
My statement in the script:
su - <sid>adm -c 'hdbsql -U SYSTEM export "'SCHEMA'"."'*'" as binary into "'Export Location'" with reconfigure'
I get the following error:
* 257: sql syntax error: incorrect syntax near "*": line 1 col 16 (at pos 16) SQLSTATE: HY000
Would really appreciate if anyone could help me with this.
Thanks and Regards,
AK
Your command line doesn't make much sense to me. It starts with
su - <sid>adm
which means that you are redirecting the contents of the file "sid" into "su" and then redirecting the result of that operation into the file "adm".
Second problem is that in the command you are giving to adm, the single quotes end right before the "" which means, that the "" will get interpreted by the shell as a file glob:
-c 'hdbsql -U SYSTEM export "'SCHEMA'"."'*'" as binary into "'Export Location'" with reconfigure'
You'll need to escape those single quotes like this: "\'".
But I think your problem solving approach is not good. Try to reduce to problem and only then start adding additional things to it. So first try to execute the SQL statement from the "hdbsql" shell. Does it work?
$ hdbsql
> YOUR SQL STATEMENT HERE
Once that works, try to execute the SQL statement from the unix shell as a user:
$ hdbsql -U SYSTEM export ...
Once that works, try to execute it via su
$ su - ...

dynamically fetching dynamic variable's value from properties file

Below unix commands works:
export myTempVar=myTempVar1
export myTempVar1=myTempVar2
eval echo '$'$myTempVar
This correctly prints myTempVar2.
However, what if myTempVar1=myTempVar2 is present in a properties file instead of directly in the script.
So my script will have
. $MYDIR/myProperties.properties
myTempVar=myTempVar1
myTempVar3=eval echo '$'$myTempVar
Above lines are not working and the value of myTempVar3 is not coming as myTempVar2.
myProperties.properties is having below line:
myTempVar1=myTempVar2
Using indirection is far safer than eval:
#!/bin/bash
. $MYDIR/myProperties.properties # myTempVar1=myTempVar2
myTempVar=myTempVar1
myTempVar3=${!myTempVar}
echo $myTempVar3
Gives:
myTempVar2
and you don't need the echo in eval:
eval myTempVar3='$'$myTempVar

How can I pass command-line parameters with whitespace to an apache pig script?

I want to write a pig script that takes a filter condition as a command line parameter. From the command line I want to type something like:
pig -p "MY_FILTER=field1 == 0 and field2 == 5" myscript.pig
In my script I have a line:
my_filtered_data = filter my_data by $MY_FILTER;
This works as expected when MY_FILTER has no spaces and I pass quotes around my value; So if I type MY_FILTER=\"field1==0\" at the command line the shell will pass the quotes with the value and pig does the expansion I want. However, the parameter will fail to expand if I supply it like MY_FILTER=\"field1 == 0\"
I've tried a bunch of different quoting techniques and even tried running the command directly from python's subprocess module to ensure my shell wasn't doing something weird.
Which version of Pig do you use? I use 0.9.2 and the following command works for me:
pig -p "F='field1 == 3 AND field2 == 5'" test.pig
But it doesn't work with 0.8.1.