How can I add this to the repl with line returns?
import Dict
fruit = Dict.fromList \
[ \
((0,0), 'Apple') \
,((0,1), ' ') \
]
Error:
> fruit = Dict.fromList \
| [ \
| ((0,0), 'Apple') \
| ,((0,1), ' ') \
-- SYNTAX PROBLEM -------------------------------------------- repl-temp-000.elm
The = operator is reserved for defining variables. Maybe you want == instead? Or
maybe you are defining a variable, but there is whitespace before it?
5| fruit = Dict.fromList
Is it just not possible to do this in the repl with lists where you want to add line returns?
Not a language I know but thought I'd take a look. This appears to work:
import Dict
fruit = Dict.fromList \
[ \
((0,0), "Apple") \
,((0,1), " ") \
]
You seem to have some trailing whitespace after ,((0,1), ' ') \
Also I needed double quotes which appears to be supported by https://elmprogramming.com/string.html
By way of minimal test - this behaves like your example if the trailing space is included:
import Dict
fruit = Dict.fromList [ \
Related
How to construct a name of a variable from data and access that variable?
For example it should somehow give the content of the alpha-file:
jq '$.var"-file"' --slurpfile alpha-file <(echo 0) --slurpfile beta-file <(echo 1) <<<'{"var": "alpha"}'
It should output:
[
0
]
Named arguments are also available to the jq program as $ARGS.named.
So there is a dictionary to extract variables by a string name from:
jq '$ARGS.named["\(.var)-file"]' --slurpfile alpha-file <(echo 0) --slurpfile beta-file <(echo 1) <<<'{"var": "alpha"}'
outputs
[
0
]
You could use getpath like this:
jq --slurpfile alphaFile <(echo 0) --slurpfile betaFile <(echo 1) '
.var as $ab | {alpha: $alphaFile, beta: $betaFile} | getpath([$ab])' <<< '{"var": "alpha"}'
Better yet:
jq --slurpfile alpha <(echo 0) --slurpfile beta <(echo 1) '
.var as $ab | {$alpha, $beta} | getpath([$ab])' <<< '{"var": "alpha"}'
I have a file that looks like this:
abc \
d \
efgh \
i
jklmnop \
q \rst \
uv
wx
y \
z
For each group of consecutive lines that have a backslash at the end, I want to arrange those backslashes in a straight vertical line. So the expected output for above sample is:
abc \
d \
efgh \
i
jklmnop \
q \rst \
uv
wx
y \
z
I managed to align all backslashes with this program:
$ awk '/\\$/ { sub(/\\$/,""); printf "%-20s\\\n",$0; next} 1' file
abc \
d \
efgh \
i
jklmnop \
q \rst \
uv
wx
y \
z
But I have no idea how to proceed from here, so I'm asking for guidance. I tried searching on SO, but top results were all about removing trailing backslashes.
Details about the actual input:
Lines may contain any character including backslash and consist of any number of characters, there is no limit.
There might be multiple blanks and tabs before and after the last backslash.
There is always at least one blank or tab preceding the last backslash.
There is no line that consists of only a backslash and zero or more blanks/tabs around it.
P.S. I'm not looking for a Perl solution.
If you don't have any tabs before the spaces-then-backslash-then-spaces at the end of each line:
$ cat tst.awk
FNR == 1 { prevHasEsc = blockNr = 0 }
hasEsc = match($0,/[[:space:]]*\\[[:space:]]*$/) {
$0 = substr($0,1,RSTART-1)
if ( ! prevHasEsc ) {
++blockNr
}
}
{ prevHasEsc = hasEsc }
NR == FNR {
if ( hasEsc ) {
lgth[blockNr] = (lgth[blockNr] > length($0) ? lgth[blockNr] : length($0))
}
next
}
hasEsc {
$0 = sprintf("%-*s \\", lgth[blockNr], $0)
}
{ print }
.
$ awk -f tst.awk file file
abc \
d \
efgh \
i
jklmnop \
q \
r
If you do then I'd suggest running the text through pr -e -t first to convert tabs to the corresponding number of blanks.
Here is one attempt using gnu-awk with a custom RS that breaks input on each substring that ends without a blackslash:
awk -v RS='[^\n]*[^\\\\[:space:]][[:blank:]]*(\n|$)' '
{
sub(/\n$/, "", RT)
}
n = split($0, lines, /[[:blank:]]+\\[[:blank:]]*\n/) {
lines[n] = RT
mlen = 0
# determine max length of a block
for (i=1; i<=n; i++)
if (mlen < length(lines[i]))
mlen = length(lines[i])
# print each segment with backslash at the end of max length
for (i=1; i<n; i++)
printf "%-" (mlen+1) "s\\\n", lines[i]
}
RT {
print RT
}' file
abc \
d \
efgh \
i
jklmnop \
q \rst \
uv
wx
y \
z
Code Demo
Details:
-v RS='[^\n]*[^\\\\[:space:]][[:blank:]]*(\n|$): Sets record separator using this regex which basically matches a line that doesn't end with a \. As a result we'll get all contiguous lines that end with \ in each record.
split($0, lines, /[[:blank:]]+\\[[:blank:]]*\n/: Splits each record by ending \ and following line break.
In the first for loop, we loop through each array element and determine longest length of the line i.e. mlen
In the second for loop, we just print each line segment using mlen+1 as length to place trailing \
Finally we print RT which is the substring we capture as a result of RS=...
Working native bash code :
while read line
do
a=${line:112:7}
b=${line:123:7}
if [[ $a != "0000000" || $b != "0000000" ]]
then
echo "$line" >> FILE_OT_YHAV
else
echo "$line" >> FILE_OT_NHAV
fi
done <$FILE_IN
I have the following file (its a dummy), the substrings being checked are both on the 4th field, so nm the exact numbers.
AAAAAAAAAAAAAA XXXXXX BB CCCCCCC 12312312443430000000
BBBBBBB AXXXXXX CC DDDDDDD 10101010000000000000
CCCCCCCCCC C C QWEQWEE DDD AAAAAAA A12312312312312310000
I m trying to write an awk script that compares two specific substrings, if either one is not 000000 it outputs the line into File A, if both of them are 000000 it outputs the line into File B, this is the code i have so far :
# Before first line.
BEGIN {
print "Awk Started"
FILE_OT_YHAV="FILE_OT_YHAV.test"
FILE_OT_NHAV="FILE_OT_NHAV.test"
FS=""
}
# For each line of input.
{
fline=$0
# print "length = #" length($0) "#"
print "length = #" length(fline) "#"
print "##" substr($0,112,7) "##" substr($0,123,7) "##"
if ( (substr($0,112,7) != "0000000") || (substr($0,123,7) != "0000000") )
print $0 > FILE_OT_YHAV;
else
print $0 > FILE_OT_NHAV;
}
# After last line.
END {
print "Awk Ended"
}
The problem is that when i run it, it :
a) Treats every line as having a different length
b) Therefore the substrings are applied to different parts of it (that is why i added the print length stuff before the if, to check on it.
This is a sample output of the line length awk reads and the different substrings :
Awk Started
length = #130#
## ## ##
length = #136#
##0000000##22016 ##
length = #133#
##0000001##16 ##
length = #129#
##0010220## ##
length = #138#
##0000000##1022016##
length = #136#
##0000000##22016 ##
length = #134#
##0000000##016 ##
length = #137#
##0000000##022016 ##
Is there a reason why awk treats lines of the same length as having a different length? Does it have something to do with the spacing of the input file?
Thanks in advance for any help.
After the comments about cleaning the file up with sed, i got this output (and yes now the lines have a different size) :
1 0M-DM-EM-G M-A.M-E. #DEH M-SM-TM-OM-IM-WM-EM-IM-A M-DM-V/M-DM-T/M-TM-AM-P 01022016 $
2 110000080103M-CM-EM-QM-OM-MM-TM-A M-A. 6M-AM-HM-GM-MM-A 1055801001102 0000120000012001001142 19500000120 0100M-D000000000000000000000001022016 $
3 110000106302M-TM-AM-QM-EM-KM-KM-A 5M-AM-AM-HM-GM-MM-A 1043801001101 0000100000010001001361 19500000100M-IM-SM-O0100M-D000000000000000000000001022016 $
4 110000178902M-JM-AM-QM-AM-CM-IM-AM-MM-MM-G M-KM-EM-KM-AM-S 71M-AM-HM-GM-MM-A 1136101001101 0000130000013001006061 19500000130 0100M-D000000000000000000000001022016 $
I am trying to parse a few SQL statements. Here is a sample:
select
ms.member_sk a,
dd.date_sk b,
st.subscription_type,
(SELECT foo FROM zoo) e
from dim_member_subscription_all p,
dim_subs_type
where a in (select moo from t10)
I am interested in getting tables only at this time. So I would like to see
[zoo, dim_member_subscription_all, dim_subs_type] & [t10]
I have put together a small script looking at Paul McGuire's example
#!/usr/bin/env python
import sys
import pprint
from pyparsing import *
pp = pprint.PrettyPrinter(indent=4)
semicolon = Combine(Literal(';') + lineEnd)
comma = Literal(',')
lparen = Literal('(')
rparen = Literal(')')
update_kw, volatile_kw, create_kw, table_kw, as_kw, from_kw, \
where_kw, join_kw, left_kw, right_kw, cross_kw, outer_kw, \
on_kw , insert_kw , into_kw= \
map(lambda x: Keyword(x, caseless=True), \
['UPDATE', 'VOLATILE', 'CREATE', 'TABLE', 'AS', 'FROM',
'WHERE', 'JOIN' , 'LEFT', 'RIGHT' , \
'CROSS', 'OUTER', 'ON', 'INSERT', 'INTO'])
select_kw = Keyword('SELECT', caseless=True) | Keyword('SEL' , caseless=True)
reserved_words = (update_kw | volatile_kw | create_kw | table_kw | as_kw |
select_kw | from_kw | where_kw | join_kw |
left_kw | right_kw | cross_kw | on_kw | insert_kw |
into_kw)
ident = ~reserved_words + Word(alphas, alphanums + '_')
table = Combine(Optional(ident + Literal('.')) + ident)
column = Combine(Optional(ident + Literal('.')) + (ident | Literal('*')))
column_alias = Optional(Optional(as_kw).suppress() + ident)
table_alias = Optional(Optional(as_kw).suppress() + ident).suppress()
select_stmt = Forward()
nested_table = lparen.suppress() + select_stmt + rparen.suppress() + table_alias
table_list = delimitedList((nested_table | table) + table_alias)
column_list = delimitedList((nested_table | column) + column_alias)
txt = """
select
ms.member_sk a,
dd.date_sk b,
st.subscription_type,
(SELECT foo FROM zoo) e
from dim_member_subscription_all p,
dim_subs_type
where a in (select moo from t10)
"""
select_stmt << select_kw.suppress() + column_list + from_kw.suppress() + \
table_list.setResultsName('tables', listAllMatches=True)
print txt
for token in select_stmt.searchString(txt):
pp.pprint(token.asDict())
I am getting the following nested output. Can anybody please help me understand what I am doing wrong?
{ 'tables': ([(['zoo'], {}), (['dim_member_subscription_all', 'dim_subs_type'], {})], {})}
{ 'tables': ([(['t10'], {})], {})}
searchString will return a list of all matching ParseResults - you can see the tables value of each using:
for token in select_stmt.searchString(txt):
print token.tables
Giving:
[['zoo'], ['dim_member_subscription_all', 'dim_subs_type']]
[['t10']]
So searchString found two SELECT statements.
Recent versions of pyparsing support summing this list into a single consolidated using Python builtin sum. Accessing the tables value of this consolidated result looks like this:
print sum(select_stmt.searchString(txt)).tables
[['zoo'], ['dim_member_subscription_all', 'dim_subs_type'], ['t10']]
I think the parser is doing all you want, you just need to figure out how to process the returned results.
For further debugging, you should start using the dump method on ParseResults to see what you are getting, which will print the nested list of returned tokens, and then a hierarchical tree of all named results. For your example:
for token in select_stmt.searchString(txt):
print token.dump()
print
prints:
['ms.member_sk', 'a', 'dd.date_sk', 'b', 'st.subscription_type', 'foo', 'zoo', 'dim_member_subscription_all', 'dim_subs_type']
- tables: [['zoo'], ['dim_member_subscription_all', 'dim_subs_type']]
['moo', 't10']
- tables: [['t10']]
Look at this makefile, it has some sort of primitive progress indication (could have been a progress bar).
Please give me suggestions/comments on it!
# BUILD is initially undefined
ifndef BUILD
# max equals 256 x's
sixteen := x x x x x x x x x x x x x x x x
MAX := $(foreach x,$(sixteen),$(sixteen))
# T estimates how many targets we are building by replacing BUILD with a special string
T := $(shell $(MAKE) -nrRf $(firstword $(MAKEFILE_LIST)) $(MAKECMDGOALS) \
BUILD="COUNTTHIS" | grep -c "COUNTTHIS")
# N is the number of pending targets in base 1, well in fact, base x :-)
N := $(wordlist 1,$T,$(MAX))
# auto-decrementing counter that returns the number of pending targets in base 10
counter = $(words $N)$(eval N := $(wordlist 2,$(words $N),$N))
# BUILD is now defined to show the progress, this also avoids redefining T in loop
BUILD = #echo $(counter) of $(T)
endif
# dummy phony targets
.PHONY: all clean
all: target
#echo done
clean:
#rm -f target *.c
# dummy build rules
target: a.c b.c c.c d.c e.c f.c g.c
#touch $#
$(BUILD)
%.c:
#touch $#
$(BUILD)
All suggestions welcome!
This one is less intrusive and more awesome.
ifneq ($(words $(MAKECMDGOALS)),1)
.DEFAULT_GOAL = all
%:
#$(MAKE) $# --no-print-directory -rRf $(firstword $(MAKEFILE_LIST))
else
ifndef ECHO
T := $(shell $(MAKE) $(MAKECMDGOALS) --no-print-directory \
-nrRf $(firstword $(MAKEFILE_LIST)) \
ECHO="COUNTTHIS" | grep -c "COUNTTHIS")
N := x
C = $(words $N)$(eval N := x $N)
ECHO = echo "`expr " [\`expr $C '*' 100 / $T\`" : '.*\(....\)$$'`%]"
endif
.PHONY: all clean
all: target
#$(ECHO) All done
clean:
#rm -f target *.c
# #$(ECHO) Clean done
target: a.c b.c c.c d.c e.c
#$(ECHO) Linking $#
#sleep 0.1
#touch $#
%.c:
#$(ECHO) Compiling $#
#sleep 0.1
#touch $#
endif
There wasn't really a question so this is less of a standalone answer and more of an extension to Giovanni Funchai's solution. This question is the first google result for "GNU Make Progress" so I ended up here looking for how to do this.
As pointed out by Rob Wells, the solution doesn't work for <10%, but the technique can be extended with the print formatting done by a helper script in whatever language you feel is portable enough for your build. For example, using a python helper script:
echo_progress.py:
"""
Print makefile progress
"""
import argparse
import math
import sys
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--stepno", type=int, required=True)
parser.add_argument("--nsteps", type=int, required=True)
parser.add_argument("remainder", nargs=argparse.REMAINDER)
args = parser.parse_args()
nchars = int(math.log(args.nsteps, 10)) + 1
fmt_str = "[{:Xd}/{:Xd}]({:6.2f}%)".replace("X", str(nchars))
progress = 100 * args.stepno / args.nsteps
sys.stdout.write(fmt_str.format(args.stepno, args.nsteps, progress))
for item in args.remainder:
sys.stdout.write(" ")
sys.stdout.write(item)
sys.stdout.write("\n")
if __name__ == "__main__":
main()
And the modified Makefile:
_mkfile_path := $(abspath $(lastword $(MAKEFILE_LIST)))
I := $(patsubst %/,%,$(dir $(_mkfile_path)))
ifneq ($(words $(MAKECMDGOALS)),1)
.DEFAULT_GOAL = all
%:
#$(MAKE) $# --no-print-directory -rRf $(firstword $(MAKEFILE_LIST))
else
ifndef ECHO
T := $(shell $(MAKE) $(MAKECMDGOALS) --no-print-directory \
-nrRf $(firstword $(MAKEFILE_LIST)) \
ECHO="COUNTTHIS" | grep -c "COUNTTHIS")
N := x
C = $(words $N)$(eval N := x $N)
ECHO = python $(I)/echo_progress.py --stepno=$C --nsteps=$T
endif
.PHONY: all clean
all: target
#$(ECHO) All done
clean:
#rm -f target *.c
# #$(ECHO) Clean done
target: a.c b.c c.c d.c e.c f.c g.c h.c i.c j.c k.c l.c m.c n.c o.c p.c q.c \
r.c s.c t.c u.c v.c w.c x.c y.c z.c
#$(ECHO) Linking $#
#sleep 0.01
#touch $#
%.c:
#$(ECHO) Compiling $#
#sleep 0.01
#touch $#
endif
yields:
$ make
[ 1/28]( 3.57%) Compiling a.c
[ 2/28]( 7.14%) Compiling b.c
[ 3/28]( 10.71%) Compiling c.c
[ 4/28]( 14.29%) Compiling d.c
[ 5/28]( 17.86%) Compiling e.c
[ 6/28]( 21.43%) Compiling f.c
[ 7/28]( 25.00%) Compiling g.c
[ 8/28]( 28.57%) Compiling h.c
[ 9/28]( 32.14%) Compiling i.c
[10/28]( 35.71%) Compiling j.c
[11/28]( 39.29%) Compiling k.c
[12/28]( 42.86%) Compiling l.c
[13/28]( 46.43%) Compiling m.c
[14/28]( 50.00%) Compiling n.c
[15/28]( 53.57%) Compiling o.c
[16/28]( 57.14%) Compiling p.c
[17/28]( 60.71%) Compiling q.c
[18/28]( 64.29%) Compiling r.c
[19/28]( 67.86%) Compiling s.c
[20/28]( 71.43%) Compiling t.c
[21/28]( 75.00%) Compiling u.c
[22/28]( 78.57%) Compiling v.c
[23/28]( 82.14%) Compiling w.c
[24/28]( 85.71%) Compiling x.c
[25/28]( 89.29%) Compiling y.c
[26/28]( 92.86%) Compiling z.c
[27/28]( 96.43%) Linking target
[28/28](100.00%) All done
One could even print a fancy progress bar with unicode characters.
Modified echo_progress.py:
"""
Print makefile progress
"""
import argparse
import math
import sys
def get_progress_bar(numchars, fraction=None, percent=None):
"""
Return a high resolution unicode progress bar
"""
if percent is not None:
fraction = percent / 100.0
if fraction >= 1.0:
return "█" * numchars
blocks = [" ", "▏", "▎", "▍", "▌", "▋", "▊", "▉", "█"]
length_in_chars = fraction * numchars
n_full = int(length_in_chars)
i_partial = int(8 * (length_in_chars - n_full))
n_empty = max(numchars - n_full - 1, 0)
return ("█" * n_full) + blocks[i_partial] + (" " * n_empty)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--stepno", type=int, required=True)
parser.add_argument("--nsteps", type=int, required=True)
parser.add_argument("remainder", nargs=argparse.REMAINDER)
args = parser.parse_args()
nchars = int(math.log(args.nsteps, 10)) + 1
fmt_str = "\r[{:Xd}/{:Xd}]({:6.2f}%) ".replace("X", str(nchars))
progress = 100 * args.stepno / args.nsteps
sys.stdout.write(fmt_str.format(args.stepno, args.nsteps, progress))
sys.stdout.write(get_progress_bar(20, percent=progress))
remainder_str = " ".join(args.remainder)
sys.stdout.write(" {:20s}".format(remainder_str[:20]))
if args.stepno == args.nsteps:
sys.stdout.write("\n")
if __name__ == "__main__":
main()
Which would result in something like this:
$ make clean && make
[12/28]( 42.86%) ███████▊ Compiling k.c
during progress and:
$ make clean && make
[28/28](100.00%) ████████████████████ All done
upon completion.
This is a slight modification to #GiovanniFunchal's excellent answer.
So I wanted to understand this better and make it work for < 10% so I dug into the documentation and learned more about expr.
# PLACE AT THE TOP OF YOUR MAKEFILE
#---------------------------------
# Progress bar defs
#--------------------------------
# words = count the number of words
ifneq ($(words $(MAKECMDGOALS)),1) # if no argument was given to make...
.DEFAULT_GOAL = all # set the default goal to all
# http://www.gnu.org/software/make/manual/make.html
# $# = target name
# %: = last resort recipe
# --no-print-directory = don't print enter/leave messages for each output grouping
# MAKEFILE_LIST = has a list of all the parsed Makefiles that can be found *.mk, Makefile, etc
# -n = dry run, just print the recipes
# -r = no builtin rules, disables implicit rules
# -R = no builtin variables, disables implicit variables
# -f = specify the name of the Makefile
%: # define a last resort default rule
#$(MAKE) $# --no-print-directory -rRf $(firstword $(MAKEFILE_LIST)) # recursive make call,
else
ifndef ECHO
# execute a dry run of make, defining echo beforehand, and count all the instances of "COUNTTHIS"
T := $(shell $(MAKE) $(MAKECMDGOALS) --no-print-directory \
-nrRf $(firstword $(MAKEFILE_LIST)) \
ECHO="COUNTTHIS" | grep -c "COUNTTHIS")
# eval = evaluate the text and read the results as makefile commands
N := x
# Recursively expand C for each instance of ECHO to count more x's
C = $(words $N)$(eval N := x $N)
# Multipy the count of x's by 100, and divide by the count of "COUNTTHIS"
# Followed by a percent sign
# And wrap it all in square brackets
ECHO = echo -ne "\r [`expr $C '*' 100 / $T`%]"
endif
#------------------
# end progress bar
#------------------
# REST OF YOUR MAKEFILE HERE
#----- Progressbar endif at end Makefile
endif
I got rid of the : '.*\(....\)$$' part. It would return the last 4 characters of the inner expr command, but would fail if it was less than 4. And now it works for sub 10%!
And here is the comment free version:
ifneq ($(words $(MAKECMDGOALS)),1) # if no argument was given to make...
.DEFAULT_GOAL = all # set the default goal to all
%: # define a last resort default rule
#$(MAKE) $# --no-print-directory -rRf $(firstword $(MAKEFILE_LIST)) # recursive make call,
else
ifndef ECHO
T := $(shell $(MAKE) $(MAKECMDGOALS) --no-print-directory \
-nrRf $(firstword $(MAKEFILE_LIST)) \
ECHO="COUNTTHIS" | grep -c "COUNTTHIS")
N := x
C = $(words $N)$(eval N := x $N)
ECHO = echo -ne "\r [`expr $C '*' 100 / $T`%]"
endif
# ...
endif
Hope that helps.
Many thanks to #Giovanni Funchal and #phyatt for the question and answers!
I just simplified it even more for my own better understanding.
ifndef ECHO
HIT_TOTAL != ${MAKE} ${MAKECMDGOALS} --dry-run ECHO="HIT_MARK" | grep -c "HIT_MARK"
HIT_COUNT = $(eval HIT_N != expr ${HIT_N} + 1)${HIT_N}
ECHO = echo "[`expr ${HIT_COUNT} '*' 100 / ${HIT_TOTAL}`%]"
endif
!= assigns from shell command
= evaluates variable each time it's used
eval executes its argument without any output
expr allows to make arithmetic calculations
( Not sure though which approach is faster: to call shell with expr or to count 'x'-es with make. )
Usage is the same:
target:
#$(ECHO) $#
Nice trick! (-:
But not really scalable for growing projects that are distributed across many directories with lots of makefiles.
I'd be more inclined to have logging sprinkled through the [Mm]akefiles* in your project and use that to keep track of progress.
Just a thought. BTW Thanks for sharing this.
Edit: Just had a thought. This could be useful in a modified form to display a throbber to show progress while a long task proceeds, e.g unpacking a large distribution tarball instead of just specifying the -v option to the tar command. Still a bit of sugar coating but a bit of fun aswell. (-:
cheers,
Rob