I am looking for a way to remove the first line of the rollup output file. I know I can do it with a script and say sed, but I am looking for something a little cleaner.
I tried plugins like replace & strip but these do not seem to let me "replace" a line of code.
Does anyone how to do this? Is there a plugin?
Update
Here is my current sed solution...
"build": "rollup -c && npm run fix:rollup",
"fix:rollup": "sed '1d' ./dist/index.js > ./tmp.js; mv ./tmp.js ./dist/index.js"
Use writeBundle hook, sketch:
import fs from 'fs';//Node.js builtin
...
plugins:[
{//inline plugin
writeBundle(bundle){
for (const [fileName, chunkOrAsset] of Object.entries(bundle)) {
//test for file you want to modify
const data = fs.readFileSync(fileName, {encoding:'utf8'});
data.replace(/^[^\n]*\n/,'');
fs.writeFileSync(...)
}
}
]
Related
I have a Mathematica notebook.
I would like to generate in an automatically way this notebook to a Wolfram script that I could make run directly from terminal under MacOS with : $ wolframscript output_wls.wls
The problem is that, if I export directly from Mathematica to .wls, I have all the lines of the code which are commented.
For example, I get from "Mathematica Notebook exporting" the following beginning into Wolfram script:
#!/usr/bin/env wolframscript
(* ::Package:: *)
(* ::Input:: *)
(*SetDirectory[NotebookDirectory[]];*)
(*(*Needs["ErrorBarPlots`"];*)
(*Needs["ComputationalGeometry`"];*)
(*Off[CompiledFunction::cflist];*)*)
(* ::Subchapter:: *)
(*Expansion rate date loading*)
(* ::Input:: *)
(*dataH=Import["H_All.txt","Table"];*)
(*dataH = DeleteCases[dataH, x_?(Length[#]==0&), 1];*)
(*ndata=Length[dataH];*)
(*zLine = dataH[[All, 1]];*)
(* ::Subchapter:: *)
(*BD solver with redshift*)
(* ::Input:: *)
(*RK4Method[dH_,d\[Phi]_,d\[Rho]dm_,du_,\[CapitalOmega]dm_,\[CapitalOmega]k_,H0_,\[Phi]0_,d\[Phi]0_,\[Omega]BD_,zLine_]:=Module[*)
(*{h, Htable, \[Rho]dmtable, \[Phi]table, utable, Hk1, Hk2, Hk3,Hk4, \[Rho]dmk1, \[Rho]dmk2, \[Rho]dmk3,\[Rho]dmk4, \[Phi]k1, \[Phi]k2,\[Phi]k3,\[Phi]k4, uk1,uk2, uk3,uk4, containsIndeterminate, containsComplex, Hval},*)
...
If I execute this .wls script, nothing happens : that's normal since there are comments everywhere.
So, I would like to fix this by automatically remove the comments for real original commands and keep the others as notebooks comments.
I know that a magic sed or awk script or command lines could do the trick but I have not enough background to create a such script. However, I think there may be other alternatives.
How can I make automatic this task from any notebook to convert to a working executable Wolfram script?
Update
I tried the solution of Nathan's method but it fails when I evaluate the notebook. Here below the message:
Please check if the following command can help
gawk -f convert.awk example.txt
where example.txt contains your example text and convert.awk the following code.
BEGIN {FS = ""}
{
for (i = 1; i <= NF; i++) {
if ((($i == "(") && ($(i+1) == "*")) || (($i == "*") && ($(i+1) == ")"))) {
i = i + 1
} else {
printf("%c", $i)
}
}
printf("\n")
}
I am trying to put up a Jenkinsfile where one of the step is to read a text file line by line and assign it to a variable. But the input to the While loop is not working.
Code Snippet:
dir(FilePath) {
sh("""
while read -r line; do
args+="--arg $line"
done < env
""")
I would use Jenkins basic step. Then you can pass it to shell or do whatever you need.
https://jenkins.io/doc/pipeline/steps/workflow-basic-steps/#readfile-read-file-from-workspace
dir(FilePath) {
script {
def file = readFile file:"file.txt"
sh("do whatever ${file}")
}
}
I'm trying to read a huge gz file line by line in Perl6.
I'm trying to do something like this
my $file = 'huge_file.gz';
for $file.IO.lines -> $line {
say $line;
}
But this give error that I have a malformed UTF-8. I can't see how to get this to read gzipped material from the help page https://docs.perl6.org/language/unicode#UTF8-C8 or https://docs.perl6.org/language/io
I want to accomplish the same thing as was done in Perl5: http://blog-en.openalfa.com/how-to-read-and-write-compressed-files-in-perl
How can I read a gz file line by line in Perl6?
thanks
I would recommend using the module Compress::Zlib for this purpose. You can find the readme and code on github and install it with zef install Compress::Zlib.
This example is taken from the test file number 3 titled "wrap":
use Test;
use Compress::Zlib;
gzspurt("t/compressed.gz", "this\nis\na\ntest");
my $wrap = zwrap(open("t/compressed.gz"), :gzip);
is $wrap.get, "this\n", 'first line roundtrips';
is $wrap.get, "is\n", 'second line roundtrips';
is $wrap.get, "a\n", 'third line roundtrips';
is $wrap.get, "test", 'fourth line roundtrips';
This is probably the easiest way to get what you want.
use the read-file-content method in the Archive::Libarchive module, but i don't know if the method read all lines into memory at once:
use Archive::Libarchive;
use Archive::Libarchive::Constants;
my $a = Archive::Libarchive.new: operation => LibarchiveRead, file => 'test.tar.gz';
my Archive::Libarchive::Entry $e .= new;
my $log = '';
while $a.next-header($e) {
$log = get-log($a,$e) if $e.pathname.ends-with('.txt');
}
sub get-log($a, $e) {
return $a.read-file-content($e).decode('UTF8-C8');
}
If you are after a quick solution you can read the lines from the stdout pipe of a gzip process:
my $proc = run :out, "gzip", "--to-stdout", "--decompress", "MyFile.gz"
for $proc.out.lines -> $line {
say $line;
}
$proc.out.close;
I am trying to convert a “sed” script I use on my FreeBSD machine to one using “Powershell” on Windows 10.
This is the sed script. It is used to strip a header from an email plus the immediately following line and send the output to “email_1.txt”. The file is fed to the script on the command line; i.e. “COMMAND file”
sed '/Received: by 2002:a17:90a:3566:0:0:0:0/,/^/d' <$1> email_1.txt
I cannot find a way to get this to work with “PowerShell”.
Thanks!
You have a couple of options.
Install sed -
Something like scoop might be helpful here.
Write a pure powershell solution.
This will be very similar to what you would write if you were to try to do the same thing in "pure" bash. Here is an attempt to do so:
--
function Delete-TargetLines {
[cmdletbinding()]
param(
[String]$needle,
[int]$count = [int]1,
[parameter(ValueFromPipeline)]
[string[]]$haystack
)
Begin {
[int]$seen = 0
}
Process {
if ($seen -gt 0) {
$seen -= 1
} elseif ( $haystack -match $needle ) {
$seen = 1
} else {
$haystack
}
}
}
And an example of running it:
> #("Pre-line", "This is a test", "second line", "post line") | Delete-TargetLines -needle "test"
Pre-line
post line
> Get-Content $myfile | Delete-TargetLines -needle 'value' > $outfile
ok so i have have this
{"status":0,"id":"7aceb216d02ecdca7ceffadcadea8950-1","hypotheses":[{"utterance":"hello how are you","confidence":0.96311796}]}
and at the moment i'm using this shell command to decode it to get the string i need,
echo $x | grep -Po '"utterance":.*?[^\\]"' | sed -e s/://g -e s/utterance//g -e 's/"//g'
but this only works when you have a grep compiled with perl and plus the script i use to get that JSON string is written in perl, so is there any way i can do this same decoding in a simple perl script or a simpler unix command, or better yet, c or objective-c?
the script i'm using to get the json is here, http://pastebin.com/jBGzJbMk and if you want a file to use then download http://trevorrudolph.com/a.flac
How about:
perl -MJSON -nE 'say decode_json($_)->{hypotheses}[0]{utterance}'
in script form:
use JSON;
while (<>) {
print decode_json($_)->{hypotheses}[0]{utterance}, "\n"
}
Well, I'm not sure if I can deduce what you are after correctly, but this is a way to decode that JSON string in perl.
Of course, you'll need to know the data structure in order to get the data you need. The line that prints the "utterance" string is commented out in the code below.
use strict;
use warnings;
use Data::Dumper;
use JSON;
my $json = decode_json
q#{"status":0,"id":"7aceb216d02ecdca7ceffadcadea8950-1","hypotheses":[{"utterance":"hello how are you","confidence":0.96311796}]}#;
#print $json->{'hypotheses'}[0]{'utterance'};
print Dumper $json;
Output:
$VAR1 = {
'status' => 0,
'hypotheses' => [
{
'utterance' => 'hello how are you',
'confidence' => '0.96311796'
}
],
'id' => '7aceb216d02ecdca7ceffadcadea8950-1'
};
Quick hack:
while (<>) {
say for /"utterance":"?(.*?)(?<!\\)"/;
}
Or as a one-liner:
perl -lnwe 'print for /"utterance":"(.+?)(?<!\\)"/g' inputfile.txt
The one-liner is troublesome if you happen to be using Windows, since " is interpreted by the shell.
Quick hack#2:
This will hopefully go through any hash structure and find keys.
my $json = decode_json $str;
say find_key($json, 'utterance');
sub find_key {
my ($ref, $find) = #_;
if (ref $ref) {
if (ref $ref eq 'HASH' and defined $ref->{$find}) {
return $ref->{$find};
} else {
for (values $ref) {
my $found = find_key($_, $find);
if (defined $found) {
return $found;
}
}
}
}
return;
}
Based on the naming, it's possible to have multiple hypotheses. The prints the utterance of each hypothesis:
echo '{"status":0,"id":"7aceb216d02ecdca7ceffadcadea8950-1","hypotheses":[{"utterance":"hello how are you","confidence":0.96311796}]}' | \
perl -MJSON::XS -n000E'
say $_->{utterance}
for #{ JSON::XS->new->decode($_)->{hypotheses} }'
Or as a script:
use feature qw( say );
use JSON::XS;
my $json = '{"status":0,"id":"7aceb216d02ecdca7ceffadcadea8950-1","hypotheses":[{"utterance":"hello how are you","confidence":0.96311796}]}';
say $_->{utterance}
for #{ JSON::XS->new->decode($json)->{hypotheses} };
If you don't want to use any modules from CPAN and try a regex instead there are multiple variants you can try:
# JSON is on a single line:
$json = '{"other":"stuff","hypo":[{"utterance":"hi, this is \"bob\"","moo":0}]}';
# RegEx with negative look behind:
# Match everything up to a double quote without a Backslash in front of it
print "$1\n" if ($json =~ m/"utterance":"(.*?)(?<!\\)"/)
This regex works if there is only one utterance. It doesn't matter what else is in the string around it, since it only searches for the double quoted string following the utterance key.
For a more robust version you could add whitespace where necessary/possible and make the . in the RegEx match newlines: m/"utterance"\s*:\s*"(.*?)(?<!\\)"/s
If you have multiple entries for the utterance confidence hash/object, changing case and weird formatting of the JSON string try this:
# weird JSON:
$json = <<'EOJSON';
{
"status":0,
"id":"an ID",
"hypotheses":[
{
"UtTeraNcE":"hello my name is \"Bob\".",
"confidence":0.0
},
{
'utterance' : 'how are you?',
"confidence":0.1
},
{
"utterance"
: "
thought
so!
",
"confidence" : 0.9
}
]
}
EOJSON
# RegEx with alternatives:
print "$1\n" while ( $json =~ m/["']utterance["']\s*:\s*["'](([^\\"']|\\.)*)["']/gis);
The main part of this RegEx is "(([^\\"]|\\.)*)". Description in detail as extended regex:
/
["'] # opening quotes
( # start capturing parentheses for $1
( # start of grouping alternatives
[^\\"'] # anything that's not a backslash or a quote
| # or
\\. # a backslash followed by anything
) # end of grouping
* # in any quantity
) # end capturing parentheses
["'] # closing quotes
/xgs
If you have many data sets and speed is a concern you can add the o modifier to the regex and use character classes instead of the i modifier. You can suppress the capturing of the alternatives to $2 with clustering parenthesis (?:pattern). Then you get this final result:
m/["'][uU][tT][tT][eE][rR][aA][nN][cC][eE]["']\s*:\s*["']((?:[^\\"']|\\.)*)["']/gos
Yes, sometimes perl looks like a big explosion in a bracket factory ;-)
Just stubmled upon another nice method of doing this, i finaly found how to acsess the Mac OS X JavaScript engine form commandline, heres the script,
alias jsc='/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/Resources/jsc'
x='{"status":0,"id":"7aceb216d02ecdca7ceffadcadea8950-1","hypotheses":[{"utterance":"hello how are you","confidence":0.96311796}]}'
jsc -e "print(${x}['hypotheses'][0]['utterance'])"
Ugh, yes i came up with another answer, im strudying python and it reads arrays in both its python format and the same format as a json so, i jsut made this one liner when your variable is x
python -c "print ${x}['hypotheses'][0]['utterance']"
figured it out for unix but would love to see your perl and c, objective-c answers...
echo $X | sed -e 's/.*utterance//' -e 's/confidence.*//' -e s/://g -e 's/"//g' -e 's/,//g'
:D
shorter copy of the same sed:
echo $X | sed -e 's/.*utterance//;s/confidence.*//;s/://g;s/"//g;s/,//g'