I'm trying to run a simple awk command using Cygwin.
I want to run through a file and put each line to a file named the same as the first field, appending lines with the same value in the first field to their relevant file.
I am running this command:
gawk -F , '{print>\$1}' ./filename.csv
The only output I get from command is: "Access is denied."
C:\projects>gawk -F , '{print>\$1}' ./filename.csv
Access is denied.
This is what it looks like in total.
I've looked at the file permissions and allowed everything but I can't seem to get it to work.
Edited with extra info as requested:
C:\projects>gawk -F , '{print > \$1}' filename.csv
Access is denied.
C:\projects>$SHELL --version
'$SHELL' is not recognized as an internal or external command,
operable program or batch file.
C:\projects>gawk --version
GNU Awk 4.1.4, API: 1.1 (GNU MPFR 3.1.5, GNU MP 6.1.2)
Copyright (C) 1989, 1991-2016 Free Software Foundation.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see http://www.gnu.org/licenses/.
I've tried running from the Cygwin Shell and I get this error now:
/cygdrive/c/projects/FileSplit
$ gawk -F , '{print > $1}' ./filename.csv
gawk: cmd. line:1: (FILENAME=./filename.csv FNR=1) fatal: expression for `>' redirection has null string value
Running it in the Cygwin Shell worked although my first field was blank on some lines so the command I ended up with was:
gawk -F , '{if(length($1)==0)print > "empty"; else print > $1}' ./filename.csv
Get rid of the backslash:
gawk -F , '{print > $1}' ./filename.csv
Given what you've now added to your question - you're running gawk from the Windows CMD prompt, not from cygwin. Don't do that, run gawk from a cygwin terminal window instead by executing something like All Programs -> Cygwin -> Cygwin64 Terminal to start a cygwin window running a bash shell.
Related
I have encountered a problem. When I use awk to match the beginning and ending of words, it can be successful on centos, but it fails on ubuntu.
enter image description here
ip a | grep "\<inet\>"
got the answer
ip a | awk "/\<inet\>/{print $0}"
No output
Execute
file /etc/alternatives/awk
I see that my awk points to mawk.
mawk-its code base is based on the fast AWK implementation of the bytecode interpreter.
According to the previous suggestions, I installed gawk, This version is the standard implementation for Linux
apt install gawk
Execute again
file /etc/alternatives/awk
At this time the file has pointed to gawk,
Pointing to awk related commands again, I got the result I want
This started as an obscure problem with RPM scriptlets occasionally failing on awk. I narrowed it down to the following: The scriptlets use a GNU extension: length(array) construct, not supported when running in the posix mode. OK so far. What I don't understand is how running awk under sudo changes the posix compliance behavior. Here is a simple awk script that should run in the GNU mode, and should fail in posix mode.
$ cat ./try
/bin/awk 'BEGIN{x[1]=foo;x[2]=bar;print length(x);}'
$ /bin/awk --version | grep Awk
GNU Awk 4.0.2
$ id
uid=0(root) gid=0(root) groups=0(root)
$ /bin/sh ./try
awk: cmd. line:1: fatal: length: received array argument
$ sudo /bin/sh ./try
2
$
What is the underlying mechanism that changes the awk behavior?
Awk (really gawk under linux) is being controlled by the POSIXLY_CORRECT environment variable, which was occasionally being inherited from the original user's environment. The installation in question must be run by root, but at times the admin would become root with "su" which keeps the environment, thus keeping his POSIXLY_CORRECT, forcing gawk into a posix mode, and failing the GNU length(array) extension. At other times the admin would run "sudo" or "su -" to become root, start with root's clean environment and successfully run the extended gawk functionality.
I have created a tcsh script with a series of gawk commands in the following form:
gawk -f InputFileName > OutputFileName
After the standard call-in (#!/bin/csh -f), I utilized the following command:
set a = $<
In the InputFileName, I proceed to use ${a}.txt, but it does not even use $a when looking up the input file.
They were initially running fine on Cygwin (Windows); now they are running on a Linux and are presenting problems.
It was a text file compatibility issue.
Since I did not have dos2unix on my computer, I used the following one line of code.
awk '{ sub("\r$", ""); print }' winfile.txt > unixfile.txt
I am new to awk script. I am trying to figure out how to run an awk file without awk -f command. I see people keep saying add "#!bin/awk -f" for the first line of an awk file. But this didn't for my awk. It still gives me "no file or directory" error.
I question is what does "#!bin/awk -f" really mean, and what does it do?
Its #!/bin/awk -f not #!bin/awk. That will probably work, but theres no guaranty. If someone who has awk installed in a different location runs your script, it won't work. What you want is this: #!/usr/bin/env awk -f.
#! is what tells bash what to use to interpret your script. It should go at the very top of your file. It's called a Shebang. Right after that, you put the path to the interpreter.
/usr/bin/env finds where awk is located, and uses that script as the interpreter. So if they installed awk into somewhere else like /usr/local/bin then it'll find it. This probably won't matter for you, but it's a good habit to get into. It's more portable, and can be shared easier.
The -f says that awk is gonna read from a file. You could do awk -f yourfilename.awk in bash, but in the shebang, -f means the rest of the code will be the file it reads from.
I hope this helped. Feel free to ask me any questions if it doesn't work, or isn't clear enough.
UPDATE
If you get the error message:
/usr/bin/env: ‘awk -f’: No such file or directory
/usr/bin/env: use -[v]S to pass options in shebang lines
then change the first line of your script to #!/usr/bin/env -S awk -f (tested with GNU bash, version 4.4.23)
You probably want
#!/bin/awk -f
(The first slash after the #! is important).
This tells unix what program it should use to 'run' the script with.
It is usually called the 'shebang' which comes from hash + bang.
If you want to run your script like this you need to make sure it is executable (chmod +x <script>).
Otherwise you can just run your script by typing the command /bin/awk -f <script>
The Shebang for Awk Explained
#! is the start of a shebang line, which tells the shell which interpreter to use for the script.
/bin/awk is the path to your awk executable. You may need to change this is your awk is installed elsewhere, or if you want to use a different version of awk.
-f is a flag to awk to tell it to interpret the flag's argument as an awk script. In a shebang, it tells some awks to interpret the remainder of the script instead of a file.
Your Shebang is (Probably) Broken
You are using #!bin/awk -f which is unlikely to work, unless you have awk installed as $PWD/bin/awk. You probably meant to use #!/bin/awk instead.
In some instances, passing a flag on the shebang line may not work with your shell or your awk. If you have the rest of the shebang line correct, you might try removing the -f flag and see if that works for you.
GNU make gives me a strange error message, which I do not understand.
gao#L8470-130213 ~
$ make
echo Test
C:\Program: C:\Program: is a directory
make: *** [test] Error 126
This is what I thought of verifying:
gao#L8470-130213 ~
$ less makefile
test:
echo Test
gao#L8470-130213 ~
$ which make
/c/Programx86/GnuWin32/bin/make
gao#L8470-130213 ~
$ /c/Progra~2/GnuWin32/bin/make.exe test
echo Test
C:\Program: C:\Program: is a directory
make: *** [test] Error 126
gao#L8470-130213 ~
$ make --version
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
This program built for i386-pc-mingw32
It feels like some other program is trying to run at the end, and that its path includes some spaces. In that case, what program could it be, and how can I prevent it from running?
I have seen this thread and tried to disable my antivirus, which did not help.
I have also looked into permissions, but I am not sure if makefile needs execution rights. I can't seem to be able to change that anyway (running in bash on windows. makefile is not read-only when I check in explorer):
gao#L8470-130213 ~
$ ls -l makefile
-rw-r--r-- 1 gao Administ 21 Apr 15 14:53 makefile
gao#L8470-130213 ~
$ chmod +x makefile
gao#L8470-130213 ~
$ ls -l makefile
-rw-r--r-- 1 gao Administ 21 Apr 15 14:53 makefile
What is going on with make, what can I do?
It's not "some other program" that's trying to run, it's the echo command. Make prints the command to be run, echo test, but you never see the output (test) so that means it failed trying to find the echo program. Unfortunately I'm not very familiar with the vagaries of running GNU make on Windows: there are lots of different options. One possibility would be to get a newer version of GNU make; 3.81 is very old. 3.82 is now available and might work better for you.
Good info you added above about your environment re: using bash; that wasn't clear from the original question and on Windows there are many different ways to do things. You're using the mingw version of make; that version (as I understand it) does NOT use bash as the shell to run commands in: it's supposed to be used with native Windows environments which do not, certainly, have bash available. I believe that the version of make you have is invoking commands directly, and/or using command.com. Certainly not a UNIX shell like bash.
If you want to use bash you should set the SHELL make variable to the path of your bash.exe program. If you're using a Cygwin environment you can use the GNU make that comes with Cygwin which behaves more like a traditional make + shell.
Otherwise you'll need to write your commands using Windows command.com statements.
Again, I don't use Windows so this is mostly hearsay.
PS. The makefile does not need to be executable.
What is going on is that make doesn't like file names or directory names with spaces in them, such as Program Files. Neither do most of the utilities that makefiles typically rely on, such as the shell to execute commands with.
I create a junction from Program Files to ProgramFiles and use the latter whenever I encounter cases like this.