Is there a way to replace all the commas except those in quotes using sed - awk

Question

# Director, Movie Title, Year, Comment
Ethan Coen, No Country for Old Men, 2007, none
Ethan Coen, "O Brother, Where Art Thou?", 2000, none
Like in here the the commas outside the quotes have to be replaced with |.
Ethan Coen| "O Brother, Where Art Thou?"| 2000| none
i did try this command
sed -e 's/,(?=(?:[^"]*"[^"]*")*[^"]*$)/|/g',
to first match those strings, put i am getting RE error: repetition-operator operand invalid
i don't know how to convert the re, totally new to shell and regex, i would be grateful for any help

Answer 1

You may try this gnu awk command with FPAT to split fields using a custom regex:
awk -v OFS='|' -v FPAT=' *"[^"]*"|[^",]+' '{$1=$1} 1' file
# Director| Movie Title| Year| Comment
Ethan Coen| No Country for Old Men| 2007| none
Ethan Coen| "O Brother, Where Art Thou?"| 2000| none

Answer 2

The proper way to deal with CSV is to use a proper csv parser.
Given:
$ cat file
# Director, Movie Title, Year, Comment
Ethan Coen, No Country for Old Men, 2007, none
Ethan Coen, "O Brother, Where Art Thou?", 2000, none
First issue, your example has a two character delimiter of ', ' vs a single comma ','. That will throw off most csv parsers unless they support multi character delimiters. For example, csvkit does not support multi character delimiters. (The place where it fails is on quoted fields since if the csv parser is looking for ," as the start of a quoted field, it fails with , "...)
The lightest weight but full featured csv parser commonly available at the command line is in ruby.
With ruby, you can do:
$ ruby -rcsv -ne 'puts (CSV.parse $_, col_sep:", ").join("|")' file
# Director|Movie Title|Year|Comment
Ethan Coen|No Country for Old Men|2007|none
Ethan Coen|O Brother, Where Art Thou?|2000|none
If you want the replacement to also be '| ' vs the single character delimiter of '|' you can do:
$ ruby -rcsv -ne 'puts (CSV.parse $_, col_sep:", ").join("| ")' file
# Director| Movie Title| Year| Comment
Ethan Coen| No Country for Old Men| 2007| none
Ethan Coen| O Brother, Where Art Thou?| 2000| none
Note that O Brother, Where Art Thou? is no longer quoted since the ', ' is no longer a delimiter.
To even be more proper you would use the csv module to re-encode back into a proper RFC 4180 compliant file.
Suppose you wanted to fix the ', ' into a compliant ',' and maintain the quoted fields. Our single line of Ruby does not do that.
Instead:
$ ruby -rcsv -ne 'out=(CSV.parse $_, col_sep:", ").map do |row|
row.to_csv(:col_sep=>",")
end
puts out' file
# Director,Movie Title,Year,Comment
Ethan Coen,No Country for Old Men,2007,none
Ethan Coen,"O Brother, Where Art Thou?",2000,none
Or into a '|':
$ ruby -rcsv -ne 'out=(CSV.parse $_, col_sep:", ").map do |row|
row.to_csv(:col_sep=>"|")
end
puts out' file
# Director|Movie Title|Year|Comment
Ethan Coen|No Country for Old Men|2007|none
Ethan Coen|O Brother, Where Art Thou?|2000|none

Answer 3

This might work for you (GNU sed):
sed -E ':a;s/^([^"]*("[^",]*"[^"]*)*"[^",]*),/\1\n/;ta;y/,\n/|,/' file
Replace all ,'s between "'s with \n's, then translate all ,'s for |s and all \n's for ,'s.

Answer 4

You may find perl better suited to handling complex regular expressions like this, when you need to avoid matching a pattern in specific context(s):
perl -i -pe 's/"[^"]*"(*SKIP)(*F)|,/|/g' file.txt
Or, if you need to match across lines,
perl -0777 -i -pe 's/"[^"]*"(*SKIP)(*F)|,/|/g' file.txt
See the regex demo and the Bash demo.
Here,
-0777 slurps the file contents into a single string (or, it will be treated as a list of lines, each of which will be fed to the regex engine separately)
-i - makes changes to the input file "inline"
"[^"]*"(*SKIP)(*F)|, - the regex that matches ", then any zero or more chars other than " and then a " and then skips the match and goes on searching for a match from the position where it failed, or (|) matches a , in any other context
| is the replacement
g - replaces all occurrences.
If your file is UTF8, and you need to make some manipulation with Unicode text, you would need to add -CSD -Mutf8, say, before -i.