Friday, March 11, 2022

sed command on OSX not supporting '+' or '?' by default

Regex pattern has 3 ways to indicate the occurrence of a character:

"c*"        matching letter 'c'    0 to N    times
"c+"       matching letter 'c'     1 to N    times
"c?"        matching letter 'c'    0 or 1     time

For example, if you want to replace "p...c" in input string,   INSTR="aaabbbcccpppccc"

On Linux, you do        echo $INSTR | sed 's/p\+c/_/'

The same thing on OSX not working, just failed silently.

$ echo $INSTR | sed 's/p\+c/_/'

aaabbbcccpppccc

$ echo $INSTR | sed 's/p+c/_/'

aaabbbcccpppccc


This is because, by default on OSX, sed interprets basic regex.  Instead, we specify using extended regex (-E)

$ echo $INSTR | sed -E 's/p+c/_/'

aaabbbccc_cc

We don't want '*' or the early ccc will be matched.

$ echo $INSTR | sed -E 's/p*c/_/'

aaabbb_ccpppccc

This in fact gives the same result as 's/z*c/_' because 'z' can be zero time

Now, similarly, to indicate 0 or 1 time, use -E as well

$ echo $INSTR | sed -E 's/p?c/_/'

aaabbb_ccpppccc

$ echo $INSTR | sed -E 's/b?c/_/'

aaabb_ccpppccc






Labels: , , ,

Saturday, November 24, 2018

[OSX] command line 'join' failed with DOS line end 0d0a


Ever frustrated how join command did not work on some file?

To get rid of the 0x0d from DOS .CSV file, i.e. make unix 0a ending, do

# note that ^M below is entered by CTRL-V CTRL-M 
$ sed 's/^M//' in.csv > out.csv

Labels: , , , , , ,

Friday, March 30, 2018

sed in-place substitute strings

For simple text search & replace, command line using sed:

# default stdout
$ sed 's/OLD_STR/NEW_STR/g'  file.txt
or
$ cat file.txt | sed 's/OLD_STR/NEW_STR/g'

output could be redirect to another file using pipe
$ !! > output.txt

if care not keeping original file, can substitute in-place using option '-i'
on linux, simply
$ sed -i 's/OLD_STR/NEW_STR/g'  file.txt

on mac osx,add an empty string after '-i'
$ sed -i '' 's/OLD_STR/NEW_STR/g'  file.txt

Labels: , , , , , , , ,