ICS 54: sed

sed Selected References

http://docs.sun.com/ab2/coll.40.5/REFMAN1/@Ab2PageView/idmatch(SED-1)

Sun's Solaris 7 man page

http://www.opengroup.org/onlinepubs/7908799/xcu/sed.html

The Open Group's Single UNIX Specification, Version 2.

http://www.cornerstonemag.com/sed/sedfaq.html

sed FAQ (Frequently Asked Questions)

http://seders.icheme.org/

The Seder's Grab-bag.

http://www.cornerstonemag.com/sed/

"sed... the stream editor" -- It does a pretty good job of describing both itself and sed as follows:

A simple page to describe sed, the stream editor. Originally written and designed for Unix, sed has been ported over to CP/M, MS-DOS, Windows 9x/NT, OS/2, and other operating systems.

I first encountered sed in my CP/M days, when Kaypros ruled the computer world. Basically, sed will change hunks of text on the fly, without making you open up a screen, push a cursor around, and press DELETE or INSERT or ENTER or function keys.

How it works: You feed sed a script of prearranged editing commands (like, "change every line that begins with a colon to such-and-such") and sed sends your revised text to the screen. To save the revisions on disk, use the DOS redirection arrow, >newfile.txt. Sample syntax:

         sed "one-or-two-sed-commands" input.file >newfile.txt

         sed -f bigger_sed.script input.file >newfile.txt

This page is an attempt to collect my FAQ (Frequently Asked Questions) files and my favorite sed utilities and introductions for novices. The text files are for any sed user; the binaries are mainly for DOS/Windows users.

...

Eric Pement, sed FAQ maintainer

sed

sed [ -n ] script [ file ... ]
sed [ -n ] [-e script] ... [-f scriptfile] ... [ file ... ]

sed reads one or more text files (if none is present, it uses standard input), makes editing changes according to a script of editing commands, and writes the results to standard output. The script is obtained from either the script operand string or a combination of the option-arguments from the -e script and -f scriptfile options.

-n

Normally, each line read is edited and then written to standard output (unless it has been deleted as part of the editing). With this option, only lines explicitly selected for output will be written.

-e script

Add the editing commands specified by the script option-argument to the end of the script of editing commands.

-f scriptfile

Add the editing commands in the file scriptfile to the end of the script.

Multiple -e and -f options may be specified.

All commands are added to the script in the order specified, regardless of their origin.

sed script commands

Examples

% cat in
1A A
2A B
3A C
4A D
5A E
1B F
2B G
3B H
4B I
5B J
%
% sed '2,6d' in
1A A
2B G
3B H
4B I
5B J
%
% sed '7,$s/^/=/' < in
1A A
2A B
3A C
4A D
5A E
1B F
=2B G
=3B H
=4B I
=5B J
%
% sed -n '5,8s/[A-E]/Z/gp' in
5Z Z
1Z F
2Z G
3Z H
%
% sed '/^2/,/^4/d' in
1A A
5A E
1B F
5B J
%
% sed -e 's/A/Z/' -e 's/Z.*E/"&"/' in
1Z A
2Z B
3Z C
4Z D
5"Z E"
1B F
2B G
3B H
4B I
5B J
%
% sed -n 's/A/Z/2p' in
1A Z
%
% head -3 in | sed 's/\([^ ]*\) \(.*\)/\2 \1/'
A 1A
B 2A
C 3A
%
% head -3 in | sed -f flip.sed
A 1A
B 2A
C 3A
% cat flip.sed
# Put first word at the end of the line
s/\([^ ]*\) \(.*\)/\2 \1/
%
% ph alias=pazzani | sed -f ph2html.sed
<p><br>
<b>Michael J. PAZZANI</b><br>
Phone: (949) 824-7405, 7403<br>
Information & Computer Science<br>
mailcode: 3425<br>
title: Professor & Chair<br>
Fax: (949) 824-3976<br>
office_address: 444E CS<br>
<img src="http://www.ics.uci.edu/~pazzani/pazzani.gif"><br>
Home Page: <a href="http://www.ics.uci.edu/~pazzani/">http://www.ics.uci.edu/~pazzani/</a><br>
E-mail: pazzani@uci.edu<br>
<p><br>
%
% ph alias=pazzani
----------------------------------------
            name: Michael J. PAZZANI
           phone: (949) 824-7405, 7403
      department: Information & Computer Science
        mailcode: 3425
           title: Professor & Chair
      fax_number: (949) 824-3976
  office_address: 444E CS
     picture_url: http://www.ics.uci.edu/~pazzani/pazzani.gif
   home_page_url: http://www.ics.uci.edu/~pazzani/
   email_address: pazzani@uci.edu
----------------------------------------
%
% cat ph2html.sed
/Not present in entry/d
s/^501:.*/No matches found./
s/^[ ]*//
s/^picture_url: \(http:\/\/.*\)/<img src="\1">/
s/^name: \(.*\)/<b>\1<\/b>/
s/^department: //
s/^phone: /Phone: /
s/^fax_number: /Fax: /
s/^email_address: /E-mail: /
s/^home_page_url: \(http:\/\/.*\)/Home Page: <a href="\1">\1<\/a>/
s/^[-]*$/<p>/
s/$/<br>/
%

Explanations

sed '2,6d' in

delete lines 2 through 6 from the file in

sed '7,$s/^/=/' < in

Put "=" at the start each line of in beginning with line 7

sed -n '5,8s/[A-E]/Z/gp' in

For lines 5-8 of in, print out only those lines containing a letter from "A" to "E" and do so only after changing all those letters to "Z"

sed '/^2/,/^4/d' in

Remove all blocks of lines which begin with a line starting with "2" and continuing until ended with a line starting with "4"

sed -e 's/A/Z/' -e 's/Z.*E/"&"/' in

In every line, replace the first "A" by a "Z" and then put quotes around any pattern beginning with "Z" and ending with "E"

sed -n 's/A/Z/2p' in

Replace the 2nd "A" in any line by "Z" and print only the lines that have been changed.

head -3 in | sed 's/\([^ ]*\) \(.*\)/\2 \1/'

Process standard input, moving first word of each line line to its end.

head -3 in | sed -f flip.sed

Process standard input, moving first word of each line line to its end.

ph alias=pazzani | sed -f ph2html.sed

Convert ph campus electronic directory output into HTML

sed -e 's/&/\&amp;/g' -e 's/</\&lt;/g' -e 's/>/\&gt;/g'

How to make HTML source look like source in an HTML document.
It actually looks better in a file:
# HTML convert & < > to &amp; &lt; &gt;
s/&/\&amp;/g
s/</\&lt;/g
s/>/\&gt;/g

sed scripts

sed addresses

• empty,
• a "$," indicating the last line of all input,
• a decimal number which counts input lines cumulatively across files,
• a regular expression, preceded and followed by a delimiter, usually a slash,
  Note: This is formally called a "context address."

A command line with one address applies to each line which matches the address. (If that address is a number, rather than a "context address," then the command applies to exactly one line, the line with that number."

A command line with two addresses applies to the (inclusive) range from the first line that matches the first address to the next line that matches the second. Having found a match for the first address, once it finds a following match for the second, sed will start looking for another match for the first address as soon as it has finished with a line which matches the second.

Common sed commands

s/pattern/replacement/[arguments]

Substitute replacement for pattern where arguments can be:
n = replace the nth instance of pattern,
g = replace all instances of pattern,
p = print this line if a replacement is made.

d

Delete the line

y/str1/str2

Transform. str1 and str2 are strings with the same number of characters and no character can appear twice in str1. Each character in str1 is transformed into the corresponding character in str2.
Example: "y/aeiou/AEIOU/g"   capitalizes all vowels.

Regular Expressions and Special Characters in sed

Examples

% cat Numb
123456789
% sed 's/[1-9]\{3,5\}/+/g' Numb
++
% sed 's/[1-9]\{5,7\}/+/g' Numb
+89
% sed 's/[3-8]\{3,5\}/+/g' Numb
12+89
% sed 's/[3-8]\{3,\}/+/g' Numb
12+9
% sed 's/[3-8]\{3\}/+/g' Numb
12++9
% sed 's/[45][3-8]/+/g' Numb
123+6789
% sed 's/\([45]\)[3-8]/-\1-&-/g' Numb
123-4-45-6789