Please contact or Mt Xia for assistance with all your shell programming needs.

  • 6. UNIX TOOLS. Go to the bottom, first,previous,next, last section, table of contents.


    6.5 sed: Edit a File to Standard Output

    UNIX provides a method of editing streams of data. It is the sed utility. The name of this utility is derived from Stream EDitor. This is not the same as the vi editor. The vi editor edits text in a file. The sed utility edits text in a stream. In order to edit a character stream two things are required. First, the line to edit must be identified (regular expressions) and second, how to edit the line. The formal form for the sed utility is as follows:
        Command Format: sed [-n] [-e script] [-f sfile] [files]    
        Details in on-line man pages                               
    The sed utility copies the named files (standard input default) to the standard output, edited according to a set (script) of commands. The -f options cause the script to be taken from file "sfile". The general form is: $sed /address/instruction Note: If no address is specified, all lines are chosen to edit. 'sed' addresses can be line numbers or regular expressions. Example: line numbers 2,4 2,$ ($ represents the last line) textual address /regular-expression/ Note: Forward slashes enclose textual addresses The sed instructions indicate what editing function to perform. Here some useful sed instructions: s substitute d delete Note: Most sed command lines contain spaces or metacharacters and they should be quoted to protect them from the shell. There are many more editing commands provided by sed. Here is a sample sed command to edit the records in the database file that we are already familiar with, namely, phone.lis: Sample session: $sed /s/Smith/Smythe/ phone.lis Smythe, Joan 7-7989 Adams, Fran 2-3876 StClair, Fred 4-6122 Jones, Ted 1-3745 Stair, Rich 5-5972 Benson, Sam 4-5587 $ sed is an editor . It simply copies the standard input to the standard output, editing the lines that match the indicated address. The original file is not changed. Here's another example of a sed command. Sample session: $sed '2,4 s/2$/3/' phone.lis Smith, Joan 7-7989 Adams, Fran 2-3876 StClair, Fred 4-6123 Jones, Ted 1-3745 Stair, Rich 5-5972 Benson, Sam 4-5587 $ What does this sed command do? If you read command in English it reads like this: On lines 2 through 4 substitute the 2 at the end of the line with a 3. Notice that the phone number for StClair,Fred changed from 4-6122 to 4-6123. The number for Stair,Rich didn't change because it was outside the range. The sed utility can also be use to delete parts of a line of data. This is done by substituting nothing for the parts you want to delete. It looks like this: Sample session: $sed 's/^.*, //' phone.lis Joan 7-7989 Fran 2-3876 Fred 4-6122 Ted 1-3745 Rich 5-5972 Sam 4-5587 $ Reading this command it means: Substitute from the beginning of the line followed by any number of characters followed by a comma with the null string (nothing). This has the effect of removing the text. Here's a delete command and how it's used. Sample session: $sed d phone.lis $ Why is there no output? Well, it read standard input and did the editing function on all the selected lines. Since no lines were specified all lines were selected to be edited. The editing was to delete the line. Question: Has the original file been destroyed? Multiple commands are allowed in sed. Each instruction is applied to each input line. Sample session: $sed '/Stair/d >/Adams/d' phone.lis Smith, Joan 7-7989 StClair, Fred 4-6122 Jones, Ted 2-1136 Benson, Sam 4-5587 $ The records for Adams and Stair have both been removed from the database. Note: The > character is the BourneShell secondary prompt.

    6.6 awk: A Pattern Matching Programming Language

    Suppose you wanted to change the format of the database phone.lis to be the first name followed by the last name. There is no easy way to do this with sed. Fortunately, UNIX not only provides a stream editor (sed) but it also has a formatting tool. The formatting tool in UNIX is called awk. This tool is named after authors who wrote it Alfred V. Aho, Peter J. Weinberger, and Brian W. Kerninghan so it really doesn't have any meaning. The awk utility is a pattern scanning and processing language. It will search one or more files for a specified pattern and then performs an action, such as writing to standard output or incrementing a counter when it finds a match. You can use awk to generate reports or filter text. It works equally well with numbers or text. The authors designed it to be easy to use and sacrificed execution speed toward this end. While the sed utility allows us to change the text in a stream, awk allows us to rearrange, add, or delete text in a stream, easily. The awk takes advantage of many constructs from the C programming language. It has the following features: flexible format conditional execution looping statements numeric variables string variables regular expressions C's printf The awk will take it's input from the files you specify on the command line or from standard input. The following is the format for awk:
        Command format: awk [-Fc] [prog] [files]                   
    The awk will scan each line of file for lines that match a set of patterns specified by prog. With each pattern in prog there can be an associated action to be performed when the line is found. The set of patterns may appear literally as prog, or in a file specified as -f file. The prog string should be enclosed in single quotes to protect it from the Shell. Files are read in order and if there are none specified the standard input is read. Each line is matched against the pattern portion of every pattern-action statement. The associated action is performed for each matched pattern. An input line is made up fields separated by white space. $1, $2.. define the fields. $0 refers to the whole line. A pattern-action statement has the form: pattern {action} A missing action means print the line; a missing pattern always makes a match. a statement can be one of the following: if (conditional) statement [else statement] while (conditional) statement for (expression;conditional;expression) statement break continue {[statement]...} variable=expression print [expression-list] [>expression] printf format [,expression-list][>expression] next # skip remaining pattern on this input line exit # skip the rest of the input Statements are terminated by semi-colons, new-lines (CR), or right braces. Let's look at the syntax for awk in a little simpler manner. awk 'commands' [filename] An awk program (commands) consists of a optional pattern to match and an action to perform if a match is found on the current line. This syntax looks like this: awk '/pattern/{action}' [filename] The pattern used is a regular expression enclosed in forward slashes. If no pattern is listed, the action will be performed for every line. An action can contain several commands. There can be multiple patterns and actions. awk '/pattern1/{action1} /pattern2/{action2}' [filename} One of awks commands is print. It puts the current line on standard output. Sample session: $awk '{print}' phone.lis Smith, Joan 7-7989 Adams, Fran 2-3876 StClair, Fred 4-6122 Jones, Ted 1-3745 Stair, Rich 5-5972 Benson, Sam 4-5587 $ The awk splits every input line at whitespace and keeps track of the number of fields on each line and counts the number of lines read. Each field is identified by its field number and a $. $1 Identifies the first field $2 Identifies the second field . $0 Identifies the entire line NF Identifies the number of fields on the line NR Identifies the number of lines that have been read Sample session: $awk '{print NR,$1}' phone.lis 1 Smith, 2 Adams, 3 StClair, 4 Jones, 5 Stair, 6 Benson, $ To change the order of the names in phone.lis use awk. The comma in the print command tells awk to separate each field with a space. Without the comma, the output would have no spacing. Sample session: $awk '{print $2, $1 ""$3}' phone.lis Joan Smith, 7-7989 Fran Adams, 2-3876 Fred StClair, 4-6122 Ted Jones, 1-3745 Rich Stair, 5-5972 Sam Benson, 4-5587 $

    Go to the top, first,previous,next, last section, table of contents.