COMP7004 - Systems Scripting
Lecture 08: AWK Scripting
Dr. Vincent Emeakaroha
vincent.emeakaroha@mtu.ie
Semester 2, 2026
AWK Highlights
• A programming language for handling common data manipulation
tasks with only a few lines of code
• awk is a pattern-action language
• The language looks a little like C but automatically handles input, field
splitting, initialization, and memory management
• Built-in string and number data types
• No variable type declarations
• awk is a great prototyping language
• Start with a few lines and keep adding until it does what you want
• awk gets its input from
• files
• redirection and pipes
• directly from standard input
Operation Style and Usages
• awk operation:
• scans a file line by line
• splits each input line into fields
• compares input line/fields to pattern
• performs action(s) on matched lines
• Useful for:
• transforming data files
• producing formatted reports
• Programming constructs:
• formatting output lines
• arithmetic and string operations
• conditionals and loops
Structure of an AWK Script
• An awk script consists of:
• An optional BEGIN segment
• For preparing execution prior to reading input
• pattern - action pairs
• Processing the input data
• For each pattern matched, the corresponding action is taken
• An optional END segment
• Processing after end of input data
BEGIN {action}
pattern {action}
pattern {action}
.
.
.
pattern { action}
END {action}
Local Variable
• awk scripts can define and use variables
BEGIN { sum = 0 }
{ sum ++ }
END { print sum }
• Some variables are predefined
• Examples are the special variables
• NR Number of Records
• FS Field Separator
• Etc.
Patterns and Action
• Search a set of files for patterns
• Perform specified actions upon lines or fields that contain
instances of patterns
• Does not alter input files
• Process one input line at a time
• pattern can be:
• the special token BEGIN or END
• regular expression (enclosed with //)
• relational or string match expression
• Arithmetic operations
Record and Fields
AWK Special Variables
FS Field separator (default=whitespace)
RS Record separator (default=\n)
NF Number of fields in current record
NR Number of the current record
OFS Output field separator (default=space)
ORS Output record separator (default=\n)
FILENAME Current filename
• Similar to Bash environment variables.
Basic AWK Command Types
1. Direct command:
• awk [options] ’command‘ file(s)
2. Script file:
• awk [options] –f scriptfile file(s)
• -f indicates usage of script file containing script commands
• Special characters
• $0 is the entire line – a record
• $1 is the first field, $2 is the second field, ….
Example: Records and Fields
cat emps.txt
Tom Jones 4424 5/12/66 543354
Mary Adams 5346 11/4/63 28765
Sally Chang 1654 7/22/54 650000
Billy Black 1683 9/23/44 336500
awk '{print NR, $0}' emps.txt
1 Tom Jones 4424 5/12/66 543354
2 Mary Adams 5346 11/4/63 28765
3 Sally Chang 1654 7/22/54 650000
4 Billy Black 1683 9/23/44 336500
Example: Space as Field Separator
cat emps.txt
Tom Jones 4424 5/12/66 543354
Mary Adams 5346 11/4/63 28765
Sally Chang 1654 7/22/54 650000
Billy Black 1683 9/23/44 336500
awk '{print NR, $1, $2, $5}' emps.txt
1 Tom Jones 543354
2 Mary Adams 28765
3 Sally Chang 650000
4 Billy Black 336500
Example: Colon as Field Separator
cat em2.txt
Tom Jones:4424:5/12/66:543354
Mary Adams:5346:11/4/63:28765
Sally Chang:1654:7/22/54:650000
Billy Black:1683:9/23/44:336500
awk -F: '/Jones/{print $1, $2}' em2.txt
Tom Jones 4424
Arithmetic Operators
Operator Meaning Example
+ Add x + y
- Subtract x – y
* Multiply x * y
/ Divide x / y
% Modulus x % y
^ Exponential x ^ y
Example:
awk '$3 * $4 > 500 {print $0}' file
Relational Operators
Operator Meaning Example
< Less than x < y
< = Less than or equal x < = y
== Equal to x == y
!= Not equal to x != y
> Greater than x > y
> = Greater than or equal to x > = y
~ Matched by reg exp x ~ /y/
!~ Not matched by reg exp x !~ /y/
Logical Operators
Operator Meaning Example
&& Logical AND a && b
|| Logical OR a || b
! NOT ! a
Examples:
awk '($2 > 5) && ($2 <= 15) {print $0}' file
awk '$3 == 100 || $4 > 50 {print $0}' file
Invoking Script File
• File: grades.txt
John 85 92 78 94 88
Andrea 89 90 75 90 86
Jasper 84 88 80 92 84
• awk script: average.awk
# average five grades
{ total = $2 + $3 + $4 + $5 + $6
avg = total / 5
print $1, avg }
• Run as:
awk –f average.awk grades.txt
Output
Output Commands
• print
• print easy and simple output
• printf
• print formatted (similar to C printf)
• Sprintf
• format string (similar to C sprintf)
Printf Format Specifier
Given: x = ‘A’, y = 15, z = 2.3, and b = Bob Smith
Printf Format
Specifier
What it Does
%c printf("The character is %c \n", x)
output: The character is A
%d printf("The boy is %d years old \n", y)
output: The boy is 15 years old
%s printf("My name is %s \n", b)
output: My name is Bob Smith
%f printf("z is %5.3f \n", z)
output: z is 2.300
Control Statements
• Conditional
• if-else
• Repetition
• for
• with counter
• with array index
• while
• do-while
• also: break, continue
“if” Statement
• Awk supports two basic types of if statement
• Simple action
• Multiple action
• Simple action if statement syntax
if (conditional-expression)
action
• Multiple action if statement syntax
if (conditional-expression)
{
action1;
action2;
}
• Note the use of semicolon for the multiple action statements
• Examples
• if.awk
if Example 1: Missing Scores
cat student-scores
Bill 2143 78 84 77
Jane 2321 56 58 45
James 2122 38 37
John 2537 87 97 95
Keith 2415 30 47
if.awk
#!/bin/awk
{
if ($3 =="" || $4 == "" || $5 == "")
print "Some score for the student",$1,"is
missing";
}
• Invoking script
• awk –f if.awk student-scores
• Output
• Some score for the student James is missing
• Some score for the student Keith is missing
22
if Example 2: Pass/Fail Report
• if2.awk
#!/bin/awk
{
if ($3 >=35 && $4 >= 35 && $5 >= 35)
print $0,"=>","Pass";
else
print $0,"=>","Fail";
}
• Invoking script
• awk –f if2.awk student-scores
• Output:
• Bill 2143 78 84 77 => Pass
• Jane 2321 56 58 45 => Pass
• James 2122 38 37 => Fail
• John 2537 87 97 95 => Pass
• Keith 2415 30 47 => Fail
If Example 3: Average Grade
if3.awk
#!/bin/awk
{
total=$3+$4+$5;
avg=total/3;
if ( avg >= 90 ) grade="A";
else if ( avg >= 80) grade ="B";
else if (avg >= 70) grade ="C";
else grade="D";
print $0,"=>",grade;
}
• Output:
• Bill 2143 78 84 77 => C
• Jane 2321 56 58 45 => D
• James 2122 38 37 => D
• John 2537 87 97 95 => A
• Keith 2415 30 47 => D
for Loop
Syntax:
for (initialization; limit-test;
increment/decrement)
statement
• It first performs initialisation
• Then checks the limit-test
• If condition is true, it executes “statement”
• Finally it increments/decrements
for.awk
#!/bin/awk
{
for (a = 0; a < 4; a++)
print a;
}
• Invoking the script
• awk –f for.awk
• (Note: script will behave as hanging since there is no input to the
invocation. Simply press enter to see output.)
• Ouput
• 0 1 2 3
while Loop
Syntax:
while (logical-expression)
statement
• It first evaluates the “logical-expression”
• If true, then executes the ”statement”
while.awk
#!/bin/awk
{
b = 1;
while(b < 5)
{
print b;
b++;
}
}
• Output
• 1 2 3 4
Piping Command
• Command outputs can be piped into awk for
processing
• The symbol for piping is |
• Example:
ls -l | awk '{print}'
total 48
-rw-r--r-- 1 bill staff 64 28 Aug 12:02 for.awk
-rw-r--r-- 1 bill staff 109 28 Aug 10:54 if.awk
-rw-r--r-- 1 bill staff 108 28 Aug 11:03 if2.awk
-rw-r--r-- 1 bill staff 175 28 Aug 11:12 if3.awk
-rw-r--r-- 1 bill staff 95 28 Aug 10:55 student-scores
-rw-r--r-- 1 bill staff 64 28 Aug 12:20 while.awk
Thank You!
