5 min to read

sed and awk Commands with Examples

Learn how to use sed and awk commands effectively for text processing

Featured image


Overview

Let’s explore the sed and awk commands in Unix/Linux environments, introducing how these tools play crucial roles in text processing, data manipulation, and task automation.

What are sed and awk?

Sed (Stream Editor)

A stream editor that parses and transforms text in data streams (e.g., file or pipeline input).

It operates on text line by line, processing input from standard input or files.

Awk

A programming language designed for data processing and report generation.

It can process input from standard input, multiple files, or output from other processes.

It’s particularly useful in shell scripts and small database management tasks.


Basic Usage of sed

1️⃣ Text Editing

# Create example.txt
cat <<EOF > example.txt
Hello World
Hello Somaz
Hello promi
EOF

# Replace 'Hello' with 'Hi' and display
sed 's/Hello/Hi/' example.txt
Hi World
Hi Somaz
Hi promi

# Original file remains unchanged
cat example.txt
Hello World
Hello Somaz
Hello promi

# Use -i option to edit the file in place
sed -i 's/Hello/Hi/' example.txt

# File is now changed
cat example.txt
Hi World
Hi Somaz
Hi promi

2️⃣ Advanced Text Editing

# Create multiple files containing 'somaz'
cat <<EOF > example1.txt
Hello somaz
EOF

cat <<EOF > example2.txt
Hi somaz
EOF

cat <<EOF > example3.txt
Hu somaz
EOF

# Replace 'somaz' with 'somazx' in all files
find . -type f -exec sed -i 's/somaz/somazx/g' {} +

# All files are changed
cat example1.txt
Hello somazx

cat example2.txt
Hi somazx

cat example3.txt
Hu somazx
Find Command Options

-type f → Instructs find to search for files only (not directories).

-exec → Allows execution of commands on files found by find command.

sed -i → In-place option that modifies files directly.

s/somaz/somazx/g → sed command to replace 'somaz' with 'somazx'. The 'g' flag makes it global (all occurrences in each line).

{} → Placeholder for file paths in find command. When using exec with find, {} is replaced with the path of each found file.

+ → Terminates exec command. More efficient as it allows multiple file paths to be passed to sed at once, rather than executing sed individually for each file.


🛠️ Basic Usage of awk

1️⃣ Text Pattern Scanning

# Create data.txt
cat <<EOF > data.txt
1,John,5000
2,Jane,5500
3,Doe,6000
EOF

# Print second and third columns
awk -F, '{print $2, $3}' data.txt
John 5000
Jane 5500
Doe 6000

2️⃣ Advanced Pattern Processing

# Using /etc/passwd file
# Print columns 1, 6, and 7
awk -F: '{print $1, $6, $7}' /etc/passwd
root /root /bin/bash
daemon /usr/sbin /usr/sbin/nologin
bin /bin /usr/sbin/nologin

# Print specific user info
awk -F: '/somaz/ {print $1, $6, $7}' /etc/passwd
somaz /home/somaz /bin/bash

# Format output with labels
cat /etc/passwd | awk -F: '/somaz/ {print "Username: "$1" \n" "Home Directory: "$6" \n" "Shell: "$7 }'
Username: somaz
Home Directory: /home/somaz
Shell: /bin/bash


🚄 Combined Usage of sed and awk

# Create example.txt
cat <<EOF > example.txt
1, John Doe, 30
2, Jane Smith, 25
3, Emily Davis, 40
EOF

# Combine sed and awk to process data
sed 's/, */,/g' example.txt | awk -F, '$3 > 30 {print $2}'
Emily Davis
🔍 Understanding sed and awk Commands

sed 's/, */,/g' example.txt
→ Replaces comma-space patterns with just commas
→ 's' indicates substitution
→ 'g' flag means global replacement (applies to all occurrences in each line)

awk -F, '$3 > 30 {print $2}'
→ Processes the output from sed
→ '-F' sets comma as the field separator
→ '$3 > 30 {print $2}' instructs awk to print the second field when the third field (age) is greater than 30


Common Options

sed Options:

awk Options:


Reference