KL010 – Working With Text

Given the text file below, write a grep -E commands to extract the lines containing an email address. write another grep -E command to extract the lines containing IP addresses:

┌──(kali㉿kali-1)-[~]
└─$ cat list
a.b.c.d
10.1.0.12
127.0.0.1
Hello@world
256.0.1.2
172.20.65.1
test@example.com
me@example.co.uk
not@valid@email.com

Here is my answer; you might come up with another answers to:

┌──(kali㉿kali-1)-[~]
└─$ grep -E "^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,6}$" list
test@example.com
me@example.co.uk

┌──(kali㉿kali-1)-[~]
└─$ grep -E "^(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$" list
10.1.0.12
127.0.0.1
172.20.65.1

sed

sed is a very powerful stream editor. It is very complex tool and I cover the surface here.

Let’s create a stream of text then pipe it to sed to replace a string on it.

s/regexp/replacement/ attempts to match regexp against the pattern space. If successful, replace that portion matched with replacement.

┌──(kali㉿kali-1)-[~]
└─$ echo "Hi World." | sed 's/Hi/Hello/'
Hello World.

Let’s try another example which has multiple string of Hellos. note that g applies the replacement to all matches to the regexp, not just the first.

┌──(kali㉿kali-1)-[~]
└─$ echo Hello World! Who said Hello? I said hello. | sed 's/Hello/Hi/'
Hi world! Who said Hello? I said hello.

┌──(kali㉿kali-1)-[~]
└─$ echo Hello World! Who said Hello? I said hello. | sed 's/Hello/Hi/g'
Hi World! Who said Hi? I said hello.

The comment lines start with sharp sign # in Linux and most programming languages. The following command will remove commented lines on a text file when displaying on stdout. d deletes pattern space.

┌──(kali㉿kali-1)-[~]
└─$ sed '/^#/d' /etc/adduser.conf

DSHELL=/bin/bash

DHOME=/home

GROUPHOMES=no

LETTERHOMES=no

SKEL=/etc/skel

FIRST_SYSTEM_UID=100
LAST_SYSTEM_UID=999

FIRST_SYSTEM_GID=100
LAST_SYSTEM_GID=999

FIRST_UID=1000
LAST_UID=59999

FIRST_GID=1000
LAST_GID=59999

USERGROUPS=yes

USERS_GID=100

DIR_MODE=0755

SETGID_HOME=no

QUOTAUSER=""

SKEL_IGNORE_REGEX="dpkg-(old|new|dist|save)"

Here is another example which complements the last example. I am going to delete the empty lines as well:

┌──(kali㉿kali-1)-[~]
└─$ sed '/^\s*#/d;/^$/d' /etc/adduser.conf
DSHELL=/bin/bash
DHOME=/home
GROUPHOMES=no
LETTERHOMES=no
SKEL=/etc/skel
FIRST_SYSTEM_UID=100
LAST_SYSTEM_UID=999
FIRST_SYSTEM_GID=100
LAST_SYSTEM_GID=999
FIRST_UID=1000
LAST_UID=59999
FIRST_GID=1000
LAST_GID=59999
USERGROUPS=yes
USERS_GID=100
DIR_MODE=0755
SETGID_HOME=no
QUOTAUSER=""
SKEL_IGNORE_REGEX="dpkg-(old|new|dist|save)"

cut

The cut command is very simple and handy. It is used to separate the fields. We need to give it delimiter with switch -d.

┌──(kali㉿kali-1)-[~]
└─$ cut -d ":" -f 1 /etc/passwd
root
daemon
bin
sys
sync
games
man
lp
mail
news

┌──(kali㉿kali-1)-[~]
└─$ echo "Names: Mustermenn, Uve, Klara" | cut -d "," -f 3
 Klara

awk

AWK is a programming language designed for text processing. Like sed and grep, it is a filter and is a standard feature of most Unix-like operating systems. Most commonly used switch is -F (field separator) and the print command which prints the result.

┌──(kali㉿kali-1)-[~]
└─$ echo "Names: Mustermenn, Uve, Klara" | awk -F "," '{print $1, $3}'
Names: Mustermenn  Klara

vi

vi is a text editor (analogous to Windows text editor Notepad) which is installed on every POSIX-compliant system. It is very powerful although It is difficult to get used to it. To edit a file pass the name of the file as an argument to vi:

┌──(kali㉿kali-1)-[~]
└─$ vi hello_world.txt

Once the file is opened, enable insert-text mode (hit i) to begin typing. To disable insert-text mode and go back to command mode, press the Esc key. In the end you typing :q! quits the vi without saving; hitting :wq! quits with saving the file. Here is good documentation should you need learn on vi.

Today, in one minute, is my last day in my current company I am working for.