DIRECTORY STRUCTURE IN UNIX

 

Directory Contains
bin/ Required Boot-time binaries
boot/ Boot configuration files for the OS loader and kernel image
dev/ Device files
etc/ System configuration files and scripts
home/ User/Sub branch directories
lib/ Main OS shared libraries and kernel modules
Lost+found/ Storage directory for “recovered” files
mnt/ Temporary point to connect devices to
proc/ Pseudo directory structure containing information about the kernel, currently running processes and resource allocation
root/ Linux (non-standard) home directory for the root user. Alternate location being the / directory itself
sbin/ System administration binaries and tools
tmp/ Location of temporary files
usr/ Difficult to define – it contains almost everything else including local binaries, libraries, applications and packages (including X Windows)
var/ Variable data, usually machine specific. Includes spool directories for mail and news

 

GENERAL UNIX COMMANDS

 

TO SEE WHERE COMMANDS ARE LOCATED

echo $PATH

/bin:/usr/bin:/usr/local/bin:$HOME/bin

which ls

SEEING HELP COMMAND

date –help

info date

man date

whatis date

Extended Documents:=/usr/share/doc

TO CHANGE CONSOLE

Ctrl+alt+(F1-F7)

 

CHANGING DIRECTORY

cd /home/deb/work

cd ..[One level up]

LISTING DIRECTORIES AND FILES

ls /

ls –al [All hidden files]

ls –l [long listing]

ls –R[recurses through sub-directories]

ls -d[Listing directories only]

ls -ltr|grep ^d[Listing directories only]

List of command options

Option Action

 

-a list hidden files

-d list the name of the current directory

-F show directories with a trailing ’/’ executable

files with a trailing ’*’

-g show group ownership of file in long listing

-i print the inode number of each file

-l long listing giving details about files and direc-tories

-R list all subdirectories encountered

-t sort by time modified instead of name

COPYING FILES

cp source destination

cp –P source destination [Preserves permissions]

cp –R source destination [Recursively copies the files]

cp ~helper/tmp/for_john tmp/comments[To copy a file from another user’s directory using a relative

pathname.]

cp /usr/lib/more.help .[To copy a file from another directory to the current working

directory, preserving the file name:]

MOVING AND RENAMING

mv file1 file2 [ Renaming]

mv source destination [ Move]

REMOVING FILES AND DIRECTORIES

rm chap[1-2][wildcard is used]

rm -i chap1[Removing interactively]

rm -rf[Deleting forefully file or directories]

rmdir deb[removing empty directory]

TO CHECK FILE TYPES

file deb.sh

Displaying the first few lines of a file

head /usr/local/doc/vi.txt[To display the first 10 lines of a file]

head -5 *.xdh[To display the first 5 lines of several files]

 

Displaying the last part of a file

 

tail fred[To display the last 10 lines of a text file]

tail -50c help.txt[To display the last 50 characters in a file]

tail -c -512 foo[Copies last 512 bytes from foo]

tail -c +512 foo[Copies everything after skippiong 512 bytes]

 

HANDLING PRINTERS

 

lp deb.lis[Printing file]

lp -d laser deb.lis[If there are more printers]

lp -n3 deb.lis[Printing 3 copies]

lpstat[To view print queue]

cancel laser[To cancel printing of current printer]

cancel pr1-320[Cancel job with request id pr1-320]

lpq[To see print que job number]

lprm 31[Removes job number 31]

lpr -p laser deb.lis[Prints on printer laser]

lpstat -c[To see pending jobs]

 

COUNTING LINES

 

wc deb.txt

wc -c[No. of characters]

wc -l[No. of lines]

 

COMPRESSING FILES

 

compress deb.txt

uncompress deb.txt

gzip deb

gunzip deb.z

 

USING TAR COMMAND

 

# display contents of a file
tar tvf myfile.tar

# display contents of a diskette (Solaris)
volcheck
tar tvf /vol/dev/rdiskette0/unnamed_floppy

# copy files to a tar file
tar cvf myfile.tar *.sql

# format floppy, and copy files to it (Solaris)
fdformat -U -b floppy99
tar cvf /vol/dev/rdiskette0/floppy99 *.sql

# append files to a tar file
tar rvfn myfile.tar *.txt

# extract files from a tar filem to current dir
tar xvf myfile.tar

 

SORTING

 

sort /etc/passwd
sort in order based on the whole line

sort -r /etc/passwd
reverse the order of the sort

sort +2n -t: /etc/passwd
sort on third field, where field delimiter is : (skip the first two fields)

sort +2n -t: -n /etc/passwd
same sort but treat the field as numbers not ASCII characters

sort –o telnos telnos(sorts telnos and replaces its content with the sorted output)

 

DUPLICATE ROW BY UNIQUE COMMAND

 

uniq names
remove duplicate lines from names and display them on the screen

uniq names uniq.names
remove duplicates lines from names and put them into uniq.names

uniq -d names
display all duplicate lines

 

Using symbolic links

 

ln -s source linkname

ln -s reports/reportA publications/my_report

 

TO DETERMINE DISK USAGE

 

# display disk free, in KB
df -kt

# display disk usage, in KB for directory
du -k mydir

# display directory disk usage, sort by largest first
du -ahx .|sort -rh|head -20

 

 

FIND Command

 Find files by text search

$ find ~ -name “*.txt” -print

To find all ‘*.txt’ files in the current directory you are in and any sub-directories:

$ find . -name “*.txt” -print

To find all filenames with at least one upper case character, in your current directory

and any sub-directories:

$ find . -name “[A-Z]*” -print

To find files in /etc directory that begin with host, and any other characters after

that:

$ find /etc -name “host*” -print

To find all files in your $HOME directory:

$ find ~ -name “*” -print or find . -print

 

To bring the system to zero per cent response time, start at the root level and drill

through all directories listing ‘all’ files. If you want to remain on good terms with

your sys admin be very careful with this option!

$ find / -name “*” -print

Find all files that begin with two lower case characters, followed by two numbers,

followed by .txt. The following find command could return a file called ax37.txt.

$ find . -name “[a-z][a-z][0–9][0–9].txt” -print

 

Find files by perm mode

 

To find files with a permission of 755 which is read, write and executable by

owner and read and execute by everyone else,

$ find . -perm 755 -print

 

To find files that are read, write and executable for everyone (watch out for these) use

this find command. Put a dash in front of the octal number. The perm stands for

permissions and 007 is based on the notation you use for the chmod (absolute) mode.

$ find . -perm -007 -print

 

Ignoring directories

 

To display the files in the apps directory when you do not want find to search

in /apps/bin:

$ find /apps -name “/apps/bin” -prune -o -print

 

Find files by user and nouser

 

To find files by owner, you supply the actual login name of that user. To find files

owned by user dave in your $HOME directory:

$ find ~ -user dave -print

 

To find files owned by uucp in /etc:

$ find /etc -user uucp -print

 

To find files of a user that has been deleted, use the -nouser option. This

finds files that have no valid userid in the /etc/passwd file. You do not have to

supply a user name with this option; find will go off and do all the work. To find

all files not owned by a current user in the /home directory tree:

$ find /home -nouser -print

 

 

Find files by group and nogroup

 

Like the user and nouser option, find has the same options for groups. To find all

files with the group membership of ‘accts’ that are in the /apps directory:

$ find /apps -group accts -print

To find all files that have no valid group membership just supply the -nogroup

option. Here find starts at the root of the file system.

$ find / -nogroup -print

 

Find files by modification times

 

To find all files that have been modified in the last five days:

$ find / -mtime -5 -print

To find files in /var/adm directory that have not been modified in the last three

days:

$ find /var/adm -mtime +3 -print

 

Finding files that are newer or older in days or minutes

 

$ find . -newer age.awk ! -newer belts.awk -exec ls -l {} \;

  

Delete old trace files from DIAG location of Oracle Database

find /u01/app/oracle -type f -name “*.trc” -mtime +1 -exec rm {} \;

find /u01/app/oracle -type f -name “*.trm” -mtime +0.5 -exec rm {}  \;

find /u01/app/oracle -type f -name “*.aud” -mtime +0.5 -exec rm {} \;

  VI EDITOR

List of insert commands

 

To do this … Command

Insert text after the cursor                             a

Insert text before the cursor                           i

Append text at the end of  the current line     A

Insert text at the start of the current line        I

Open a new line above the  current line        o

Open a new line below the current line         O

 

Recovering files after a system crash

 

vi -r help.xdh

 

Viewing a file

view filename

 

Moving the cursor along a line

Press the ESC key to enter command mode before using these com-mands.

To move to Do this …

 

next character                        l

previous character                 h

next word                              w

next n words                         wn

previous word                       b

previous n words                  bn

end of current word              e

start of current line                0 (zero)

end of current line                  $

Moving the cursor between lines

Press the ESC key to enter command mode before using these com-mands.

To move to Do this …

 

next line down                        j

(same column)

start of next line down           +

previous line                           k

(same column)

start of previous line

Deleting characters

Press the ESC key to enter command mode before using these com-mands.

To delete Do this

 

current character                   x

previous character                dh

Deleting words and lines

Press the ESC key to enter command mode before using these com-mands.

To delete Do this

current word                         dw

previous word                       db

entire line                               dd

to end of line                         d$

to start of line                        d0 (zero)

next n lines                           ndd

 

To get back a word or line that you have just deleted enter the command:

p

Searching for text

Press the ESC key to enter command mode before using these com-mands.

To search Do this …

forward for a pattern /pattern

backward for a pattern ?pattern

repeat previous search n

repeat previous search in N

reverse direction

Examples of replacing text

To replace one word in the current line with another:

:s/that/which

 

To replace every instance of one word in the current line with

another:

:s/destroy/delete/g

This replaces every occurrence of the word “destroy” with the

word “delete” in the current line.

 

To replace every occurrence of a word throughout the file, with

another:

:g/insert/s//add/g

This replaces every occurrence of “insert” with “add”.

To replace ^M from text file

:%s/^v^M/ /g

 

UNIX Scheduler and Process

CRONTAB

 1st column Minutes 1–59

2nd column Hour 1–23 (0 is midnight)

3rd column Day_of_month 1–31

4th column Month 1–12

5th column Weekday 0–6 (0 is Sunday)

6th column Command to run

 

Here are some examples of crontab entries:

 

30 21 * * * /apps/bin/cleanup.sh

 

runs a script called cleanup.sh in /apps/bin at 9:30 every night.

Cron and crontab 33

45 4 1,10,22 * * /apps/bin/backup.sh

runs a script called backup.sh in /apps/bin at 4:45 a.m., on the 1st, 10th and 22nd

of each month.

10 1 * * 6,0 /bin/find -name “core” -exec rm {} \;

runs a find command at 1:10 a.m. only on Saturday and Sunday.

0,30 18-23 * * * /apps/bin/dbcheck.sh

runs a script called dbcheck.sh in /apps/bin at every 30 minutes past the hour,

between 18:00 and 23:00.

0 23 * * 6 /apps/bin/qtrend.sh

runs a script called qtrend.sh in /apps/bin at 11:00 p.m. every Saturday.

 

The general format of the crontab is:

Crontab [-u user] -e -l -r

where:

-u is the user login name

-e edits the crontab file

-l lists the crontab file

-r deletes the crontab file

 

Submitting a command to the background

 

$ find /etc -name “srm.conf” -print >find.dt 2>&1 &

 

Checking the process using ps

 

$ ps x|grep 28305

28305 p1 S 0:00 sh /root/ps1

28350 p1 S 0:00 grep 28305

 

Killing a background job

 

kill -signal [process_number]

 

$ kill 28305

$ kill -9 28305

 

Submitting a job using nohup

 

nohup command >myout.file 2>&1

 

$ nohup ps1 &

 

 CONTROLLING ACCESS TO FILE SYSTEM

 

1.To display the permissions on a single file:

ls -l file1

-rw-r–r– 2 unixjohn 3287 Apr 8 12:10 file1

2.This displays the following information about the file file1.

 

-rw-r–r– – access permissions

2 – number of links to this file

unixjohn – owner

3287 – size in bytes

Apr 8 12:10 – date and time last modified

3.There are three types of permissions:

r – read the file or list files in the directory

w – write to the file or directory

x – execute the file or search the directory

4.Each of these permissions can be set for any one of three types of

user:

u – the user who owns the file (usually you)

g – members of the group the owner belongs to

o – all other users

The access permissions for all three types of user can be given as a

string of nine characters:

user      group      others

r w x         r w x            r w x

5.Changing permission

  1. To give yourself permission to execute a file that you own:

chmod u+x file1

This gives you execute permission for the file file1.

  1. To give members of your group permission to read a file:

chmod g+r file2

This gives the group permission to read the file file2.

  1. To give read permission to everyone for a particular type of file:

chmod a+r *.pub

This gives everyone permission to read all files with the extension

.pub.

  1. To give the group write and execute permission:

chmod g+wx $HOME/SCCS

This gives all members of the group permission to place files in

5.Setting access permissions numerically

There is a shorthand way of setting permissions by using octal num-bers.

Read permission is given the value 4, write permission the value

2 and execute permission 1.

r w x

4 2 1

These values are added together for any one user category:

1 = execute only

2 = write only

3 = write and execute (1+2)

4 = read only

5 = read and execute (4+1)

6 = read and write (4+2)

7 = read and write and execute (4+2+1)

So any access permission can be expressed as three digits. For exam-ple:

user group others

chmod 640 file1 rw- r– —

chmod 754 file1 rwx r-x r–

chmod 664 file1 rw- rw- r–

6.Creating user and groups

useradd -u 21- -g dba -c “THE RDBMS” -d /home/oracle -s /bin/ksh -m oracle

groupadd -g 241 dba

Informations are stored in /etc/passwd annd /etc/shadow

7.Setting limits on file size

ulimit 234567

8.Modifying and removing users

usermod -s /bin/bash oracle

userdel oracle

9.Changing password

passwd

# change group to staff for this file
chgrp staff myfile

# change owner to jsmith for this file
chown jsmith myfile

 

Shell input and output

 

ECHO

 

$ echo “What is your name :\c”

$ read name

 

You can put variables within echo statements as well as escape codes. In this

example, the terminal bell is rung, your $HOME directory is displayed, and the

command tty is evaluated by the shell.

$ echo “\007your home directory is $HOME, you are connected on ‘tty‘”

your home directory is /home/dave, you are connected on /dev/ttyp1

 

If it’s LINUX then . . .

You have to put a ‘-n’ after the echo to suppress the new line:

$ echo -n “What is your name :”

You have to put a ‘-e’ after echo for the escape code to work:

$ echo -e “\007your home directory is $HOME, you are connected on

‘tty‘”

your home directory is /home/dave, you are connected on /dev/ttyp1

 

READ

 

read variable1 variable2 . . .

 

$ read name

Hello I am superman

$ echo $name

Hello I am superman

 

$ read name surname

John Doe

$ echo $name $surname

John Doe

 

$ pg var_test

#!/bin/sh

# var_test

echo “First Name :\c”

read 51

read name

echo “Middle Name :\c”

read middle

echo “Last name :\c”

read surname

 

If it’s LINUX then . . .

Remember to use the ‘-n’ echo option.

$ pg var_test

#!/bin/sh

# var_test

echo “First Name :\c”

read name

echo “Middle Name :\c”

read middle

echo “Last name :\c”

read surname

 

PIPES

 

$ who | awk ‘{print $1″\t”$2}’

matthew pts/0

louise pts/1

 

$ df -k | awk ‘{print $1}’| grep -v “Filesystem”

 

$ df -k | awk ‘{print $1}’| grep -v “Filesystem”|sed s’/\/dev\///g’

hda5

hda8

hda6

hdb5

hdb1

hda7

hda1

 

TEE

 

The tee command acts the way it is pronounced. It takes the input and sends

one copy to the standard output and another copy to a file. If you want to see

your output and save it to a file at the same time, then this is the command for

you.

The general format is:

tee -a files

 

$ who | tee who.out

louise pts/1 May 20 12:58 (193.132.90.9)

matthew pts/0 May 20 10:18 (193.132.90.1)

cat who.out

louise pts/1 May 20 12:58 (193.132.90.9)

matthew pts/0 May 20 10:18 (193.132.90.1)

 

$ find etc usr/local home -depth -print | cpio -ovC65536 -O \

 

$ echo ” myscript is now running, check out any errors…in

myscript.log” | tee -a myscript.log

$ myscript | tee -a myscript.log

 

$ echo “stand-by disk cleanup starting in 1 minute” | tee/dev/console

$ sort myfile | tee -a accounts.log

$ myscript | tee -a accounts.log

 

 

Standard input, output and errors

 

Redirecting standard output

 

$ cat passwd | awk -F: ‘{print $1}’| sort 1>sort.out

$ ls -l | grep ^d >>files.out

$ ls account* >> files.out

 

Redirecting standard input

sort < names.txt

sort <names.txt > names.out

 

$ cat >> myfile <<MAYDAY

> Hello there I am using a $TERM terminal

> and my user name is $LOGNAME

> bye…

> MAYDAY

$ pg myfile

Hello there I am using a vt100 terminal

and my user name is dave

 

Redirecting standard error

 

To redirect standard errors you specify the file descriptor ‘2’. Let’s first look at an

example, since they always explain better. Here grep searches for a pattern called

‘trident’ on a file called missiles.

$ grep “trident” missiles

grep: missiles: No such file or directory

 

 

grep reports no such file, and sends its errors to the terminal which is the default.

Let’s now send all errors to a file, in fact to the system dustbin, ‘/dev/null’.

$ grep “trident” missiles 2>/dev/null

 

Combining standard output and error

 

$ cat account_qtr.doc account_end.doc 1> accounts.out 2> accounts.err

 

Merging standard output and standard error

 

When merging output and errors, the shell evaluates the command from left to

right, which is really all you need to know when putting mergers together. Here’s an

example.

$ cleanup >cleanup.out 2>&1

In the above example the script cleanup directs all output (>) to a file called

cleanup.out, and all errors (2), are directed to (>) the same place as the output

(&1), which is cleanup.out.

$ grep “standard” * > grep.out 2>&1

In the above example all output from the grep command is put into the output file

grep.out. When you use here documents, you will probably need to capture all the

output to a file in case errors are encountered. To do this you need to use 2>&1 as

part of the command. Here’s how to do it:

$ cat>> filetest 2>&1 <<MAYDAY

> This is my home $HOME directory

> MAYDAY

 

 

Command execution order

Using &&

 

Here’s the general format of &&:

command1 && command2

 

The operation of these commands is fairly straightforward. The command on

the left of the && (command1) must be returned true (successfully executed) if the

command on the right (command2) is to be executed; or to put it another way, ‘If

this command works’ && ‘then execute this command’.

 

$ cp justice.doc justice.bak && echo “if you are seeing this then cp was

OK”

if you are seeing this then cp was OK

 

$ mv /apps/bin /apps/dev/bin && rm -r /apps/bin

In the above example, the directory /apps/bin is to be moved to /apps/dev/bin;

if this fails then the deletion of the directory /apps/bin will not happen.

 

In the following example, a file is to be sorted with the output going to a file

called quarter.sort; if this is successful, the file will be printed.

$ sort quarter_end.txt > quarter.sorted && lp quarter.sorted

 

Using ||

 

command1 || command2

The operation of the || is slightly different. If the command on the left of the ||

(command1) fails then execute the command on the right of the || (command2); or

to put it another way, ‘If this command fails’ || ‘then execute this command’.

Here’s another simple example that illustrates the use of the ||:

$ cp wopper.txt oops.txt || echo “if you are seeing this cp failed”

cp: wopper.txt: No such file or directory

if you are seeing this cp failed

The copy has failed so now the command on the right-hand side of the || is

executed.

A more practical example is this. I want to extract the first and fifth fields

from an accounts file and stick the output into a temp file. If the extraction does not

work, I want to be mailed.

$ awk ‘{print$1,$5}’ acc.qtr >qtr.tmp || echo “Sorry the payroll

extraction didn’t work” | mail dave

 

You don’t always have to use system commands; here I run the script comet on a file

called month_end.txt. If the script bombs out, the shell should terminate (exit).

$ comet month_end.txt || exit.

 

 

Using GREP

 

Searching more than one file

If I wanted to search for the string “sort” in all my .doc files in my directory I

would do this:

$ grep “sort” * .doc

Or I could search through all the files that have the word “sort it”.

$ grep “sort it” *

 

Line matches

 

$ grep -c “48” data.f

$ 4

 

Line numbers

 

$ grep -n “48” data.f

1:48 Dec 3BC1997 LPSX 68.00 LVX2A 138

2:483 Sept 5AP1996 USP 65.00 LVX2C 189

5:484 Nov 7PL1996 CAD 49.00 PLV2C 234

6:483 May 5PA1998 USP 37.00 KVM9D 644

 

Do not match

 

$ grep -v “48” data.f

47 Oct 3ZL1998 LPSX 43.00 KVM9D 512

219 Dec 2CC1999 CAD 23.00 PLV2C 68

216 Sept 3ZL1998 USP 86.00 KVM9E 234

 

Getting an exact match

 

$ grep ’48\>’ data.f

48 Dec 3BC1997 LPSX 68.00 LVX2A 138

 

Being case-sensitive

 

$ grep -i “sept” data.f

483 Sept 5AP1996 USP 65.00 LVX2C 189

216 sept 3ZL1998 USP 86.00 KVM9E 234

 

Pattern ranges

 

$ grep ’48[34]’ data.f

483 Sept 5AP1996 USP 65.00 LVX2C 189

484 nov 7PL1996 CAD 49.00 PLV2C 234

483 may 5PA1998 USP 37.00 KVM9D 644

 

Don’t match at the beginning of a line

 

$ grep ‘^[^48]’ data.f

219 dec 2CC1999 CAD 23.00 PLV2C 68

216 sept 3ZL1998 USP 86.00 KVM9E 234

 

Trapping upper and lower cases

 

$ grep ‘[Ss]ept’ data.f

483 Sept 5AP1996 USP 65.00 LVX2C 189

216 sept 3ZL1998 USP 86.00 KVM9E 234

 

Matching any characters

 

$ grep ‘K…D’ data.f

47 Oct 3ZL1998 LPSX 43.00 KVM9D 512

483 may 5PA1998 USP 37.00 KVM9D 644

 

$ grep ‘[A-Z][A-Z]..C’ data.f

483 Sept 5AP1996 USP 65.00 LVX2C 189

219 dec 2CC1999 CAD 23.00 PLV2C 68

484 nov 7PL1996 CAD 49.00 PLV2C 234

 

Date searching

 

$ grep ‘5..199[6,8]’ data.f

483 Sept 5AP1996 USP 65.00 LVX2C 189

483 may 5PA1998 USP 37.00 KVM9D 644

 

$ grep ‘[0-9]\ {3\}[8]’ data.f

47 Oct 3ZL1998 LPSX 43.00 KVM9D 512

483 may 5PA1998 USP 37.00 KVM9D 644

216 sept 3ZL1998 USP 86.00 KVM9E 234

 

Combining ranges

 

Staying with the use of the [ ] brackets to extract information, suppose we want to

get city codes where the first character could be any number, the second character

between 0 and 5 and the third between 0 and 6. If we use this pattern we’ll get the

following output:

$ grep ‘[0-9][0-5][0-6]’ data.f

48 Dec 3BC1997 LPSX 68.00 LVX2A 138

483 Sept 5AP1996 USP 65.00 LVX2C 189

47 Oct 3ZL1998 LPSX 43.00 KVM9D 512

219 dec 2CC1999 CAD 23.00 PLV2C 68

484 nov 7PL1996 CAD 49.00 PLV2C 234

483 may 5PA1998 USP 37.00 KVM9D 644

216 sept 3ZL1998 USP 86.00 KVM9E 234

 

Well, we certainly got a lot of information back. What we want is included but we

also got other records we didn’t want. However, looking at our pattern, all records

that have been returned are the correct ones according to our rules. We need to

specify that the pattern must start at the beginning of each line. We can use the ^ for

that.

$ grep ‘^[0-9][0-5][0-6]’ data.f

216 sept 3ZL1998 USP 86.00 KVM9E 234

 

Occurrences in a pattern

 

If we want to extract any row that has a number 4 repeated at least twice, we could

use this:

$ grep ‘4\{2,\}’ data.f

483 may 5PA1998 USP 37.00 KVM9D 644

 

$ grep ‘9\{3,\}’ data.f

219 dec 2CC1999 CAD 23.00 PLV2C 68

 

If you want to search for only so many occurrences then take the comma out.

This will search for only two occurrences of the number 9.

$ grep ‘9\{2\}’ data.f

88 The grep family

 

There may be a need to match between, say, two and six occurrences of a

number or maybe a letter. This will match between two and six occurrences of the

number 8 that ends with 3:

 

$ grep ‘6\{2,6}3’ myfile

83 – no match

888883 – match

8884 – no match

88883 – match

 

Blank lines

$ grep ‘^$’ myfile

 

Matching special characters

 

If you wish to search for characters that have special meanings, like one of the

following, $ . ‘ ” * [ ] ^ ( ) | \ + ?, then you must place a \ in front of them. Suppose

you want to search for all lines that contain a period (.), you would do this:

$ grep ‘\.’ myfile

 

Searching for ip addresses

Part of my job is looking after our DNS servers, which means maintaining a lot of

ip addresses that cover different networks. Our address ip file can contain over 200

addresses. Sometimes I want to look at just, say, the ‘nnn.nnn’ network addresses,

and forget about the rest that have only two digits in the second part, i.e. nnn.nn..

To extract all these nnn.nnn. addresses, use [0-9]\{3\}\.[0-0\[3\}\. This expression is

saying any number repeated three times followed by a period, any number repeated

three times followed by a period.

$ grep ‘[0-9]\{3\}\.[0-0\[3\}\.’ Ipfile

 

Pattern matching with wildcards

Let’s take a look at the use of wildcards in using grep. Suppose we have a file like

this:

$ pg testfile

looks

likes

looker

long

Here is what is displayed when using the following grep pattern:

$ grep ‘l.*s’ testfile

looks

likes

$ grep ‘l.*k.’ testfile

looks

likes

$ grep ‘ooo*’ testfile

looks

If you want to find a word only at the end of a line, try this:

$ grep ‘device$’ *

That will search all files for lines that have the word ‘device’ at the end of each line.

Class names 91

 

All this is saying is that you have typed a filename that does not exist. If we

use the -s switch in grep we can silence these error messages.

$ grep -s “louise” /etc/password

 

Using grep on a string

grep is not only reserved for files; you can also use grep on strings. All you need to

do is echo the string then pipe it through to grep.

$ STR=”Mary Joe Peter Pauline”

$ echo $STR | grep “Mary”

Mary Joe Peter Pauline

 

 

egrep

 

egrep stands for expression or extended grep depending on who you listen

Egrep accepts the full range of regular expressions. One of the nice features of egrep is that you can store your strings in a file and pass them into egrep. We do this with the -f switch. If we create a file called grep strings and then type 484 and 47 into it:

$ pg grepstrings

484

47

$ egrep -f grepstrings data.f

this would match all records with 484 or 47 in them. The -f switch really becomes

useful if you want to match a lot of patterns, and typing them in on the command

line becomes awkward.

If we want to search for store codes 32L or 2CC we can use the bar sign (|),

which means one or the other or both (all), separated by the bar sign.

$ egrep ‘(3ZL|2CC)’ data.f

47 Oct 3ZL1998 LPSX 43.00 KVM9D 512

219 dec 2CC1999 CAD 23.00 PLV2C 68

216 sept 3ZL1998 USP 86.00 KVM9E 234

You can use as many bars as you want. If we wanted to see if users louise,

matty or pauline were logged into the system we could use the who command and

pipe the output through to egrep.

$ who | egrep (louise|matty|pauline)

louise pty8

matty tty02

pauline pty2

You can also exclude certain strings using the caret sign (^). If I wanted to see who

was on the system, but I did not want to know if users matty and pauline were on I

could do this:

$ who | egrep -v ‘^(matty|pauline)’

If you want to search a directory listing for files that were called shutdown,

shutdowns, reboot or reboots, this is easily accomplished with egrep.

$ egrep ‘(shutdown | reboot) (s)?’ *

 

USING AWK

 

$ pg grade.txt

M.Tansley 05/99 48311 Green 8 40 44

J.Lulu 06/99 48317 green 9 24 26

P.Bunny 02/99 48 Yellow 12 35 28

J.Troll 07/99 4842 Brown-3 12 26 26

L.Tansley 05/99 4712 Brown-2 12 30 28

 

 

Printing all records

$ awk ‘{print $0}’ grade.txt

 

Printing individual records

 

$ awk ‘{print $1,$4}’ grade.txt

 

Printing report headers

 

$ awk ‘BEGIN {print “Name Belt\n——————————–“}

{print $1″\t”$4}’ grade.txt

Name Belt

—————————————

M.Tansley Green

J.Lulu green

P.Bunny Yellow

J.Troll Brown-3

L.Tansley Brown-3

 

Printing report trailers

$ awk ‘BEGIN {print “Name\n——-“} {print $1} END {“end-of-report”}’

grade.txt

 

Matching

 

$ awk {if($4~/Brown/) print $0}’ grade.txt

J.Troll 07/99 4842 Brown-3 12 26 26

L.Tansley 05/99 4712 Brown-2 12 30 28

 

$ awk ‘$0 ~ /Brown/’ grade.txt

J.Troll 07/99 4842 Brown-3 12 26 26

L.Tansley 05/99 4712 Brown-2 12 30 28

 

Exact match

 

$ awk ‘{if($3~/48/) print $0}’ grade.txt

M.Tansley 05/99 48311 Green 8 40 44

J.Lulu 06/99 48317 green 9 24 26

P.Bunny 02/99 48 Yellow 12 35 28

J.Troll 07/99 4842 Brown-3 12 26 26

 

Not matched

 

$ awk ‘$0 !~ /Brown/’ grade.txt

M.Tansley 05/99 48311 Green 8 40 44

J.Lulu 06/99 48317 green 9 24 26

P.Bunny 02/99 48 Yellow 12 35 28

We could have targeted just the belt grade field ‘field-4’ and done the test this way:

$ awk ‘{if($4!~/Brown/) print $0}’ grade.txt

M.Tansley 05/99 48311 Green 8 40 44

J.Lulu 06/99 48317 green 9 24 26

P.Bunny 02/99 48 Yellow 12 35 28

 

Less than

 

$ awk ‘{if ($6 < $7) print $0 “$1 Try better at the next comp”}’

grade.txt

M.Tansley Try better at the next comp

J.Lulu Try better at the next comp

 

Any characters

$ awk ‘$1 ~/^ . . . a/’ grade.txt

M.Tansley 05/99 48311 Green 8 40 44

L.Tansley 05/99 4712 Brown-2 12 30 28

 

Match either

 

$ awk ‘$0~/(Yellow|Brown)/’ grade.txt

P.Bunny 02/99 48 Yellow 12 35 28

J.Troll 07/99 4842 Brown-3 12 26 26

L.Tansley 05/99 4712 Brown-2 12 30 28

 

AND

$ awk ‘{if ($1==”P.Bunny” && $4==”Yellow”)print $0}’ grade.txt

P.Bunny 02/99 48 Yellow 12 35 28

 

OR

$ awk ‘{if ($4==”Yellow” || $4~/Brown/) print $0}’ grade.txt

P.Bunny 02/99 48 Yellow 12 35 28

J.Troll 07/99 4842 Brown-3 12 26 26

L.Tansley 05/99 4712 Brown-2 12 30 28

 

awk built-in variables

 

ARGC The number of command-line arguments

ARGV The array of command-line arguments

ENVIRON Holds the current system environment variables in the array

FILENAME The name of the current file awk is scanning

FNR The record number in the current file

FS Sets the input field separator; same as the command-line -F option

NF Number of fields in the current record

NR The number of records read so far

OFS Output field separator

ORS Output record separator

RS Controls the record separator

 

NF, NR and FILENAME

 

$ awk ‘{print NF,NR,$0}END{print FILENAME}’ grade.txt

7 1 M.Tansley 05/99 48311 Green 8 40 44

7 2 J.Lulu 06/99 48317 green 9 24 26

7 3 P.Bunny 02/99 48 Yellow 12 35 28

7 4 J.Troll 07/99 4842 Brown-3 12 26 26

7 5 L.Tansley 05/99 4712 Brown-2 12 30 28

grade.txt

 

$ awk ‘{if (NR >0 && $4~/Brown/)print $0}’ grade.txt

J.Troll 07/99 4842 Brown-3 12 26 26

L.Tansley 05/99 4712 Brown-2 12 30 28

 

Assigning input fields to field variable names

 

$ awk ‘{name=$1;belts=$4; if(belts ~/Yellow/)print name” is belt

“belts}’ grade.txt

P.Bunny is belt Yellow

 

Comparing fields with values

 

$ awk ‘{if($6 < 27)print$0}’ grade.txt

J.Lulu 06/99 48317 green 9 24 26

J.Troll 07/99 4842 Brown-3 12 26 26

 

 

$ awk ‘BEGIN {BASELINE=”27 “}{if($6 < BASELINE)print$0}’ grade.txt

J.Lulu 06/99 48317 green 9 24 26

J.Troll 07/99 4842 Brown-3 12 26 26

 

SED COMMAND

 

[root@debasis cls]# cat deb

debasis 1

debraj  1

debajyoti 1

debasis 1

 

[root@debasis cls]# sed ‘s/debasis/deb/g’ deb

deb 1

debraj  1

debajyoti 1

deb 1

 

[root@debasis cls]# sed ‘s/[^ ]*/M&/’ deb

Mdebasis 1

Mdebraj  1

Mdebajyoti 1

Mdebasis 1

 

[root@debasis cls]# sed ‘s/[^ ]*/&M/’ deb

debasisM 1

debrajM  1

debajyotiM 1

debasisM 1

 

[root@debasis cls]# sed ‘s/[^ ]*/&M/’ deb

debasisM 1

debrajM  1

debajyotiM 1

debasisM 1

 

[root@debasis cls]# sed ‘/debasis/ s/1/2/’ deb

debasis 2

debraj  1

debajyoti 1

debasis 2

 

[root@debasis cls]# sed ‘1 s/1/2/’ deb

debasis 2

debraj  1

debajyoti 1

debasis 1

 

[root@debasis cls]# sed ‘1,2 s/1/2/’ deb

debasis 2

debraj  2

debajyoti 1

debasis 1

 

[root@debasis cls]# sed -e ‘s/debasis/deb/’ -e ‘s/debraj/deba/’ deb

deb 1

deba  1

debajyoti 1

deb 1

 

[cls@debasis cls]$ sed ‘

> /debraj/ s/1/2/

> /debajyoti/ s/1/3/’ deb

debasis 1

debraj  2

debajyoti 3

debasis 1

 

[cls@debasis cls]$ sed ‘/debraj/ d’ deb

debasis 1

debajyoti 1

debasis 1

 

[cls@debasis cls]$ sed ‘1,3 d’ deb

 

[cls@debasis cls]$ sed ‘/^deb/ d’ deb

 

[cls@debasis cls]$ sed ‘/1$/ d’ deb

 

[cls@debasis cls]$ sed ‘/debasis/ s/1/2/;/debraj/ d’ deb

debasis 2

debajyoti 1

debasis 2

 

[cls@debasis cls]$ sed ‘$a\

> stop!!!\

> I am lost’ deb

debasis 1

debraj  1

debajyoti 1

debasis 1

stop!!!

I am lost

 

[cls@debasis cls]$ sed ‘3a\

stop!!!\

I am lost’ deb

debasis 1

debraj  1

debajyoti 1

stop!!!

I am lost

debasis 1

 

[cls@debasis cls]$ sed ‘3i\

stop!!!\

I am lost’ deb

debasis 1

debraj  1

stop!!!

I am lost

debajyoti 1

debasis 1

 

[cls@debasis cls]$ sed ‘/debasis/ c\

> no use’ deb

no use

debraj  1

debajyoti 1

no use

 

 

 ENVIRONMENT AND SHELL

 

Local variables

$variable_name=value or ${variable_name = value}

 

Displaying a variable

 

$ GREAT_PICTURE=”die hard”

$ echo ${GREAT_PICTURE}

die hard

$ DOLLAR=99

$ echo ${DOLLAR}

99

$ LAST_FILE=ZLPSO.txt

$ echo ${LAST_FILE}

ZLPSO.txt

 

Clearing a variable

 

unset variable_name

 

$ PC=enterprise

$ echo ${PC}

enterprise

$ unset PC

$ echo ${PC}

 

Displaying all local shell variables

 

$ set

PWD=/root

SHELL=/bin/sh

SHLVL=1

TERM=vt100

UID=7

USER=dave

dollar=99

great_picture=die hard

last_file=ZLPSO.txt

 

Using variables to hold arguments for system commands

 

$ SOURCE=”/etc/passwd”

$ DEST=”/tmp/passwd.bak”

$ cp ${SOURCE} ${DEST}

 

Making a variable read-only

 

$ TAPE_DEV=”/dev/rmt/0n”

$ echo ${TAPE_DEV}

/dev/rmt/0n

$ readonly TAPE_DEV

$ TAPE_DEV=”/dev/rmt/1n”

sh: TAPE_DEV: read-only variable

 SHELL SCRIPTING

 

File status tests

-d This is a directory

-f This is a regular file

-L This is a symbolic link

-r This file is readable

-s This file has a size greater than zero, not empty

-w This file is writeable

-u This file has the suid bit set

-x This is executable

 

We will use both test methods to test if the file scores.txt is writeable. We will use

the last status command to test it. Remember a zero status is OK, anything else is

an error.

$ ls -l scores.txt

-rw-r–r– 1 dave admin 0 May 15 11:29 scores.txt

$ [ -w scores.txt ]

$ echo $?

0

$ test -w scores.txt

$ echo $?

0

 

cutting columns(-c)

 

 cut -c 1-20 deb [show columns from c1-c10]

cut -c1 /etc/passwd
get the first character from every line

cut -c1,5,10-20 /etc/passwd
get the first, fifth character and every character between 10 and 20

cut -d: -f2 /etc/passwd
get the second field

cut -d: -f3- /etc/passwd
get all fields from the third on

cut –f1,3- telneos > deb

cut –d’ ‘ –f2- file (delimiter as space)

cut –f1,3 telnos address>telnos.all(selecting particular columns from two files and pasting in one)

colrm 8 12 twinkle (deleting 8 and 12 column)

 

pasting columns

 

paste adc efg>>abcefg

paste –d: abc efg>>abcefg

paste telos – > telnos.new

 

using cut and paste to reorganize a file

 

cut –f1,3 telnos > temp

cut –f4- telnos>temp2

cut –f2 telos|paste temp-temp2>telnos.new

 

 

Using logical operators with tests

 

-a Logical AND, true, if both sides of the operator are true

-o Logical OR, true, if either sides of the operator can be true

! Logical NOT, true, if the condition is false

 

-rw-r–r– 1 root root 0 May 15 11:29 scores.txt

-rwxr-xr– 1 root root 0 May 15 11:49 results.txt

The following example tests whether both files are readable.

$ [ -w results.txt -a -w scores.txt ]

$ echo $?

0

 

Testing strings

 

= The two strings are equal

!= The two strings are not equal

-z This string is null

-n This string is not null

 

To test if the environment variable EDITOR is empty:

$ [ -z $EDITOR ]

$ echo $?

1

No, it isn’t. Is it set to vi?

$ [ $EDITOR = “vi” ]

$ echo $?

0

 

Testing numbers

 

-eq The two numbers are equal

-ne The two numbers are not equal

-gt The first number is greater than the second number

-lt The first number is less than the second number

-le The first number is less than or equal to the second number

-gt The first number is greater than or equal to the second number

 

$ NUMBER=130

$ [ “$NUMBER” -eq “130” ]

$ echo $?

0

 

Using expr

 

$ expr 900 + 600

1500

$ expr 30 / 3

10

$ expr 30 / 3 / 2

5

 

Incrementing counters

 

Expr does the incrementing of values when using loops. First the loop is initialized

to zero. Then one is added to the variable loop. The use of the quotes means

command substitution, which basically means take the output from the command

(expr) and put it in the variable loop.

$ LOOP=0

$ LOOP=‘expr $LOOP + 1‘

 

 

Pattern matching

 

$ VALUE=accounts.doc

$ expr $VALUE : October 8, ‘.*’

12

 

Control flow structures

 

$ pg iftest

#!/bin/sh

# iftest

# this is a comment line, all comment lines start with a #

if [ “10” -lt “12” ]

then

# yes 10 is less than 12

echo “Yes, 10 is less than 12”

fi

 

Testing values of variables

 

$ pg iftest2

#!/bin/sh

# if test2

echo -n “Enter your name :”

read NAME

# did the user just hit return ????

if [ “$NAME” = “” ] ; then

echo “You did not enter any information”

fi

 

 

 

 

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *